Fine grained ability to turn recording on and off on a per request basis

Edit Subject

On first glance this seems more like a 'good to have' feature than one you would actually use.

I am marking this as 'under consideration' now.

I would like to see more votes and more specific use cases for this.

any thoughts on this? do you not agree that this is one way of reducing the amount of recording that needs to happen?

so consider this:
1.) user logs into a software and starts having a problem, calls up tech support and they guide the user to reproduce the error, when they ask the user to do that, if the system is smart enough to add a DEBUG parameter to a thread, or a web request, then it would be great to trigger the chronon recording as that would that record the actual problem that happened. this way, we can reduce the overhead of having to record on every request.

I do agree that this is not as proactive as it can be, but lowers the barrier of entry and makes it more palatable for folks to use this because of the lowered performance hit/cost.

thanks for the link. to make sure i understand correctly, the embedded version lets you start stop the recorder, from within the app, but at that point everything happening in the system is getting recorded, so in a very large system with thousands of API requests per second, this can amount to a lot of data recorded for the thousand odd requests when in fact you are interested in the data only for 2-3 requests that make up the interaction the user is having a problem with.

AM i correct in that understanding?

So this is not only a question of custom start stop and duration of recording, but is also a matter of granularity, so this is getting turned on only for a few specific web requests/execution threads etc.

Well, no matter how fine grained you try to get, you can always get into a situation where you will record millions of lines of execution.

Our goal with the Chronon Recorder is not to limit the amount you can record. The filtering and start/stop functionality is provided only for extreme purposes where performance is a huge issue.

We believe that almost every recording *will* contains a hundred million method calls and that's why we have built a special time travelling debugger to allow you to query and dissect through the code, which would be almost impossible with a traditional breakpoint based debugger or with log files.

Let me clarify, I believe we are talking about different things.

eg: Lets say I am building an API tier that serves data and computation to clients. Now one possible client is a web browser based UI.

User logs in and the UI calls a few of these APIs behind the scenes and there are exceptions/errors which of course we want to find the root cause for.

lets assume all this happened at the same instance, not unlikely for a large scale system:
User1 looks up their messages : UI calls server (http://api.foo.com/messages)
User2 looks up their messages : UI calls server (http://api.foo.com/messages)
...
User999 looks up their messages : UI calls server (http://api.foo.com/messages)
User1000 looks up their messages : UI calls server (http://api.foo.com/messages)

Each of this call to the server translates into a thread running on the server that processes that request and responds with the data.

Lets say User2 is facing issues. we would like to troubleshoot whats going on with their call to /messages.

Now since this user is running into errors, we would like to turn on the recording just for the requests that are specific to this user. Not for the thousand other users who will be using the system at the same time as this client is not the only one using this API backend.

I am interested in recording all the execution points of the request for {User2}, and only User2, however many ever classes/methods/lines of code are involved for that request or thread. It may amount to millions of lines of code, thats perfectly fine. In fact IMO it makes my point even more pertinent.

Now if I only have the capability to turn the recording on and off on a global scale which starts capturing all the code running in the JVM for that time period(even with filters for just my code) then in the case above, I am recording the data for all the other 999 users, unnecessarily and hence paying a penalty in performance, storage, server resources, etc.

Hopefully this clarifies and elaborates on what i am looking for. this is something that would IMO lower the barrier for entry because OPS and DEV folks will not have to worry about unnecessary overhead and performance issues. I am aware that you are making this system faster, but why incur a cost when you can avoid it altogether? In the above case, if we do not have the ability to turn recording on and off on a per request basis, we end up capturing 1000 times more data than what is needed to identify the issue.

Quite a few places I know and us included already have some kind of APM tools in the environment, that are already instrumenting applications to capture various runtime and performance metrics. Adding a tool that does recording in addition to that is just going to make it tougher as every level of instrumentation adds to the response time and when there are tens to potentially hundreds of millions of calls per day every few milliseconds count.

Let me know if you have any questions,

Shekar,
Our aim with Chronon is to have the recorder running on your server all the time, like your apm tool, so that you dont have to reproduce the error. In this case, you would not know in advance which user is going to get the error.

This is the same as APM tools, the point is to keep them running all the time so that when you see a performance issue you can look at the dashboard and see what happened that caused that issue. If you turn on your apm tool *after* your performance issue, it wont be of much help at all.

As for the performance overhead of Chronon itself, we have significantly improved it in Chronon 3 so you can have it running all the time:
http://eblog.chrononsystems.com/chron...

If you are still concerned with the performance overhead, I would highly recommend trying out the beta and giving us some feedback. We would love to hear from you and we will definitely take actions if the performance does not match your expectations.