Timeline Tracking

[Hajto]

Added:

• epl_timeline_observer.erl - genserver that hosts timeline changes and state changes

Edited:

• epl_tracer - two new ets tables, one for storing pids that are tracked and one for caching changes

To test

{ok, TS} = epl_timeline_observer:start_link(list_to_pid("<13611.385.0>")).
%% will yield timeline so far
epl_timeline_observer:timeline(TS).

[Hajto]

[michalslaski]

Thank you very much for contributing! I will review this PR asap, hopefully tonight.

BTW, how do you like lectures on discrete mathematics? ;)

[baransu]

I know we had conversation about this plugin but I dont really remember when and how should we display time travel debug info in our UI.

[Hajto]

About UI. Ideal way would be a separate pane with list of tracked timelines, and input to type in the pid manually. (That's partially why pids are compared as lists).

Very nice thing would be also a track timeline button in one of the other views in process details. The click would start trackign and move user to timeline view.

Yet another option would be a pane with all the pids in a filterable lists, user could tick interesting pids of and see their timeline on the other subpane. (left half pane list, right half timeline)

At least that's how I would imagine it.

[baransu]

I'll try to add think about something for it. Is it possible to have it as separate plugin? It can be good starting point for plugin system for frontend as well because I'm thinking about moving all core plugins to one "core/official" plugins monorepo at some point and we need plugin system for that.

[Hajto]

The problem with making it Plugin is fact that I had to severely modify already existing tracer. (At stock it gives only information about amount of messages between processes)

[michalslaski]

Hi @Hajto, I was thinking how we could visualise the trace information you gather and one idea is to render so-called flame graphs. It could be rendered in the 3rd level of Vizceral, when one double-clicks a process. In other words can you please review your PR and see if you gather enough information to render a flame graph? Such graphs for Erlang processes are already implemented by @proger in his eflame.

[baransu]

I've added epl_timeline_EPL plugin serving state over WebSocket's timeline-info topic. I'll be working on UI part of that.

/cc @michalslaski @arkgil I would really appreciate your review on my Erlang code 🙂

[baransu]

You can checkout this version. Basic functionality is done, only some UI improvements and of course tracker changes if required :)

/ cc @Hajto

[baransu]

This should be pretty close to what we want. You can add new pid from /sup-tree (the same icon as /timeline plugin) or from input on /timeline view.

Waiting for your review @michalslaski @arkgil @Hajto

[michalslaski]

@Baransu and @Hajto thanks a lot for your contributions to this PR. I will review asap, which hopefully means this weekend.

[baransu]

@michalslaski Thank you a lot. Waiting for your review then 🙂

[arkgil]

Unfortunately this doesn't work for me. I add the PID from the supervision tree view, send some messages and nothing shows up 😢

[baransu]

Do you have any UI or Erlang logs?

[arkgil]

Woops, it showed up after refreshing 😄

[baransu]

After refreshing what? UI? Was it cached or UI didn't receive/proceed some websocket message?

[arkgil]

I have no idea, but everything seems to show up fine now. Although there might be problems with ordering of messages, let e investigate.

[baransu]

I was mainly testing on cowboy gen_server which updates time and all messages were in correct order.

[arkgil]

Yes, the order is correct, it's just we're too slow with retrieving process' state. When we do sys:get_state some other messages might have been already received, and state has been further changed. But I see no way to solve that problem.

[baransu]

So we're retrieving messages from past with some lag but we're capturing all messages or some are missing?

[arkgil]

One more thing - PID input is not working right now. When we type in some PID from the node we observe, say <0.93.0>, its representation is different from erlangpl node POV, e.g. <9932.93.0>. What we could do is somehow retrieve node part of the PID (9932 here) and display it as a label to the input, so that the user only needs to fill out two other parts of PID. I imagine something like:

<9932..>

This will also make it easy for new users to fill in PIDs - I had no idea what format should I use to make it work (😅 ). I'll look into how we could get the node prefix of the PID.

[arkgil]

Hm, we could probably retrieve the PID of a tracer process (since it's started on observer node) and parse it to get node prefix.

[arkgil]

And to answer your question - yes, we retrieve them all, but we can't capture changing state step by step if messages come too fast. Because messages can come to process and change its state after we receive trace message but before we make sys:get_state call.

[arkgil]

One more question - is it possible to show whole message received by the process? I can't find a way to extend the column with messages.

[baransu]

No right now. I'll add this message in state's header because side panel has limited space for that.

[baransu]

I've moved epl_timeline_observer.erl logic into epl_timeline.erl so it's more consistent. I'll try to retrieve observed node pid to change input method as well as full message text 🙂

[baransu]

@arkgil I'm retrieving node part of pid from kernel's application_master on observed node and then using it during the add WS message. I've also split input into two separate parts for second and third part of the pid. Waiting for your feedback 😊

[michalslaski]

@Baransu I tried the new view and I like it very much. The UX of the timeline feature is intuitive and easy to figure out.

@Hajto I tried tracing the mnesia_locker process and noticed that throughput of messages reported in the dashboard view increased to over 45,000 msg. The system is not processing any Mnesia transactions, so most of the internal traffic is between epl_tracer and mnesia_locker processes. Is it expected?

[Hajto]

@michalslaski I am sorry for inactivity. It is not expected to ignore those messages. I will have to investigate it more.

[michalslaski]

No worries, good to have you back :)

[baransu]

I've added resizable panes with state and messages so when messages are to long or you want to focus on state better it's now easier 🎉

@Hajto what is progress on issues with to many messages?

[baransu]

@michalslaski @mkacper I've fixed mailbox arrow navigation so right now moving through large list of messages by up/down arrows should super smooth ☺️. Waiting for your review. Let me know if you have any concerns or ideas for improvements.

[mkacper]

@Baransu moving through mailbox by up/down arrows works perfectly but if you want to see details of a message -> state pair you have to click on a particular message. Maybe it would be easier to use when you can hit enter on a highlighted message to see the details or we can just automaticly switch the detailed view when moving through the mailbox? What do you think?

[baransu]

It should automatically select message on which you're on. Did you run yarn (or any make rule including it) after pulling?

[mkacper]

First I used only yarn start but after your comment yarn && yarn start and now it works as you said. Sorry for the confusion :) Well done! :+1:

[baransu]

@mkacper It was because of old version of react-virtualized (very effective lib for large lists, grids etc) we're using for displaying mailbox 🙂