Consider implementing searching through terminal I/O stored in journal

[sabbaka]

I've worked on this issue, have some ideas now.

[spbnick]

Could you just dump whatever you have so far here?

[sabbaka]

I'll add this to design in Google Docs also. I think it might good to list input commands for search with like position related to player. But, search input will search through everything, but showing command entered and as a drop-down output of that command and after this it will be possible to go to the same place in player + correlated logs will be shown also. That's soft of my idea for now. The only problem for now it's protocol, cause there are a lot of different special characters and I want to search for the begging only for input commands.

[sabbaka]

So, right now it's something like this, just the whole output from session. So, my idea is to group all of this by items. One item is one command. Command name will be a name of a "bar" and when this "bar" will be clicked it will hover down and show output of command.

[sabbaka]

I was trying to split string and somehow differ input commands from output, but I have not succeed with this idea. I've tried to split by ] symbol. My idea is to cut off all special characters and then make search upon everything. In JSON there are fields out_txt and in_txt, but everything is in out_txt, will it be like this or maybe entered command will be provided in in_txt - here I'm not sure, of course it's related to how terminals work.

[spbnick]

We don't need to locate commands in the text output. We'll be extracting them from the associated audit log parts (audit log messages having the session ID of recording). The commands can wait for later. For now we can just search through everything.

It would be best if we could avoid loading all the recording log messages into the browser for searching, and instead let journalctl search them, if possible, or perhaps some program working on top of journalctl.

Also, searching mostly makes sense only if we support rewinding to the time of a matched message (e.g. by clicking on a match), but that might be left for when we implement our own browser-side playback, otherwise we'll need to implement random positioning in tlog-play, and that's far from easy.

[sabbaka]

I've one more idea for filters on page with list of sessions - there could be field to search in recordings.

[sabbaka]

Searching could be implemented now with starting tlog-play with specific journal entry.

[sabbaka]

Is it possible to launch tlog-play for specific journal entry? Probably not, it should go from start to the end, right? As it says in documentation.

[spbnick]

You can provide it with matches that match a specific entry, but at the moment it will only play a recording from the start.

Why would you need that, BTW? What's your idea?

On Aug 21, 2017 2:20 PM, "Kyrylo Gliebov" notifications@github.com wrote:

Is it possible to launch tlog-play for specific journal entry?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Scribery/cockpit/issues/17#issuecomment-323730005, or mute the thread https://github.com/notifications/unsubscribe-auth/AAhhfgevFmMnfojbebggw_PTF5elDvDXks5saXYhgaJpZM4OoGgl .

[sabbaka]

Idea is to show list of entries from a journal ( table ) and make search through out_txt and then on click on row launch tlog-play from this journal entry - but that is what you mean by random position.

[spbnick]

I think the right way here would be to implement random positioning, or at least fast-forward to specific position, and then send that position as a series of "keypresses" through the terminal. Still, this is an advanced feature and perhaps better done with a JS-based player.

[sabbaka]

Yes, agree, I'll skip this for now, we've discussed this before, but I decided to take a look once again. I'm working on logs correlation now.

[sabbaka]

Create a script to grab only IDs from journalctl which match provided string in out_txt.

[spbnick]

Alright, this quick-and-dirty python script can filter Journal entries in JSON format based on matching tlog's out_txt field with a string:

#!/usr/bin/env python

import sys
import json

if len(sys.argv) != 2:
    sys.stderr.write("Invalid number of arguments\n"
                     "Usage: filter STRING_TO_MATCH\n")
    sys.exit(1)

for line in sys.stdin:
    if json.loads(json.loads(line)["MESSAGE"])["out_txt"]. \
                    find(sys.argv[1]) >= 0:
        sys.stdout.write(line)

It can be used like this:

journalctl TLOG_REC=e56b648b6811409c8b21ad8c3cd36dea-6c0b-326687c -o json | ./filter nkondras

The output will be journal entries that matched. This will not be very fast, but better than doing this in the browser.

From this we can build a script which could output matching recording IDs, one per line, or perhaps recording IDs followed by timestamps of matched messages.

Therefore I declare that we'll be able to search terminal I/O as required, albeit with low performance.