new-log-viewer: Implement log-level filtering.

[davemarco]

References

new-log-viewer series: #45 #46 #48 #51 #52 #53 #54 #55 #56 #59 #60 #61 #62

Description

• Implemented buildIdx() to do deserialization in JSONL decoder. Previously, deserialization was done inside constructor. In addition, now we are parsing log level inside in buildIdx() and not in decode().
• Implement log level filtering with JSON logs.
• TODO: CLP IRv1 log-level filtering is to be added once the equivalent is implemented in clp-ffi-js. The implementation in clp-ffi-js will start once we agree with the decoder handling in this PR.

Validation performed

1. Referred to #46 , launched debug server.
2. Referred to #54, tested page switching with Monaco editor with filters off.
3. Referred to #54, tested page switching with single (TRACE) log level filter.
4. Referred to #54, tested page switching with multiple (TRACE, DEBUG) log level filter.

[davemarco]

Need to handle case where decoder options can change

[davemarco]

Despite the fact that not necessary to update PR since the the page is always refreshed (i.e. logs are rebuilt) when decoder option change. I added a guard to reparse logs when decoder options change in case option design changes in future.

[junhaoliao]

@davemarco I haven't been able to take a careful look into the code yet, but it seems there's one issue whenever I was switching the log level filter from ["TRACE"] to ["DEBUG"] - there are exactly 200 [Renderer -> MainWorker] code=loadPage requests sent to the service worker, and occasionally the logEventNum does not update on editor cursor updates. Could you take a look before we proceed to the review?

Here's an example file I used - example.zip

pageSize was set to 10 and I started from http://localhost:3010/?filePath=http://localhost:3010/test/example.jsonl#logEventNum=10

[davemarco]

I believe there is something buggy with the log event num changing retriggering requests. I am looking into

[davemarco]

I committed changes which appear to fix the issue. Changing the log filter was actually sending out 2 load page requests and they were fighting for control over the logEventNum, which then sent out more requests. There was also a synchonization issue with some of firstLogEventNumPerPage: number[], lastLogEventNumPerPage: number[], states. I believe it works for now.

[davemarco]

There are some issues with page num now that merged. I am looking into solution.

[davemarco]

Added temporary select element for testing.

[davemarco]

Clean up select element and make more user friendly.

[davemarco]

FIxed bugs when changing page. Previous implementation did not consider filtered events.

[davemarco]

I changed to the new interface. I also went through and started fixing a bunch of small bugs. There are a decent amount of changes, but some are just the linter.

[davemarco]

Another thing I want to do, but not in code is remove the filepath useEffect, and do something like this

  const [prevFilePath, setprevFilePath] = useState(filePath);
  if (filePath !== prevFilePath) {
    setprevFilePath(filePath);
    loadFile(filePath, cursor);
  }

The current implementation is taking advantage of useRef on logEventNum, to avoid having logEventNum as a dependency. If it were a dependency, it would reload file of every logEventNum change. I think the code above is better as it dosent need useRef on logEventNum, and gets us one step closer to removing it from code and just using state

May also be possible to keep useEffect, but also have prevFilePath state, and just exit the effect if states are the same.

[kirkrodrigues]

Thanks for the additional comments, @junhaoliao. Since this PR ended up being quite large and we are reworking the design, @davemarco is going to submit smaller PRs rather than trying to refactor this one completely.

[junhaoliao]

@kirkrodrigues @davemarco i think we can close this now, right?

[junhaoliao]

Closing this since it has been implemented in the other PRs.