search

y-scope / yscope-log-viewer

A tool that can be used to view logs compressed in CLP's IR Stream format.
Apache License 2.0
9 stars 12 forks source link

new-log-viewer: Add `ClpIrDecoder`. #62

Closed junhaoliao closed 2 weeks ago

junhaoliao commented 2 weeks ago

References

new-log-viewer series: #45 #46 #48 #51 #52 #53 #54 #55 #56 #59 #60 #61

Description

  1. Integrate clp-ffi-js as a dependency.
  2. Add ClpIrDecoder.
  3. Since now decoder creations could be asynchronous, move decoder initialization out from LogFileManager's constructor into its factory function.
  4. Call buildIdx() in LogFileManager's constructor to deserialize the whole input file.

Validation performed

  1. Referred to #46 , started dev server at default address http://localhost:3010/ .
  2. Loaded example CLP Stream ("IRv1") file: http://localhost:3010/?filePath=https://yscope.s3.us-east-2.amazonaws.com/sample-logs/yarn-ubuntu-resourcemanager-ip-172-31-17-135.log.1.clp.zst
  3. Validated below functions and observed behaviour matched expectaions:
    1. When loaded without specifying the logEventNum in the app URL, the last page was loaded with the cursor set to the last event.
    2. When loaded with logEventNum in the URL, the first page is shown with the cursor set to the first event.
    3. Within the Monaco editor, tested prevPage, nextPage, firstPage, nextPage with shortcuts, and all worked as expected. (One issue was found when the page switching happens too frequently: it was observed that if nextPage is requested too frequently at a time, we could be seeing pages being loaded in order N (currentPageNum) -> N+1 -> N+2 -> N+1; initial analysis believe this was due to multiple requests getting sent to the service worker on page N which caused concurrency issues. It should be addressed in another PR where posting messages are disabled for certain requests that a response from the server and its corresponding handling completion in the render is expected before another new request can be sent.)
    4. Set page size to 1. Reloaded the app without logEventNum and observed the last page containing only the last event was loaded.
    5. Setting invalid (spaces; i.e., " ") JSONL decoder options does not affect the correct behaviours of the CLP Stream decoder.
kirkrodrigues commented 2 weeks ago

One issue was found when the page switching happens too frequently

Let's file an issue for this so we don't forget.

junhaoliao commented 2 weeks ago

One issue was found when the page switching happens too frequently

Let's file an issue for this so we don't forget.

Reported in #64

junhaoliao commented 2 weeks ago

@kirkrodrigues please trigger the merge from your end