grafana / jfr-parser

Java Flight Recorder parser library written in Go.
Apache License 2.0
43 stars 16 forks source link

improve memory allocation during parsing #25

Closed korniltsev closed 1 year ago

korniltsev commented 1 year ago

This PR is a complete rewrite of the parser to improve memory allocations.



Future improvements:
- batch allocate stackframes to reduce number of allocations
- some of the fields were skipped to save memory(for example line numbers) as they are currently not used. we may want to improve and skip them with a runtime option
- readin metadata currently allocates a lot of objects with temporary maps, we could reduce it, by using slices and reusing slices
- when field is skipped in the generated code we can avoid computng the result skipped value with shifts and ors. 
cyriltovena commented 1 year ago

I think we should move this code into pyroscope. Specially if in the future we only want to stream and output pprof. This will make it easier to avoid allocation

some of the fields were skipped to save memory(for example line numbers) as they are currently not used. we may want to improve and skip them with a runtime option

We should pass the line number in pprof if we can, this is supported by the database. Does it also have filename ?

zdyj3170101136 commented 1 year ago

very big merge request