timholy / SnoopCompile.jl

Provide insights about latency (TTFX) for Julia packages
https://timholy.github.io/SnoopCompile.jl/dev/
Other
309 stars 48 forks source link

Support for cross-environment (de-)serialization of records and offline analysis #385

Closed kimikage closed 3 months ago

kimikage commented 5 months ago

We have greatly improved the dependency problem by letting SnoopCompileCore handle the recording phase. Also, I am aware that SnoopCompile is currently working on reducing hard dependencies.

However, I think that cross-environment analysis is not yet fully supported (due to the dependency of deserialization on the environment or world). An example of a specific problem is: https://github.com/julia-actions/julia-invalidations/issues/17

I cannot estimate how technically difficult this would be, nor do I personally think it is important. As the internal data structures of julia change, I believe that compatible serialization is also very costly to maintain. However, supporting this is not generally a bad approach in the long run, since it means that the analysis can always be done with the latest (or more stable than nightly) julia.

timholy commented 3 months ago

I've thought about this, but I think it's well-nigh impossible. In a certain sense, it's moved away from a serialization format and more towards direct usage of internal compiler structures. The linkages between these structures are enormously valuable and we'd have to duplicate everything to restore functionality. Given that I think https://github.com/julia-actions/julia-invalidations/issues/17 has a better solution (use SnoopCompileCore to collect the data, and then after all relevant invalidations are collected load SnoopCompile itself), I'll close.