timholy / SnoopCompile.jl

Provide insights about latency (TTFX) for Julia packages
https://timholy.github.io/SnoopCompile.jl/dev/
Other
309 stars 48 forks source link

Allow to SnoopPrecompile to dump the precompilation directives in a file #312

Open ronisbr opened 1 year ago

ronisbr commented 1 year ago

Hi!

I would like to use SnoopPrecompile in a package that perform a lot activity using a database through ODBC.jl. However, I cannot use @precompile_all_calls because it will require a working database connection every time the user loads the package.

Is it possible to allow SnoopPrecompile to dump all precompilation statements into a file (just like we usually did before) so that I can generate them from a controlled environment before tagging a new package version?

timholy commented 1 year ago

That is possible, though understand it also causes all the old problems: internal changes in Julia can make the specific precompiles break. For example, Julia 1.9 eliminates kwfuncs in favor of a generic Core.kwcall; any precompile directives generated on Julia 1.8 that referenced the kwfunc would break the package on 1.9 (as in, you'd get an error just from using MyPkg). For that reason, I'd really like to get away from automatic dumps of long lists of MethodInstances.

FYI the precompile workload doesn't get executed when you load the package: *.jl files are basically build scripts, and they only get executed at build time. So as long as you had database connection when the package was being precompiled, it should work. You can also use try/catch wrappers to ensure that lack of connectivity during precompilation doesn't trigger an error.

Does that solve your issue?

ronisbr commented 1 year ago

Thanks @timholy !

Does that solve your issue?

Unfortunately not. The database I am connecting is a telemetry database in an internal network. Most of the time I will not have access to it. Hence, any dependency update will trigger a recompilation of the package and it will fail. The try/catch can improve a little bit, but then I will not have the performance gain until I recompile with the connection, right?

Furthermore, the connection to the database is protected by a password and I do not want to write it down at the precompilation statements, even if the package is only used inside my institute.

I understand the old problems related to Julia versioning. In my case, it would be better to handle those different versions than any other work arround. But I fully understand if you really do not want to support it since it can lead to those problems. I see that my problem seems very particular. Thus, feel free to close this issue :)

timholy commented 1 year ago

I'll leave this open since I see this might be important for you. If you want to prepare a PR yourself, notice that SnoopPrecompile is almost trivial, and that verbose[] = true almost does what you want.

ronisbr commented 1 year ago

Sure! I will try :) Thanks!