search

SciML / SciMLSensitivity.jl

A component of the DiffEq ecosystem for enabling sensitivity analysis for scientific machine learning (SciML). Optimize-then-discretize, discretize-then-optimize, adjoint methods, and more for ODEs, SDEs, DDEs, DAEs, etc.
https://docs.sciml.ai/SciMLSensitivity/stable/
Other
329 stars 71 forks source link

importing SciMLSensitivity on mybinder with julia 1.8.1 kills julia #734

Closed schlichtanders closed 1 year ago

schlichtanders commented 1 year ago

Locally everything worked, but going to mybinder I get an immediate kill if I import SciMLSensitivity

you can reproduce this by

image

SciMLSensitivty version 7.10.0

schlichtanders commented 1 year ago

I just tested the docker locally via jupyter-repo2docker https://github.com/jolin-io/KI2022-tutorial-universal-differential-equations and it works.

The memory usage peaked at 2.05 GB, which is indeed more than the memory limit available on mybinder docker in the cloud. (sometimes you get 8 GB but most often 2 GB).

It would be really great to be able to use SciMLSensitivity within mybinder. Is it possible to massively reduce the memory footprint during import?

ChrisRackauckas commented 1 year ago

Yeah, I'm not sure what to do here. It would need to get elevated to Julialang/julia since packages don't have control over the memory usage in the loading process. In general it should be made to GC appropriately, and maybe cut back on cores if on a smaller memory machine (since the multithreading makes the concurrent memory usage increase).

But anyways, those are discussions for Base Julia since this will happen with any sufficiently large package. Let me know when you open it since I'll want to follow it.

schlichtanders commented 1 year ago

opened: https://github.com/JuliaLang/julia/issues/46912