search

CalebBell / fluids

Fluid dynamics component of Chemical Engineering Design Library (ChEDL)
MIT License
356 stars 74 forks source link

IPython dependency with Numba #55

Closed dbrattli closed 1 year ago

dbrattli commented 1 year ago

When using fluids.numba it will set the file path of the function in numba_exec_cacheable to "<ipython-%s>" which will make Numba fallback to _IPythonCacheLocator which will again import from IPython.paths import get_ipython_cache_dir.

It feels that having the IPython dependency for caching is too big of a hammer, since any library that depends on fluids now need to take a direct dependency on ipython as well. Is there a way to avoid the IPython dependency and still use Fluids with Numba?

CalebBell commented 1 year ago

Hi, I didn't realize that bit of code created a dependency on IPython when using fluids.numba. That bit of code is the only way to get the caching feature to work with numba at this time. In my opinion numba is an incredibly heavy dependency compared to IPython, and the value of the speed of cached compilation is well worth it. Numba is slow to compile many functions. I can take another look at this if numba ever adds another way of caching dynamically compiled python code. Caleb

dbrattli commented 1 year ago

Thanks for the quick answer. My comment about "heavy dependency" was more concerning the number of dependencies needed. Adding ipython will also add appnope, asttokens, backcall, decorator, executing, jedi, matplotlib-inline, parso, pexpect, pickleshare, prompt-toolkit, ptyprocess, pure-eval, pygments, stack-data, traitlets, and wcwidth as transient dependencies. But I understand that it may not be easy to fix from fluids alone and that there needs to be another way of caching in Numba for this to work.

CalebBell commented 1 year ago

Hi, I've pushed the latest version of fluids, 1.0.23, and I believe it has support for running numba without IPython. I didn't actually test without IPython, and I can't really guarantee the feature will continue to work. For today it seemed achievable. This feature can be activated by setting the environmental variable NUMBA_FUNCTION_CACHE_SIZE to 0, before loading fluids.

Generally it has been difficult lately to keep numba running at all due to all the changes in CPython.

Sincerely, Caleb