observingClouds / xbitinfo

Python wrapper of BitInformation.jl to easily compress xarray datasets based on their information content
https://xbitinfo.readthedocs.io
MIT License
52 stars 21 forks source link

`import xbitinfo` too slow #189

Open ayoubft opened 1 year ago

ayoubft commented 1 year ago

Is taking 1min to import bitinfo normal?

image

How can I investigate what's causing this?

rsignell-usgs commented 1 year ago

I installed xbitinfo-python from conda-forge and it loads in about 12s. Definitely nice if you don't need the Julia stuff.

observingClouds commented 1 year ago

Importing xbitinfo when installed with e.g. conda install xbitinfo does indeed take a long time as it does a lot of Julia initialisations (https://github.com/observingClouds/xbitinfo/blob/c917b4304e26a8cc69a9eabd2cc128287f8e7cd4/xbitinfo/xbitinfo.py#L22-33). We should document these options. The pure python option is still fairly new and does not offer all options of the Julia version.

Arya-Nigam commented 3 weeks ago

you can use PyOxidizer to speed up the import of Python modules like xbitinfo. PyOxidizer is a tool that packages Python applications into a single executable, which can significantly reduce import times by bundling everything together.

`# pyoxidizer.bzl

Import the Python interpreter

from pyoxidizer import PythonInterpreter

Create a Python interpreter

python = PythonInterpreter()

# `