umr-lops / xsar

Synthetic Aperture Radar (SAR) Level-1 GRD python mapper for efficient xarray/dask based processing
https://cyclobs.ifremer.fr/static/sarwing_datarmor/xsar/
MIT License
24 stars 8 forks source link

import xsar is sometime very slow #65

Closed oarcher closed 2 years ago

oarcher commented 2 years ago

a simple import xsar can take many times (up to 8 minutes with many workers !) on datarmor. This is a possible IO issue, be we might find a solution on xsar side to reduce it.

oarcher commented 2 years ago

import time can be profiled with the following command:

python -X importtime -c 'import xsar' 2> >(grep "import time" > xsar.prof)

and the output can be parsed with tuna

image

We can see here that import xsar take ~80s, and that import xsar.ipython_backends take ~45s, because of holoviews and geoviews.

But we should detect that we are not using a notebook, and don't import those module, as xsar is able to work without them.

This will not solve the IO issue, but will probably save some import time.