Importing ecephys_project_cache is slow

[ahwillia]

Importing ecephys_project_cache is very slow -- over 10 seconds, as shown below. Is this typical or just me? I would like to use this in a package I'm developing, but this is adding noticeable friction that I'd rather avoid.

This issue seems specific to that submodule. When I run import allensdk or even import allensdk.brain_observatory.ecephys it loads within a second.

Python 3.9.9 (main, Jan 13 2022, 00:56:30) 
Type 'copyright', 'credits' or 'license' for more information
IPython 8.4.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: %time import allensdk.brain_observatory.ecephys.ecephys_project_cache
CPU times: user 9.37 s, sys: 3.41 s, total: 12.8 s
Wall time: 11.2 s

[ahwillia]

Digging a bit deeper it seems like importing ecephys_session_api is the minimal way to reproduce this behavior as shown below.

Python 3.9.9 (main, Jan 13 2022, 00:56:30) 
Type 'copyright', 'credits' or 'license' for more information
IPython 8.4.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: %time import allensdk.brain_observatory.ecephys.ecephys_session_api.ecephys_session_api
CPU times: user 9.18 s, sys: 3.24 s, total: 12.4 s
Wall time: 10.3 s

However, I'm also surprised to see that simply import pynwb is pretty slow. It accounts for at least some of this.

Python 3.9.9 (main, Jan 13 2022, 00:56:30) 
Type 'copyright', 'credits' or 'license' for more information
IPython 8.4.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: %time import pynwb
CPU times: user 2.77 s, sys: 1.61 s, total: 4.38 s
Wall time: 3.26 s

[aamster]

Hi @ahwillia thanks for pointing this out. It doesn't take quite as long on my laptop to import this module. I think the amount of time to import all the libraries is just the cost of doing business with a lot of external dependencies.

[ahwillia]

@aamster can I ask how long it takes you to load and what versions you are using? If there is a faster configuration, that would be ideal for my workflow.

[aamster]

It takes about 6 seconds and I am using the most recently released version of allensdk on pypi

Python 3.8.0 (default, Nov  6 2019, 15:49:01) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.30.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: %time import allensdk.brain_observatory.ecephys.ecephys_project_cache
CPU times: user 6.59 s, sys: 516 ms, total: 7.11 s
Wall time: 6.43 s

In [2]: allensdk.__version__
Out[2]: '2.13.4'