Open perlman opened 6 years ago
Hi Eric,
Glad you find the package useful!
The namespace issue has been in a thorn in my side for a while. I would love to have the flat namespace (i.e. importing modules in pymaid/init.py to make them available at the package level) while also being able to import modules individually. AFAIK there is no way to have the cake and eat it too but if you have a suggestion, please let me know. Currently, I think the flat namespace is more important for most users.
There are a few things I can try to help though:
If this is urgent, you could also fork the repository and simply delete the contents of pymaid/init.py. This would allow you to skip a few submodules when importing pymaid.fetch
(will still import some dependencies from other pymaid modules though). I just ran a quick test:
With __init__.py
as is:
zoopc651:~ philipps$ time python3 -c "from pymaid import fetch"
real 0m2.208s
user 0m1.783s
sys 0m0.403s
zoopc651:~ philipps$
With empty __init__.py
:
zoopc651:~ philipps$ time python3 -c "from pymaid import fetch"
real 0m1.170s
user 0m0.886s
sys 0m0.267s
So this seems to cut the import time almost in half. Maybe I can squeeze out a bit more by improving imports. I'll keep you posted.
I see nearly identical import time
__init__.py
as is:
real 0m2.770s
user 0m2.497s
sys 0m0.461s
With empty __init__.py
:
real 0m1.217s
user 0m1.169s
sys 0m0.259s
The times were a lot worse due to an issue with matplotlib and my font cache, but I seem to have that resolved.
All of your suggestions sound good. I have not dug in enough to figure out which bit of code triggers the "app launch" (OS X). It has to be more than a simple import vispy
-- maybe a viewer is being initialized at import in one of the modules?
Nice.
Re the GUI being initialised: The only vispy-related things that happen at import time are these (across two different modules):
import vispy
from vispy import scene
from vispy.geometry import create_sphere
from vispy.gloo.util import _screenshot
vispy.use(app='PyQt5')
Does any of this spawn a GUI on your system?
None of those do it.
But import pymaid
does.
Hmm.
Maybe it's matplotlib then? Try these import statements:
import matplotlib as mpl
import matplotlib.pyplot as plt
import matplotlib.lines as mlines
import matplotlib.patches as mpatches
import mpl_toolkits
from mpl_toolkits.mplot3d.art3d import Line3DCollection
from mpl_toolkits.mplot3d import proj3d
from matplotlib.collections import LineCollection
import matplotlib.colors as mcl
Yes, it's matplotlib. I feel bad for trying to place the blame on VisPy.
Specifically, it's ``matplotlib.pyplot'', and is where 0m0.647s of the startup time goes. :-)
In that case this should do the trick for you: If you don't need plotting simply import matplotlib in headless mode before importing pymaid:
import matplotlib as mpl
mpl.use('template')
import pymaid
To change the default backend permanently, have a look at your matplotlibrc file.
Thanks! While the startup is still a bit slow, I'm quite content to no longer see the gui launcher. 👍
I tried optimizing imports in c7cd8478c2eb88dd478c82fa37819c81694237b3. Unfortunately, there wasn't much to optimize in the first place, so I think any difference will be hardly noticeable.
There are two more options:
pymaid.fetch
by creating a "buffer" module of sorts that takes care of higher functions (e.g. the conversion to CatmaidNeuron/List
) so that fetch
does not have any other pymaid-internal imports. For reference: non-pymaid imports in fetch
take 2.2s
on my machine, all of pymaid takes 2.3s
. Removal of networkx
brings fetch
down to 1s
.nat
and rcatmaid
: one for general neuroanatomy and one for Catmaid interfaceThe first one option would obviously be easier in the short term, the other one might (a) make the package more interesting for a general audience and (b) be easier to maintain in the long run.
I'll have a think....
Came across lazy-import package. Might be worth using this for e.g. vispy and matplotlib
Edit: demand-import also looks good
Thanks for this great library for interfacing with CATMAID.
It would be great if I could include a single module (e.g. pymaid.fetch) without the other modules. This is currently not possible due to pymaid/__init__.py automatically importing all of the submodules.
The full set of modules can take a while to load on my machine, and the initialization of VisPy leads to the spawning of a GUI application even if no visualization routine is called.