h5py dependency not listed in setup.py

[kjurka]

After installing sarpy into a bare virtualenv, trying to open a SICD file using sarpy.io.complex.open(fname) fails with:

reader = sarpy.io.complex.open(input_file) File "xxx/venv/lib/python3.6/site-packages/sarpy/io/complex/init.py", line 70, in open modules = [sys.modules[names] for names in module_names if import(names)] File "xxx/venv/lib/python3.6/site-packages/sarpy/io/complex/init.py", line 70, in modules = [sys.modules[names] for names in module_names if import(names)] File "xxx/venv/lib/python3.6/site-packages/sarpy/io/complex/csm.py", line 20, in import h5py ModuleNotFoundError: No module named 'h5py'

So csm wants h5py and even though I'm trying to read a SICD, this is tripping me up. Reasonable solutions are:

1) Just list h5py as a required dependency. 2) try/catch around reader module setup and ignore readers that we don't have dependencies for. 3) For CSM only import h5py after checking that the input is a H5 file by examining it and checking its magic number.

[utwade]

I vote for #2.

I don't think we want #1, since we want most modules of the the code to still run, even if that library isn't available. A user may not need the CSM module, but still may want to use the SICD module.

2 would also be a bit more generic than #3 since it would conceivably work for other modules with similar issues as well.

I'll try to make the change.

[kjurka]

I looked at this a little bit and unfortunately my original test with a virtualenv was not bare enough. With a truly empty (aside from numpy), I get:

File "test-sicd.py", line 3, in reader = sarpy.io.complex.open('/home/kjurka/sample-sicds/sicd_example_1_PFA_RE32F_IM32F_HH.nitf') File "/home/kjurka/git/kjurka/sarpy/sarpy/io/complex/init.py", line 70, in open modules = [sys.modules[names] for names in module_names if import(names)] File "/home/kjurka/git/kjurka/sarpy/sarpy/io/complex/init.py", line 70, in modules = [sys.modules[names] for names in module_names if import(names)] File "/home/kjurka/git/kjurka/sarpy/sarpy/io/complex/csm.py", line 6, in from .sicd import MetaNode File "/home/kjurka/git/kjurka/sarpy/sarpy/io/complex/sicd.py", line 10, in from ...geometry import point_projection as point File "/home/kjurka/git/kjurka/sarpy/sarpy/geometry/point_projection.py", line 15, in from sarpy.io.DEM import DEM as DEM File "/home/kjurka/git/kjurka/sarpy/sarpy/io/DEM/DEM.py", line 12, in from scipy.interpolate import interpn ModuleNotFoundError: No module named 'scipy'

So there is already a dependency on scipy that is not explicitly stated, so even SICD reading doesn't work. Looking at sicd.py lines 1304 and 2176, it's already trying to avoid a hard dependency on scipy, but that hasn't been carried over to the DEM code. So is scipy a hard or optional dependency?

I've also found some documentation on how to declare optional dependencies in setuptools, which should be used for CSM/h5py:

https://setuptools.readthedocs.io/en/latest/setuptools.html#declaring-extras-optional-features-with-their-own-dependencies

[utwade]

Sorry for the delayed response. Out on spring break this week. You are right on all of that. Those should be declared as conditional dependencies. I didn't realize that DEM commit added dependencies to the SICD portion. There are some other things from that DEM commit that I want to clean up soon as well. Let me see if I can get that and the setup.py stuff out next week.

[utwade]

I went ahead and made scipy a dependency for the entire thing. In general, we have tried to avoid scipy dependencies whereever possible, and its not required for most things. However, it has enough dependencies scatterred throughout that its too messy to add a whole bunch of different extras.

I also add a CSK extra (also changed the acronym to be more consistent with what's internationally accepted) in setup.py that allows you specify if want the dependencies for that extra functionality (h5py). The file format checker now does have a try/catch so it will run just fine, even if you don't have the CSK dependencies.

Let me know if there are any problems with this.