search

cedadev / nappy

NASA Ames Processing in PYthon (NAPPy) - a Python library for reading, writing and converting NASA Ames files.
BSD 3-Clause "New" or "Revised" License
9 stars 13 forks source link

fix test dependencies #1

Closed tomkralidis closed 3 years ago

tomkralidis commented 6 years ago

Running python setup.py test yields 6 errors all of type:

Exception: Could not import third-party software. Nappy requires the CDMS and Numeric packages to be installed to convert to CDMS and NetCDF.

Scanning the code it seems the following packages are required:

I don't see cdms2, cdms or Numeric on PyPI, are they elsewhere? Perhaps we can update the tests so that the base tests that are run are those that do not require NetCDF support (numpy/cdms2 or cdms/Numeric).

agstephens commented 6 years ago

Thanks for looking at this. A few comments:

  1. We should definitely remove references to ("Numeric")
  2. The cdms2 library is currently integral to the netCDF conversion code and would probably take considerable time to disentangle. It can be installed from the cdat-lite package (https://pypi.python.org/pypi/cdat-lite/). (I appreciate that it would be better to use a more modern package here).
tomkralidis commented 6 years ago

  1. I see Numeric in the following areas, but unsure how to handle them as the package is used in the codebase:

    $ grep -R Numeric *
doc/nasa_ames_format_specification_1998.html:       records.  Numeric constants in the file headers have been
lib/nappy/nc_interface/na_to_nc.py:        raise Exception("Could not import third-party software. Nappy requires the CDMS and Numeric packages to be installed to convert to CDMS and NetCDF.")
lib/nappy/nc_interface/cdms_to_na.py:        import Numeric as N
lib/nappy/nc_interface/cdms_to_na.py:        raise Exception("Could not import third-party software. Nappy requires the CDMS and Numeric packages to be installed to convert to CDMS and NetCDF.")
lib/nappy/nc_interface/cdms_to_na.py:        Loops through variables to convert singleton variables (i.e. Masked Arrays/Numeric Arrays)
lib/nappy/nc_interface/cdms_objs_to_na_file.py:        raise Exception("Could not import third-party software. Nappy requires the CDMS and Numeric packages to be installed to convert to CDMS and NetCDF.")
lib/nappy/nc_interface/nc_to_na.py:        raise Exception("Could not import third-party software. Nappy requires the CDMS and Numeric packages to be installed to convert to CDMS and NetCDF.")
lib/nappy/nc_interface/na_content_collector.py:        import Numeric as N
lib/nappy/nc_interface/na_content_collector.py:        raise Exception("Could not import third-party software. Nappy requires the CDMS and Numeric packages to be installed to convert to CDMS and NetCDF.")
lib/nappy/nc_interface/na_to_cdms.py:        import Numeric as N
lib/nappy/nc_interface/na_to_cdms.py:        raise Exception("Could not import third-party software. Nappy requires the CDMS and Numeric packages to be installed to convert to CDMS and NetCDF.")
lib/nappy/contrib/aircraft/aircraftData.py:import MA, Numeric, cdms, MV, cdtime
lib/nappy/contrib/aircraft/aircraftData.py:            newArray=Numeric.zeros(shape[0], 'f')
lib/nappy/contrib/aircraft/aircraftData.py:            if type(value) in (type((1,1)), type([1,2]), type(Numeric.array([1.]))) and len(value)==1:
lib/nappy/contrib/aircraft/aircraft_utils.py:import Numeric
lib/nappy/contrib/aircraft/aircraft_utils.py:        if type(value) in (type((1,1)), type([1,2]), type(Numeric.array([1.]))) and len(value) == 1:
lib/nappy/contrib/aircraft/old_na_to_nc_with_aircraft.py:    import cdms, Numeric
lib/nappy/contrib/aircraft/old_na_to_nc_with_aircraft.py:    raise Exception("Could not import third-party software. Nappy requires the CDMS and Numeric packages to be installed to convert to CDMS and NetCDF.")
lib/nappy/contrib/aircraft/old_na_to_nc_with_aircraft.py:               varObj=cdms.createVariable(Numeric.array(varObj), id=getBestName(varMetadata).replace(" ", "_"), attributes=varMetadata)
lib/nappy/contrib/aircraft/old_na_to_nc_with_aircraft.py:            ######## It  might handle writing from a Numeric array ########
lib/nappy/contrib/aircraft/old_na_to_nc_with_aircraft.py:            ######## It  might handle writing from a Numeric array ########
lib/nappy/contrib/aircraft/old_na_to_nc_with_aircraft.py:                if type(value) in (type([]), type(Numeric.array([0]))) and len(value)==1:
lib/nappy/utils/list_manipulator.py:    Takes an n-dimensional Numeric array and converts it to an

  2. Looks like cdat-lite requires NetCDF libraries to be installed on the system. Any idea which one(s)?

agstephens commented 6 years ago

@tomkralidis: The sensible thing to do here is:

  1. Deprecate (delete) all the .../aircraft/... modules - I don't think they are used.
  2. Replace cdat-lite with cfpython or iris.
  3. Replace Numeric with numpy. It would be a reasonably big task - and would depend the test convertors being converted into a proper test suite first - so that we could verify the results.