cositools / cosipy

The COSI high-level data analysis tools
Apache License 2.0
4 stars 18 forks source link

SpaceCraftFile refactor for orbital info breaks 511keV-DC2-ScAtt-DataReduction notebook #265

Open augustus-thomas opened 4 days ago

augustus-thomas commented 4 days ago

The recent refactor of SpaceCraftFile to support orientation files with orbital information makes the default orientation file in 511keV-DC2-ScAtt-DataReduction.ipynb un-parsable. This is because this line in the new SpaceCraftFile.parse_from_file() implies the source file has orbital info.

Potential fixes

  1. Change the suggested orientation file to be 20280301_3_month_with_orbital_info.ori instead of 20280301_3_month.ori in the Jupyter Notebook.
  2. Continue to support parsing files that do not have orbital information.

Trace of Error

File /project/cassini/cosi/cosipy/cosipy/spacecraftfile/SpacecraftFile.py:134, in SpacecraftFile.parse_from_file(cls, file)
    117 @classmethod
    118 def parse_from_file(cls, file):
    120     """
    121     Parses timestamps, axis positions from file and returns to __init__.
    122 
   (...)
    131         The SpacecraftFile object.
    132     """
--> 134     orientation_file = np.loadtxt(file, usecols=(1, 2, 3, 4, 5, 6, 7, 8),delimiter=' ', skiprows=1, comments=("#", "EN"))
    135     time_stamps = orientation_file[:, 0]
    136     axis_1 = orientation_file[:, [2, 1]]

File /project/cassini/miniconda3/envs/cosipy/lib/python3.10/site-packages/numpy/lib/npyio.py:1373, in loadtxt(fname, dtype, comments, delimiter, converters, skiprows, usecols, unpack, ndmin, encoding, max_rows, quotechar, like)
   1370 if isinstance(delimiter, bytes):
   1371     delimiter = delimiter.decode('latin1')
-> 1373 arr = _read(fname, dtype=dtype, comment=comment, delimiter=delimiter,
   1374             converters=converters, skiplines=skiprows, usecols=usecols,
   1375             unpack=unpack, ndmin=ndmin, encoding=encoding,
   1376             max_rows=max_rows, quote=quotechar)
   1378 return arr

File /project/cassini/miniconda3/envs/cosipy/lib/python3.10/site-packages/numpy/lib/npyio.py:1016, in _read(fname, delimiter, comment, quote, imaginary_unit, usecols, skiplines, max_rows, converters, ndmin, unpack, dtype, encoding)
   1013     data = _preprocess_comments(data, comments, encoding)
   1015 if read_dtype_via_object_chunks is None:
-> 1016     arr = _load_from_filelike(
   1017         data, delimiter=delimiter, comment=comment, quote=quote,
   1018         imaginary_unit=imaginary_unit,
   1019         usecols=usecols, skiplines=skiplines, max_rows=max_rows,
   1020         converters=converters, dtype=dtype,
   1021         encoding=encoding, filelike=filelike,
   1022         byte_converters=byte_converters)
   1024 else:
   1025     # This branch reads the file into chunks of object arrays and then
   1026     # casts them to the desired actual dtype.  This ensures correct
   1027     # string-length and datetime-unit discovery (like `arr.astype()`).
   1028     # Due to chunking, certain error reports are less clear, currently.
   1029     if filelike:

ValueError: invalid column index 6 at row 1 with 6 columns
ckarwin commented 4 days ago

Thanks, @augustus-thomas. I think we can implement your suggested fix #1. Do you want to update this?

augustus-thomas commented 4 days ago

@ckarwin Yes, I can do that

ckarwin commented 4 days ago

Great, thank you!

augustus-thomas commented 3 days ago

Also being discussed in Issue #263