mrghg / agage-archive

Code for producing AGAGE archival files
MIT License
3 stars 0 forks source link

instrument_type getting changed to float #77

Closed joe-pitt closed 2 months ago

joe-pitt commented 2 months ago

Somewhere in the code for dealing with duplicate timestamps in io.py, instrument_type gets changed to a float. I'm also not sure it is dropping the right things. I have added the following print statements

    if len(ds_combined.time) != len(ds_combined.time.drop_duplicates(dim="time")):

        timestamps = ds_combined.time.to_series()
        duplicated_timestamps = timestamps[timestamps.duplicated(keep="first")]
        print(duplicated_timestamps)

        for timestamp in duplicated_timestamps:
            print(ds_combined.instrument_type.sel(time=timestamp).values)

For multiple duplicate timestamps (with two Picarros) I get:

time
2018-10-03 12:00:00   2018-10-03 12:00:00
2018-10-03 13:00:00   2018-10-03 13:00:00
2018-10-03 14:00:00   2018-10-03 14:00:00
2018-10-03 15:00:00   2018-10-03 15:00:00
2018-10-03 16:00:00   2018-10-03 16:00:00
                              ...        
2023-10-19 13:00:00   2023-10-19 13:00:00
2023-10-19 14:00:00   2023-10-19 14:00:00
2023-10-19 15:00:00   2023-10-19 15:00:00
2023-10-19 16:00:00   2023-10-19 16:00:00
2023-10-19 17:00:00   2023-10-19 17:00:00
Name: time, Length: 44214, dtype: datetime64[ns]
[7 7]
[7. 7.]
7.0

So on the first loop it prints integers (fine), then on the second loop it prints floats (weird), on the third loop it is no longer an array with two instruments (surely not the intention?). In my error log I get: HFD co: TypeError in stack: run (line 397) / io (line 798). iteration over a 0-d array