search

pysat / pysatCDAAC

pysat support for CDAAC instruments
BSD 3-Clause "New" or "Revised" License
2 stars 2 forks source link

BUG: Fixed errant sat_id->inst_id. Altered COSMIC file times. #12

Closed rstoneback closed 3 years ago

rstoneback commented 3 years ago

Started on meta but found a time issue instead. Looks like the meta is handled over in #11.

The times created by pysat would be unique, but when written to an internal .pysat file, the resolution was insufficient to maintain a unique file list. So unless the file list was updated each time a cosmic_gps Instrument was instantiated, a few files would get removed.

rstoneback commented 3 years ago

Unique on over 2.3 million files, checked more than once to ensure the stored file list is good as well.


In [5]: gps.files.files                                                                                                                                                                                                 
Out[5]: 
2009-01-01 00:12:00.125700096    2009.001/ionPrf_C001.2009.001.00.12.G26_2013.3...
2009-01-01 00:16:00.185369856    2009.001/ionPrf_C002.2009.001.00.16.G24_2013.3...
2009-01-01 00:17:00.195489792    2009.001/ionPrf_C002.2009.001.00.17.G09_2013.3...
2009-01-01 00:27:00.351749888    2009.001/ionPrf_C001.2009.001.00.27.G10_2013.3...
2009-01-01 00:28:00.365440000    2009.001/ionPrf_C002.2009.001.00.28.G13_2013.3...
2009-01-01 00:31:00.408920064    2009.001/ionPrf_C005.2009.001.00.31.G31_2013.3...
2009-01-01 00:33:00.441420032    2009.001/ionPrf_C002.2009.001.00.33.G15_2013.3...
2009-01-01 00:33:00.446680064    2009.001/ionPrf_C002.2009.001.00.33.G20_2013.3...
2009-01-01 00:34:00.455239936    2009.001/ionPrf_C002.2009.001.00.34.G23_2013.3...
2009-01-01 00:35:00.467400192    2009.001/ionPrf_C001.2009.001.00.35.G12_2013.3...
2009-01-01 00:36:00.485690112    2009.001/ionPrf_C001.2009.001.00.36.G05_2013.3...
2009-01-01 00:38:00.524689920    2009.001/ionPrf_C002.2009.001.00.38.G26_2013.3...
2009-01-01 00:38:00.525170176    2009.001/ionPrf_C002.2009.001.00.38.G04_2013.3...
2009-01-01 00:43:00.596410112    2009.001/ionPrf_C005.2009.001.00.43.G29_2013.3...
2009-01-01 00:43:00.596420096    2009.001/ionPrf_C005.2009.001.00.43.G30_2013.3...
2009-01-01 00:45:00.627500032    2009.001/ionPrf_C002.2009.001.00.45.G28_2013.3...
2009-01-01 00:46:00.646919936    2009.001/ionPrf_C005.2009.001.00.46.G07_2013.3...
2009-01-01 00:48:00.683820032    2009.001/ionPrf_C001.2009.001.00.48.G04_2013.3...
2009-01-01 00:49:00.693619968    2009.001/ionPrf_C005.2009.001.00.49.G05_2013.3...
2009-01-01 00:51:00.725910016    2009.001/ionPrf_C002.2009.001.00.51.G31_2013.3...
2009-01-01 00:52:00.736480000    2009.001/ionPrf_C002.2009.001.00.52.G27_2013.3...
2009-01-01 00:52:00.742220032    2009.001/ionPrf_C005.2009.001.00.52.G12_2013.3...
2009-01-01 00:54:00.764679936    2009.001/ionPrf_C005.2009.001.00.54.G24_2013.3...
2009-01-01 00:56:00.797619968    2009.001/ionPrf_C001.2009.001.00.56.G13_2013.3...
2009-01-01 00:58:00.833910016    2009.001/ionPrf_C002.2009.001.00.58.G29_2013.3...
2009-01-01 00:58:00.839279872    2009.001/ionPrf_C002.2009.001.00.58.G30_2013.3...
2009-01-01 01:00:00.859469824    2009.001/ionPrf_C001.2009.001.01.00.G23_2013.3...
2009-01-01 01:01:00.881560064    2009.001/ionPrf_C005.2009.001.01.01.G21_2013.3...
2009-01-01 01:01:00.882110208    2009.001/ionPrf_C005.2009.001.01.01.G10_2013.3...
2009-01-01 01:03:00.013040128    2009.001/ionPrf_C002.2009.001.01.03.G07_2013.3...
                                                       ...                        
2018-01-02 19:48:00.324039936    2018.002/ionPrf_C006.2018.002.19.48.G14_2016.1...
2018-01-02 19:50:00.341569792    2018.002/ionPrf_C006.2018.002.19.50.G32_2016.1...
2018-01-02 20:05:00.587480064    2018.002/ionPrf_C006.2018.002.20.05.G04_2016.1...
2018-01-02 20:07:00.627419904    2018.002/ionPrf_C006.2018.002.20.07.G10_2016.1...
2018-01-02 20:17:00.784160256    2018.002/ionPrf_C006.2018.002.20.17.G18_2016.1...
2018-01-02 20:25:00.020969984    2018.002/ionPrf_C006.2018.002.20.25.G21_2016.1...
2018-01-02 20:33:00.146650112    2018.002/ionPrf_C006.2018.002.20.33.G15_2016.1...
2018-01-02 20:37:00.210190080    2018.002/ionPrf_C006.2018.002.20.37.G13_2016.1...
2018-01-02 20:48:00.393910016    2018.002/ionPrf_C006.2018.002.20.48.G05_2016.1...
2018-01-02 21:05:00.674610176    2018.002/ionPrf_C006.2018.002.21.05.G12_2016.1...
2018-01-02 21:10:00.760590080    2018.002/ionPrf_C006.2018.002.21.10.G06_2016.1...
2018-01-02 21:24:00.083950080    2018.002/ionPrf_C006.2018.002.21.24.G23_2016.1...
2018-01-02 21:27:00.125540096    2018.002/ionPrf_C006.2018.002.21.27.G31_2016.1...
2018-01-02 21:36:00.273620224    2018.002/ionPrf_C006.2018.002.21.36.G22_2016.1...
2018-01-02 21:45:00.425809920    2018.002/ionPrf_C006.2018.002.21.45.G16_2016.1...
2018-01-02 21:58:00.625780224    2018.002/ionPrf_C006.2018.002.21.58.G27_2016.1...
2018-01-02 21:59:00.642990080    2018.002/ionPrf_C006.2018.002.21.59.G08_2016.1...
2018-01-02 22:06:00.747149824    2018.002/ionPrf_C006.2018.002.22.06.G10_2016.1...
2018-01-02 22:21:00.094249984    2018.002/ionPrf_C006.2018.002.22.21.G15_2016.1...
2018-01-02 22:23:00.126490112    2018.002/ionPrf_C006.2018.002.22.23.G24_2016.1...
2018-01-02 22:30:00.231830016    2018.002/ionPrf_C006.2018.002.22.30.G20_2016.1...
2018-01-02 22:45:00.473180160    2018.002/ionPrf_C006.2018.002.22.45.G05_2016.1...
2018-01-02 22:49:00.541289984    2018.002/ionPrf_C006.2018.002.22.49.G29_2016.1...
2018-01-02 22:55:00.642720000    2018.002/ionPrf_C006.2018.002.22.55.G02_2016.1...
2018-01-02 23:08:00.847249920    2018.002/ionPrf_C006.2018.002.23.08.G09_2016.1...
2018-01-02 23:11:00.896290048    2018.002/ionPrf_C006.2018.002.23.11.G23_2016.1...
2018-01-02 23:16:00.064670208    2018.002/ionPrf_C006.2018.002.23.16.G07_2016.1...
2018-01-02 23:27:00.252140032    2018.002/ionPrf_C006.2018.002.23.27.G03_2016.1...
2018-01-02 23:33:00.348640000    2018.002/ionPrf_C006.2018.002.23.33.G22_2016.1...
2018-01-02 23:36:00.383579904    2018.002/ionPrf_C006.2018.002.23.36.G28_2016.1...
Length: 2319390, dtype: object

In [6]:                                                                                                                                                                                                                 ```
jklenzing commented 3 years ago

Tests are passing locally for me. I've fixed the travis setup in #11.