search

cioos-siooc / cioos-pacific-erddap

CIOOS Pacific ERDDAP main repository used to maintain, develop and integrate ERDDAP datasets.
https://data.cioospacific.ca/erddap
2 stars 0 forks source link

TOB datasets Time difference between first record and metadata Start TIme #13

Closed JessyBarrette closed 1 year ago

JessyBarrette commented 1 year ago

Some files present a significiant difference in time between the first record available the Start time value present within the header *FILE -> START_TIME attribute

This is just a warning, we're still ingesting that data.

25 files associated with this issue

file self.obs_time[0]-self.start_dateobj
www.waterproperties.ca/osd_data_archive/Cruise_Data/2014/2014-019/Underway/2014-019-0003 0:02:00
www.waterproperties.ca/osd_data_archive/Cruise_Data/2014/2014-019/Underway/2014-019-0005 0:03:00
www.waterproperties.ca/osd_data_archive/Cruise_Data/2006/2006-033/Underway/2006-033-0001 0:25:01
www.waterproperties.ca/osd_data_archive/Cruise_Data/2006/2006-033/Underway/2006-033-0002 0:24:59
www.waterproperties.ca/osd_data_archive/Cruise_Data/2015/2015-048/Underway/2015-048-0001 0:04:00
www.waterproperties.ca/osd_data_archive/Cruise_Data/2015/2015-056/Underway/2015-056-0001 0:21:31
www.waterproperties.ca/osd_data_archive/Cruise_Data/2016/2016-017/Underway/TSG_20160710_1809 365 days, 0:00:06
www.waterproperties.ca/osd_data_archive/Cruise_Data/2007/2007-011/Underway/2007-011-0002 19 days, 0:39:13
www.waterproperties.ca/osd_data_archive/Cruise_Data/2016/2016-017/Underway/TSG_20160710_1809 365 days, 0:00:06
www.waterproperties.ca/osd_data_archive/Cruise_Data/2016/2016-017/Underway/TSG_20160702_2038 365 days, 0:00:09
www.waterproperties.ca/osd_data_archive/Cruise_Data/2016/2016-017/Underway/TSG_20160713_2044 365 days, 0:00:07
www.waterproperties.ca/osd_data_archive/Cruise_Data/2016/2016-017/Underway/TSG_20160714_0343 365 days, 0:00:05
www.waterproperties.ca/osd_data_archive/Cruise_Data/2007/2007-011/Underway/2007-011-0003 38 days, 5:00:08
www.waterproperties.ca/osd_data_archive/Cruise_Data/2007/2007-016/Underway/2007-016-0001 0:06:00
www.waterproperties.ca/osd_data_archive/Cruise_Data/2022/2022-001/Underway/TSG/2022-001-20220308-092814 9:28:09
www.waterproperties.ca/osd_data_archive/Cruise_Data/2008/2008-010/Underway/2008-010-0001 -236 days, 0:00:00
www.waterproperties.ca/osd_data_archive/Cruise_Data/2008/2008-015/Underway/2008-015-0004 -5 days, 0:00:02.520000
www.waterproperties.ca/osd_data_archive/Cruise_Data/2008/2008-027/Underway/2008-027-0002 0:37:30
www.waterproperties.ca/osd_data_archive/Cruise_Data/2009/2009-011/Underway/2009-011-0001 2:30:00
www.waterproperties.ca/osd_data_archive/Cruise_Data/2010/2010-001/Underway/2010-001-0002 0:03:00
www.waterproperties.ca/osd_data_archive/Cruise_Data/2010/2010-015/Underway/2010-015-0003 0:01:00
www.waterproperties.ca/osd_data_archive/Cruise_Data/2011/2011-017/Underway/2011-017-0001 0:07:33
www.waterproperties.ca/osd_data_archive/Cruise_Data/2011/2011-027/Underway/2011-027-0002 0:04:11
www.waterproperties.ca/osd_data_archive/Cruise_Data/2013/2013-001/Underway/2013-001-0002 0:05:00
www.waterproperties.ca/osd_data_archive/Cruise_Data/2006/2006-014/Underway/2006-014-0001 0:09:59
--- ---

related to HakaiInstitute/cioos-pacific-erddap-public#20

guanlu129 commented 1 year ago

We would generally expect the start time in header match with the time of the first record in data section. But it is possible that after it was set, some records were removed. We need to check the files with time difference over 365 days on the list.

JessyBarrette commented 1 year ago

sounds good to me

guanlu129 commented 1 year ago

" Start time " has been updated in the files on the list.

JessyBarrette commented 1 year ago

@guanlu129 Awesome we can then close that issue!