search

inbo / fish-tracking

🐟 Collection of scripts for processing and analysing fish tracking data
3 stars 0 forks source link

Remove records faulty dates from historical data #45

Closed peterdesmet closed 8 years ago

peterdesmet commented 8 years ago

@bartaelterman, as requested by @PieterjanVerhelst, the following records with faulty dates should be removed from the historical data. I can also give the file as a csv.

Date/Time   Code Space  ID  Receiver Name   Receiver S/N    Station Name
1970-03-23 01:15:27 A69-1601    19439   VR2W-110783 110783  S4
1970-09-04 19:18:20 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-04 22:08:00 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-05 17:22:38 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-05 17:33:35 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-06 03:09:52 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-06 03:13:45 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-06 03:18:38 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-06 03:23:10 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-06 03:27:02 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-06 03:36:19 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-06 03:40:00 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-06 03:44:28 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-06 03:52:46 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-06 03:56:41 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-06 04:01:37 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-06 04:06:16 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-06 04:10:21 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-06 04:13:18 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-06 04:16:20 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-07 04:50:39 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-07 05:27:24 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-07 05:36:09 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-07 05:39:07 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-07 05:42:11 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-07 05:45:24 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-07 06:04:49 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-07 06:08:43 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-08 05:09:05 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-08 05:13:15 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-08 05:16:21 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-08 05:19:41 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-08 05:23:28 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-08 05:28:07 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-08 05:32:13 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-08 05:35:13 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-08 05:38:19 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-08 05:41:38 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-08 05:45:23 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-08 05:49:59 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-08 05:59:03 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-08 06:45:00 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-08 06:53:31 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-08 07:03:16 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-08 07:07:36 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-08 07:11:01 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-08 07:14:59 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-08 07:20:00 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-08 07:29:20 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-14 15:24:22 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-14 15:27:45 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-14 16:27:18 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-14 16:35:46 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-14 16:43:02 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-14 16:46:21 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-14 16:50:06 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
1970-09-15 03:55:06 A69-1601    13631   VR2W-112299 112299  17 Iso 8s 18
2005-02-12 09:02:48 A69-1601    13639   VR2W-123812 123812  Nieuwpoort Kust 8
2005-02-12 09:05:56 A69-1601    13639   VR2W-123812 123812  Nieuwpoort Kust 8
2005-02-12 09:09:18 A69-1601    13639   VR2W-123812 123812  Nieuwpoort Kust 8
peterdesmet commented 8 years ago

Uploaded to repository: https://github.com/LifeWatchINBO/fish-tracking/blob/master/data/records_with_incorrect_dates.csv

Maybe remove when no longer necessary?

PieterjanVerhelst commented 8 years ago

Indeed, they can be removed as the detections with the correct dates where uploaded on Google Drive (reprocessed files).

peterdesmet commented 8 years ago

Ah, that's why we have files called _reprocessed. :+1:

PieterjanVerhelst commented 8 years ago

2 original files with wrong date and time were removed from 1.Raw. One file contained both correct and incorrect date + time (VR2W_112299_20141015_1.csv). In this file, the records with the wrong date were removed and the file was uploaded again in 1.Raw (reprocessed files contain only records of which the wrong date was changed).