search

colinrsmall / mp-dl-unh

0 stars 0 forks source link

Model results do not seem to be reproducible #5

Open argallmr opened 4 years ago

argallmr commented 4 years ago

The model seems to produce new selections every time it runs, even if it is run on the same set of data. Below are the sorted, non-overlapping selections for a few days' worth of selections. TSTART, TSTOP, FOM, SOURCEID, and, DISCUSSION describe the time interval, ranking, gls model, and description of the event. CREATETIME is the generation time of the file containing the respective selection.

import datetime as dt
from pymms.pymms import selections as sel

t1 = dt.datetime(2019, 12, 3, 15, 40, 58)
t2 = dt.datetime(2019, 12, 7, 0, 0)

data = sel.selections('gls', t1, t2, unique=True)
sel.print_selections(data, full=True)

Gives are the results. Note that if overlapping segements are included, the number of selections increases from 105 to 616.

             TSTART                 TSTOP            CREATETIME     FOM    SOURCEID   DISCUSSION
2019-11-30 06:06:53   2019-11-30 06:11:13   2019-12-04 01:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-11-30 06:24:53   2019-11-30 06:33:13   2019-12-04 01:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-11-30 06:37:13   2019-11-30 06:37:43   2019-12-04 01:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-11-30 06:39:43   2019-11-30 07:01:03   2019-12-04 01:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-03 15:41:43   2019-12-03 15:50:33   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-03 15:54:13   2019-12-03 15:55:03   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-03 15:58:33   2019-12-03 15:59:33   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-03 16:05:33   2019-12-03 16:06:13   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-03 18:45:03   2019-12-03 18:45:13   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-03 18:47:54   2019-12-03 18:48:43   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-03 18:56:13   2019-12-03 18:59:23   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-03 19:01:43   2019-12-03 19:04:33   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-03 21:43:03   2019-12-03 21:43:13   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-03 21:48:53   2019-12-03 21:52:33   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-03 21:55:13   2019-12-03 21:58:13   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-03 22:10:33   2019-12-03 22:11:23   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-03 22:11:53   2019-12-03 22:12:53   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 03:02:43   2019-12-05 03:03:03   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 03:15:33   2019-12-05 03:15:43   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 03:36:13   2019-12-05 03:36:23   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 03:45:13   2019-12-05 03:45:23   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 03:51:53   2019-12-05 03:52:03   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 03:52:23   2019-12-05 03:52:33   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:02:04   2019-12-05 04:02:14   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:02:23   2019-12-05 04:02:33   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:05:34   2019-12-05 04:05:44   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:06:54   2019-12-05 04:07:03   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:07:24   2019-12-05 04:07:34   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:07:44   2019-12-05 04:09:04   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:09:43   2019-12-05 04:10:03   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:10:03   2019-12-05 04:10:54   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:12:14   2019-12-05 04:12:24   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:12:53   2019-12-05 04:13:03   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:13:23   2019-12-05 04:13:43   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:14:13   2019-12-05 04:14:23   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:14:53   2019-12-05 04:15:03   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:15:13   2019-12-05 04:15:23   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:15:33   2019-12-05 04:15:53   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:16:13   2019-12-05 04:17:23   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:17:33   2019-12-05 04:19:13   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:21:43   2019-12-05 04:22:23   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:23:44   2019-12-05 04:23:54   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:24:03   2019-12-05 04:24:13   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:24:23   2019-12-05 04:24:33   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:25:33   2019-12-05 04:26:23   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:27:04   2019-12-05 04:28:34   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:29:03   2019-12-05 04:29:23   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:31:13   2019-12-05 04:32:24   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:32:34   2019-12-05 04:32:44   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:34:34   2019-12-05 04:36:23   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:36:43   2019-12-05 04:36:53   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:37:24   2019-12-05 04:37:33   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:37:44   2019-12-05 04:37:54   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:38:43   2019-12-05 04:39:13   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:41:23   2019-12-05 04:41:33   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:42:43   2019-12-05 04:42:53   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:44:03   2019-12-05 04:45:24   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:46:13   2019-12-05 04:46:43   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:47:03   2019-12-05 04:47:13   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:47:43   2019-12-05 04:49:24   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:49:43   2019-12-05 04:49:53   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:50:03   2019-12-05 04:50:13   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:50:43   2019-12-05 04:50:53   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:51:14   2019-12-05 04:51:53   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:52:03   2019-12-05 04:53:23   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:54:13   2019-12-05 04:55:53   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:56:53   2019-12-05 04:57:23   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:57:24   2019-12-05 04:58:14   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:59:13   2019-12-05 04:59:23   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:59:33   2019-12-05 04:59:43   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:59:44   2019-12-05 05:00:04   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:00:53   2019-12-05 05:02:03   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:02:13   2019-12-05 05:03:13   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:05:03   2019-12-05 05:05:54   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:06:13   2019-12-05 05:06:33   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:08:54   2019-12-05 05:09:04   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:09:13   2019-12-05 05:10:53   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:10:54   2019-12-05 05:13:13   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:13:54   2019-12-05 05:16:34   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:16:44   2019-12-05 05:16:54   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:16:54   2019-12-05 05:19:04   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:20:03   2019-12-05 05:23:23   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:23:23   2019-12-05 05:23:24   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:23:34   2019-12-05 05:23:44   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:24:14   2019-12-05 05:24:34   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:25:33   2019-12-05 05:25:53   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:26:33   2019-12-05 05:27:23   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:28:34   2019-12-05 05:30:53   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:31:33   2019-12-05 05:31:43   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:31:54   2019-12-05 05:32:14   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:33:23   2019-12-05 05:34:13   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:35:13   2019-12-05 05:35:33   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:35:53   2019-12-05 05:36:03   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:36:04   2019-12-05 05:37:13   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:44:53   2019-12-05 05:45:03   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:45:53   2019-12-05 05:46:03   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:48:13   2019-12-05 05:48:23   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:49:23   2019-12-05 05:49:33   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:49:43   2019-12-05 05:49:53   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:50:14   2019-12-05 05:50:24   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:50:34   2019-12-05 05:50:44   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:53:43   2019-12-05 05:54:03   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:56:23   2019-12-05 05:57:33   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:57:53   2019-12-05 05:58:03   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:58:43   2019-12-05 05:58:53   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
colinrsmall commented 4 years ago

Thanks for the report. Did you update pymms to include mp-dl-unh? I looked at your pymms repo and it doesn't look like it's been updated since last March.

colinrsmall commented 4 years ago

This will need more looking into. The model itself should be deterministic.

argallmr commented 4 years ago

I made a significant update to pymms yesterday. The pymms.pymms.selections module has some useful tools for handling selections. In pymms.bin there are scripts for writing and plotting selections (SITL/ABS/GLS). Have to figure out how to update it on pypi...

colinrsmall commented 4 years ago

Awesome, thanks for writing those.

So what about the model isn't reproducible? If I'm reading your initial issue message correctly, I only see one set of selections. Is the issue in that CREATTIME isn't ascending as the time of the selection ascends?

argallmr commented 4 years ago

First, note that the science region of interest that the SITL is making selections in runs from 2019-12-03T15:40:58 to 2019-12-06T18:54:02. The way the burst management system works is that only the selections from latest file will be queued for downlink.

Below, is the same list, now sorted by CREATETIME. Note that all of the unique selections made with CREATETIMEs earlier than 2019-12-06 22:26:14 were not reproduced by the model and would not be downlinked (see caveat at bottom).

             TSTART                 TSTOP            CREATETIME     FOM    SOURCEID   DISCUSSION
2019-11-30 06:06:53   2019-11-30 06:11:13   2019-12-04 01:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-11-30 06:24:53   2019-11-30 06:33:13   2019-12-04 01:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-11-30 06:37:13   2019-11-30 06:37:43   2019-12-04 01:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-11-30 06:39:43   2019-11-30 07:01:03   2019-12-04 01:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 03:02:43   2019-12-05 03:03:03   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 03:15:33   2019-12-05 03:15:43   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 03:36:13   2019-12-05 03:36:23   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 03:51:53   2019-12-05 03:52:03   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 03:52:23   2019-12-05 03:52:33   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:13:23   2019-12-05 04:13:43   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:14:53   2019-12-05 04:15:03   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:15:13   2019-12-05 04:15:23   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:15:33   2019-12-05 04:16:03   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:16:13   2019-12-05 04:17:23   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:17:33   2019-12-05 04:19:13   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:21:43   2019-12-05 04:22:23   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:24:23   2019-12-05 04:24:33   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:25:33   2019-12-05 04:26:23   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:29:23   2019-12-05 04:29:33   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:36:43   2019-12-05 04:36:53   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:46:13   2019-12-05 04:46:43   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:56:53   2019-12-05 04:57:23   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:59:33   2019-12-05 04:59:43   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:45:53   2019-12-05 05:46:03   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:49:23   2019-12-05 05:49:33   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:49:43   2019-12-05 05:49:53   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:53:43   2019-12-05 05:54:03   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:56:23   2019-12-05 05:57:33   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:57:53   2019-12-05 05:58:03   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:58:43   2019-12-05 05:58:53   2019-12-06 19:16:20   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-03 16:05:33   2019-12-03 16:06:13   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-03 18:47:54   2019-12-03 18:48:43   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-03 21:43:03   2019-12-03 21:43:13   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:02:04   2019-12-05 04:02:14   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:02:23   2019-12-05 04:02:33   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:05:34   2019-12-05 04:05:44   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:06:54   2019-12-05 04:07:03   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:07:44   2019-12-05 04:09:04   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:14:13   2019-12-05 04:14:23   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:27:04   2019-12-05 04:28:34   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:31:13   2019-12-05 04:32:24   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:32:34   2019-12-05 04:32:44   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:34:34   2019-12-05 04:36:23   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:37:24   2019-12-05 04:37:33   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:37:44   2019-12-05 04:37:54   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:38:43   2019-12-05 04:40:03   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:41:23   2019-12-05 04:41:33   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:49:43   2019-12-05 04:49:53   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:50:03   2019-12-05 04:50:13   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:50:43   2019-12-05 04:50:53   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:51:03   2019-12-05 04:51:13   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:53:13   2019-12-05 04:54:03   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:54:13   2019-12-05 04:55:53   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:59:13   2019-12-05 04:59:23   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:02:13   2019-12-05 05:03:13   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:06:13   2019-12-05 05:06:33   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:25:33   2019-12-05 05:25:53   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:35:13   2019-12-05 05:35:33   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:35:53   2019-12-05 05:36:03   2019-12-06 20:46:48   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-03 15:41:43   2019-12-03 15:50:33   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-03 15:54:13   2019-12-03 15:55:03   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-03 15:58:33   2019-12-03 15:59:33   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-03 18:45:03   2019-12-03 18:45:13   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-03 18:56:13   2019-12-03 18:59:23   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-03 19:01:43   2019-12-03 19:04:33   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-03 21:48:53   2019-12-03 21:52:33   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-03 21:55:13   2019-12-03 21:58:13   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-03 22:10:33   2019-12-03 22:11:23   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-03 22:11:53   2019-12-03 22:12:53   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 03:45:13   2019-12-05 03:45:23   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:07:24   2019-12-05 04:07:34   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:10:03   2019-12-05 04:10:54   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:12:14   2019-12-05 04:12:24   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:12:53   2019-12-05 04:13:03   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:23:44   2019-12-05 04:23:54   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:24:03   2019-12-05 04:24:13   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:29:03   2019-12-05 04:29:23   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:42:43   2019-12-05 04:42:53   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:44:03   2019-12-05 04:45:24   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:47:03   2019-12-05 04:47:13   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:47:43   2019-12-05 04:49:24   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:51:14   2019-12-05 04:51:53   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:52:03   2019-12-05 04:52:33   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:57:24   2019-12-05 04:58:14   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 04:59:44   2019-12-05 05:00:04   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:00:53   2019-12-05 05:02:03   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:05:03   2019-12-05 05:05:54   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:08:54   2019-12-05 05:09:04   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:09:13   2019-12-05 05:09:23   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:10:54   2019-12-05 05:13:13   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:13:54   2019-12-05 05:16:34   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:16:44   2019-12-05 05:19:04   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:20:03   2019-12-05 05:21:24   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:22:33   2019-12-05 05:22:44   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:22:54   2019-12-05 05:23:13   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:23:23   2019-12-05 05:23:24   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:23:34   2019-12-05 05:23:44   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:24:14   2019-12-05 05:24:34   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:26:33   2019-12-05 05:27:23   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:28:34   2019-12-05 05:30:53   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:31:33   2019-12-05 05:31:43   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:31:54   2019-12-05 05:33:04   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:33:23   2019-12-05 05:34:13   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:36:04   2019-12-05 05:37:13   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:44:53   2019-12-05 05:45:03   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:48:13   2019-12-05 05:48:23   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:50:14   2019-12-05 05:50:24   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)
2019-12-05 05:50:34   2019-12-05 05:50:44   2019-12-06 22:26:14   150.0   mp-dl-unh   MP crossing (automatically generated)

Note that the 501 segments not appearing in the list above do not appear there because they are exact duplicates of selection in the list. For example,

             TSTART                 TSTOP            CREATETIME
2019-12-03 19:01:43   2019-12-03 19:04:33   2019-12-06 02:55:34
2019-12-03 19:01:44   2019-12-03 19:04:34   2019-12-06 04:05:34
2019-12-03 19:01:43   2019-12-03 19:04:33   2019-12-06 05:05:45
2019-12-03 19:01:43   2019-12-03 19:04:33   2019-12-06 09:56:20
2019-12-03 19:01:44   2019-12-03 19:04:34   2019-12-06 11:56:57
2019-12-03 19:01:43   2019-12-03 19:04:33   2019-12-06 14:16:18
2019-12-03 19:01:43   2019-12-03 19:04:33   2019-12-06 19:16:20
2019-12-03 19:01:44   2019-12-03 19:04:34   2019-12-06 20:46:48
2019-12-03 19:01:43   2019-12-03 19:04:33   2019-12-06 22:26:14
colinrsmall commented 4 years ago

I think I see the issue now. We know that the model is running multiple times (from the multiple CREATETIMEs), but we would expect that there would be no unique selections across the same date range if the model is behaving correctly, right? We would expect n duplicate selections, where n is the number of times the model has run.

Is it possible that this is an issue with how the job is being created/started at the SDC? To me, it looks multiple jobs of the same date range are being created at different times, hence duplicate selections of different CREATETIMEs. If jobs are created as MMS data becomes available, is possible that later segments of data becoming available before earlier ones is causing the model to run multiple times? The model will certainly change its predictions depending on how much data it is given.

argallmr commented 4 years ago

there would be no unique selections across the same date range if the model is behaving correctly, right?

Yes

If jobs are created as MMS data becomes available, is possible that later segments of data becoming available before earlier ones is causing the model to run multiple times?

I have never seen data become available out of order, but new data is appended to the end of old data after every DSN (Deep Space Network) contact. There are typically four dsn contacts per spacecraft each orbit (16 total contacts/orbit). At the end of each contact, the model is run.

The model will certainly change its predictions depending on how much data it is given.

Makes sense. I will look into caching the data after each contact for one orbit to see if this is the case. It would also be interesting to plot each selection and color them by CREATETIME to see if these not reproducible selections are meaningful.