MTX tags recovered and transmitted tags

[CindyTribuzio-NOAA]

In cases when there are both recovered and transmitted data, does the most recent upload by tag_num/tag_type overwrite the previous? For example, if I upload the transmitted data, then the recovered data, does the transmitted data go poof? Need to be able to retain both versions. Maybe add a retained/transmitted flag?

Example tag ST118356

[tristan-sebens]

The transmitted data should not be deleted when recovered data is uploaded. The DB enforces constraints on the data tables such that each tag is only allowed a single record per instant in time (or period of time in the case of summary data).

If transmitted data from tag A is uploaded to the DB, and then later tag A is recovered and its data is reuploaded to the DB, then the only time that the initially uploaded data would be overwritten would be if one of the records in the recovered data had the exact same timestamp/period as a transmitted record. If this were the case, however, then we would expect this to in fact be that same record being re-reported to the DB by the tag a second time.

[CindyTribuzio-NOAA]

Not necessarily for MTX tags. Transmitted data are stored differently from archived data and are really apples to oranges. Transmitted data values are based on the change in value from the previous, which is anchored to the actual value every once in a while. Because the data types are different, they can't be inserted into the same time series. Unfortunately, there needs to be a way to identify "recovered" vs "transmitted" data (here "recovered" may be the same as "archived").

[tristan-sebens]

I see. I will need to think on a way to address this, as based on your explanation I believe that the current configuration of the package would overwrite any transmitted records with archived records that had the same timestamp. Unfortunately, the current architecture of the DB doesn't really support having multiple records for the same tag with the same timestamp.

How have you kept these records distinguished from one another in the DB in the past?

[CindyTribuzio-NOAA]

We had archived in different tables, see TAG_MT_ARCHIVAL.......for examples

[tristan-sebens]

Well, we can pursue another instance of that solution.

My one concern is that this issue could expand and require duplicates of every table we've created so far, one for transmitted data, one for archival data. But maybe that's just the way things have to be.