search

SuperDARN / rst

Radar Software Toolkit (RST)
https://superdarn.github.io/rst/
GNU General Public License v3.0
22 stars 18 forks source link

Incorrect CP ID's #259

Closed mts299 closed 4 years ago

mts299 commented 4 years ago

While getting some scripts up and running with pyDARN, I ran into a bug where a cp=0 I notified @kevinkrieger about this and we added it to our file checking script since CPID's probably shouldn't be between -99 - 99 reading: http://superdarn.thayer.dartmouth.edu/WG-sched/cpids.html On the DDWG side, if we find a file with an incorrect CP ID, we label it as a "failed file" then @kevinkrieger will notify the PI about it, to discuss what they want to do with the files.

However, I am wondering if this is something we would want to add to RST as well since we are moving towards error checking? Maybe a warning or error to notify the user: Warning: CP value is incorrect with SuperDARN standards, please use this file with caution. Error: CP value is incorrect with SuperDARN standards, please contact PI and DDWG chair about this file.

Thoughts @ksterne @egthomas @ecbland @mardet987 ?

I can reflect on what we chose into pyDARN as well.

ksterne commented 4 years ago

Sorry I've been slow on this and all things RST all. I think this sounds like a decent idea, though I need to give it a little more thought.

egthomas commented 4 years ago

CPID guidelines have never been enforced within the SuperDARN community and should (in my opinion) always be treated with at least some degree of caution. For example, there have been several cases of users forgetting (or ignoring) to change the CPID from the default 150/151 when writing a new control program. So while a CPID of 0 (or < 100) may suggest a potential issue, that parameter alone is not sufficient evidence that a data record is corrupt or invalid.

I'm guessing this issue may have been raised with respect to recent Buckland Park data, for which the VT summary plots show a CPID of 0 (eg here). Interestingly, the fitacf3 plots for recent days look reasonable while the fitacf2.5 plots are significantly noisier.

ecbland commented 4 years ago

@egthomas is correct that the CPID cannot be used to determine if a data record is invalid. Since the CPID guidelines have not been enforced, I worry that an error/warning message would create more confusion because most likely the data record is fine.

mts299 commented 4 years ago

Thank you, I agree there might not be much we can do given the CPID's are not enforced.

However, if we want to enforce them in the near future, a warning message would be a good first step. Another thing is by having a warning message, it lets people know about it right away, and then they can contact the PI, which can then fix the potentially ongoing problem. But this check could always go into the data checking scripts and then DDWG can look into it first.

For example, by pyDARN breaking on the Buckland Park data, I was able to notify the PI's with @kevinkrieger help. They seemed to be unaware of the problem and are now looking into the problem with their new python ROS code.

ecbland commented 4 years ago

@ksterne Do you have anything else to add to this discussion? Would checking CPIDs be the responsibility of the DDWG? I don't see this as a data analysis issue because the CPID is not actually used for any data processing in RST - it's just copied into the data structure.

ksterne commented 4 years ago

There's been some good points brought up here and it's certainly a complicated issue (as most are). It would be nice to start calling records with CPID of 0 invalid, but there hasn't been a standard for that. I feel like that will be left up to the DSWG to set as a standard for our data, this is still TBD after the DSWG is formed and going though. Checking the CPIDs in rawacf files sounds like a task for DDWG. So I'm not sure there is much we can do about it now, but it is something to keep in mind.

If the DSWG does enforce CPID values then we will need to modify somethings to help detect these. We could help out the DDWG by adding a binary that checks data integrity but, I think this exists in another software package. Something that could help inform the DSWG of this problem is getting a forked repo (or altogether separate repo) going that helps quantify this problem.

It sounds good, and I like it. I just don't know if there's much we can do for this repo for now.

ecbland commented 4 years ago

We agreed during today's telecon that this issue is outside the scope of DAWG so we won't take any action for now. @ksterne's previous comment is a good summary of our discussion.