NASA-PDS / validate

Validates PDS4 product labels, data and PDS3 Volumes
https://nasa-pds.github.io/validate/
Apache License 2.0
16 stars 11 forks source link

Validate reports unexpected NaN error for ASCII_String fields #956

Open jennifergward opened 2 months ago

jennifergward commented 2 months ago

Checked for duplicates

No - I haven't checked

🐛 Describe the bug

We realize this won't be addressed until all the NaN issues are sorted out, but validate is reporting an error for values of "NaN" in ASCII tables that have fields defined as ASCII_String. NaN is not allowed for data type ASCII_Real. We can't find a statement in the SR that says NaN is not allowed for ASCII_String.

🕵️ Expected behavior

We didn't expect to see this error for ASCII_String fields.

📜 To Reproduce

validate validate_error\ -r validate_report.txt

🖥 Environment Info

Windows

📚 Version of Software Used

validate 3.6.0-SNAPSHOT

🩺 Test Data / Additional context

validate_error(1).zip

🦄 Related requirements

🦄 #xyz

⚙️ Engineering Details

No response

🎉 Integration & Test

No response

matthewtiscareno commented 2 months ago

I see that the description for the column in question says that it "may or may not contain data." However, in practice, it seems to either be "NaN" or a number. What is the meaning of the NaNs? Should that meaning be documented? Why not substitute a numerical special constant for the NaNs and then use ASCII_Real?

jennifergward commented 2 months ago

I wasn't involved with this migration, but I'm guessing "ASCII_String" was chosen because "CHARACTER" had been used in the PDS3 label. I'm also guessing that "CHARACTER" was chosen because vtool was balking at the NaN characters. I agree that ASCII_Real should have been used with a special constant. However, I still think that validate shouldn't return an error for "NaN" in "ASCII_String," whether it should have been used or not.

matthewtiscareno commented 2 months ago

@jennifergward: I'm not sure I agree, but I respect your point of view.

matthewtiscareno commented 1 month ago

Per last week's discussions at the PDS Management Council, I now agree with @jennifergward. Validate should be fixed so that "NaN" is not rejected when the data type is ASCII_String or related.