Tsid warnings at the validation step from the LiPD utilities

[khider]

Tsids error when using lipd.validate() but Tsids are being generated nonetheless ODP659.Tiedemann.1994.zip ODP659.Tiedemann.1994.xlsx

[chrismheiser]

Not reproducible

[chrismheiser]

Kawahata.2009.zip

I created a file through Excel -> LiPD that gave me the error. File has TSids in JSON. { "TSid" : "PYTKXWTD3OW", "TakenAtDepth" : "Depth", "number" : 1, "variableName" : "sample type" }, { "TSid" : "PYTGN31D2ND", "TakenAtDepth" : "Depth", "number" : 2, "variableName" : "specific name" }, { "TSid" : "PYTIEZNZ4IP", "TakenAtDepth" : "Depth", "number" : 3, "variableName" : "sample #" },

Validator API shows missing 109 TSids

Same file passes without the TSid warning on the website.

[chrismheiser]

The issue is still elusive. I did the same steps as earlier and the warning no longer shows up. It seems like it triggers for the first validation of some files, but not again.

• Convert Excel file -> validate -> No TSid warnings
• Using the same file that caused the issue in the post above -> validate - > No TSid warnings

[khider]

Got the warning again. Screenshot and file attached GeoB1023-5.Kim.2002.xlsx

[chrismheiser]

I found what may be the issue. It's a sneaky one. I'll note the reasoning to make it known here in the issue tracker.

The process of converting an excel file into a LiPD file is:

1. Parse and convert the excel data
2. Write a LiPD file (using "manual" steps in the excel_main.py, NOT writeLipd()... )
3. readLipd() - Read the LiPD file back into the workspace. (to gain the benefits of inferred calculations, structure updates, etc)

<< User validates file before TSids are added in step 4 >>

4. writeLipd() - This is where TSids are created and stored in the LiPD file

The issue is between steps 3 and 4. The reason the bug only happens on the first validation is because that's the only time that the LiPD data in the workspace differs from the LiPD data in the file that's saved. The workspace data does not have TSids, while the file saved to the computer does have Tsids. If you were to restart the Python console and load in that LiPD file again, that would be the file that had TSids, and you wouldn't get the validation warning again.