NHMA - Will not validate in Workbench

[jlegind]

What is the issue ?

In the test Specify installation (https://specify-test.science.ku.dk/ when it was pointing to NHMA) the workbench import could not validate the mock dataset due to an odd error.

Detailed description of the issue.

The validation error appears almost right away saying: { "uploaderstatus": { "operation": "validating", "taskid": "4c6f6459-4a29-43e4-a6bb-93e77b7d7703" }, "taskstatus": "FAILURE", "taskinfo": "DoesNotExist('Spdataset matching query does not exist.',)" }

Why is it needed/relevant ?

If this is not resolved, then Aarhus data won't import through workbench.

Estimate level of effort required.

?

What is the expected acceptable result.

A regular validation message.

[FedorSteeman]

A strange, unexplained error we need to take heed of when importing real data.

[FedorSteeman]

To be clear: I want to see what happens with real data first. Maybe it was just a glitch or caused by something about the mock-up data. But we'll get it fixed if it's a pervasive issue

[FedorSteeman]

It's hard to predict how much time this would take to mitigate since the root cause is unknown and hidden in the Specify system. It may take a day or more and may also involve waiting for Kansas in case we need their help. The actual effort may be a couple of hours work but spread over a week's period.

[FedorSteeman]

Given the latest feedback from Kansas, it's possible that the WorkBench upload issues experienced by @Sosannah may be related to a problem in the mapping. I will go through Jan's upload to NHMAtest to inspect.

[FedorSteeman]

Although these may not be the source of the issues, I found the following mapping errors:

This field should actually be mapped to as follows:

The follwing mapping error, however, may be the culprit:

The field preptypeid should not be mapped at all, let alone to a second preparation's name!

I also found the following invalid mapping:

The collection name is already mapped from field "collection", so this secondary mapping could also be a source of the issue.

I will save and revalidate.

[FedorSteeman]

Error persists despite the faulty mapping being fixed.

[FedorSteeman]

Answer from Kansas:

It looks like the primary issue with this dataset is that the date format established in the Remote Preferences app resource does not match the format in the import document. This can be resolved by either changing the date format in the imported spreadsheet or by changing how dates are handled in the application by following these instructions. "value": "2023-09-28", "format": "%d/%m/%Y"

[FedorSteeman]

Changing the date format does not take away the error message. Concerning that Kansas could not replicate the problem. Perhaps they're not using a matching version of Specify7 and the future upgrade will eliminate this issue. I suggest we shelf this ticket until that time, so we're not spending more time on this than necessary.

[FedorSteeman]

Moving to upcoming sprints as NHM Aarhus digitisation has not yet been initiated.

[FedorSteeman]

Moved to current sprint as NHMA implementation is approaching.

[FedorSteeman]

I've successfully validate a test file in the production site of NHMA without any errors occurring so this can no longer be reproduced. We'll look at it again when problems arise with actual data.