First output of persons data ready for review

[wlpotter]

@dlschwartz I have the first outputs of the persons transform, here: https://github.com/wlpotter/csv-to-srophe/tree/main/test/out/csv-tests/persons

The first two to look at are

1. 3229
2. 3774

These are the two sample records I used to build the persons transform module. So any glaring errors there are likely to be shared across the other data.

Then it would be worthwhile to spot-check several other records to identify issues.

Unfortunately, we can't rely here on the schema validation as we haven't updated that yet.

Finally, here are a few notes of known issues and problems

1. The anonymous description vs abstract has yet to be resolved. The script currently looks for sources associated with the anonymous description but we do not have separate columns for that data right now, so they all have the @resp="http://syriaca.org even if they should be sourced
2. Not all the anonymi records had the column for "trait.en" filled in, so all but record 3774 are missing that trait. I also haven't included that trait in an @ana or @role attribute as I couldn't remember if we had decided for sure to include a "#syriaca-anonymous" value for those.
3. The items in #22 have yet to be implemented.

Feel free either to post other issues you notice here or create separate issues for them.

[dlschwartz]

@wlpotter this is looking really good! Thank so much. I've noticed a couple of issues in my spot-checking.

• The dates are not compliant. I think the solution here is to make sure we put a compliant date format into the spreadsheet (yyyy-mm-dd). I should be able to write a regex to validate the data in columns U, V, and W unless you'd like to do that. Come to think of it, I'm sure there is a regex out there we can search for and steal.
• Also, check out record 3246. These should be one tei:citedRange instead of two. The transform seems to have worked correctly with 3215, 3229, and 3770 but not here with 3246.

Beyond these couple of quick things, I think the transform is working really well and we'll just need to wait for the schema updates to take things further.

Thanks Will!

[wlpotter]

@dlschwartz Thanks for these!

• For dates, I completely forgot to ensure it constructed the attribute with the correct, ISO format. We can implement input validation in the spreadsheet; the only trick will be to ensure that those cells get treated as string/text-type data rather than numerical data as spreadsheets tend to snip leading zeroes from numbers. Elsewhere, if I remember right, I implemented something similar in XQuery that parses dates and constructs ISO-compliant ones, so I'd be happy to add that check to the script if I can find it.
• For record 3246, that was a typo in the test spreadsheet I created. It was looking for "#"-separated characters in the cited range and unit fields, and I accidentally put "912-913##Probus 8", so it created an extra empty tei:citedRange. This is good to flag, though, as the script should either raise an error in such a case or ignore any 'empty' cited ranges.

I think once we sort out the few remaining decisions, and once we've implemented batch changes, this transform should be ready to go.

[wlpotter]

FYI, I should have mentioned this above, but the script was working on my adaptation of the Google sheet, here: https://github.com/wlpotter/csv-to-srophe/blob/main/in/test/persons_test.csv

[wlpotter]

Closing this as I've made issues for the dates, and everything else requires CSV data changes.