Modify USJ spec - Githubissues

Bridgeconn / usfm-grammar

An elegant USFM parser.

https://usfmgrammar.vachanengine.org/

MIT License

36 stars 14 forks source link

Modify USJ spec #226

Closed kavitharaju closed 10 months ago

kavitharaju commented 10 months ago

Closes #225 Implementing the changes suggested in USJ schema by the USFM/X Committee

Change oneOf condition to anyOf
Instead of the field {type: "val1:val2"} keep two fields {type:"val1", marker: "val2"}

Changes in this PR

[x] Update the schema definition at schemas/usj.js
[x] Change the USJ generation scripts in python usfm-grammar module
[x] Change the List generation scripts the module, that depends on the USJ output
[x] Change the code in filtering of USJ based on include_markers and exclude_markers
[x] Change the code where USFM is generated back from USJ
[x] Change scripts in tests where the output USJ is worked on (to check if all markers are present, unwanted markers are filteres etc)
[x] Update the origin-usj.json samples in USFM-Grammar testsuite, (which is a copy of the tcdocs/tests)
Tested and fixed issues found

joelthe1 commented 10 months ago

The changes look good. Are you planing to add the change to Book Code regex separately (basically to mirror what's followed in USX)?

kavitharaju commented 10 months ago

Honestly I had forgotten that change. But now I remember what we had decided was

To keep checking strictly with enum in usfm-grammar at our grammar level(no change needed for that)
In the USJ schema keep as any 3 letter string using regex. I had sent a pr to tcdocs repo with the changes already in this pr and it is merged. So how about I make the json schema change as next task?

About point 2, is that okay? Or should we do it just like in usx by listing all codes as enum and then opening it up for any 3 letter string? I see the following reasons to not to

the listing of enums would have no effect programmatically as the validation will solely be based on the regex
our schema for USJ is a very high level one, as of now. Which has its beauty and simplicity. (Mark was planning on defining one with detailed rules of which node is valid where). In the current one, if we are to add such along list and treat one field with special validations, there might be others rules also we would need which are there in USX and USFM

joelthe1 commented 10 months ago

Agreed. I will merge this in and we can work on the 'regex for book code' change as a separate PR. Also, let's leave out the special schema handling for now- enforcing any three letter regex is probably good enough (probably using what USX specifies already).