Fix USFM parsing issues

sillsdev / machine

Machine is a natural language processing library for .NET that is focused on providing tools for processing resource-poor languages.

MIT License

26 stars 15 forks source link

Fix USFM parsing issues #228

Closed Enkidu93 closed 1 month ago

Enkidu93 commented 1 month ago

After running the manual tests on all SF projects, here are the 5 that failed, @ddaspit, @johnml1135:

[Removed traces - contact me if needed]

ddaspit commented 1 month ago

@Enkidu93 Where can I access the projects that failed?

Enkidu93 commented 1 month ago

I can send you them if you like. Do you want to take this issue, @ddaspit ? (It's next on my plate after I finish up the file deletion issue).

ddaspit commented 1 month ago

You can go ahead and take this issue. In the past, the underlying USFM parsing issues have not been obvious and the surface-level cause was often a red herring. I just want to be able to verify the core issues against the failing projects.

Enkidu93 commented 1 month ago

Update:

I've added a new exception to give us/SF better insight into where the USFM parsing failed (in an upcoming PR)
The System.ArgumentOutOfRangeException are the result of an off-by-one error in the UsfmTextUpdater class (also in an upcoming PR)
I'm currently investigating the stack issue. I'll probably hold out and submit a single PR with all these changes since it's moving along quickly enough.