search

sillsdev / silnlp

A set of pipelines for performing experiments on various NLP tasks with a focus on resource-poor/minority languages.
Other
35 stars 3 forks source link

Book names not read properly, preventing parsing. #570

Closed davidbaines closed 3 weeks ago

davidbaines commented 3 weeks ago

The report in #379 mentioned JOL in the NRT23 I tested to see whether that would parse correctly, and it failed to read the Book names correctly.

F:\GitHub\silnlp>poetry run python -m silnlp.common.check_books NRT23 --books JOL
2024-10-21 18:08:17,230 - silnlp.common.environment - INFO - Using workspace: S:/ as per environment variable SIL_NLP_DATA_PATH. 2024-10-21 18:08:17,610 - silnlp.common.check_books - INFO - Found these books in the project_directory: Aru Bru Cru Dru Eru Gru Hru Iru Kru Lru Mru Nru Oru Pru Rru Sru Tru Uru Vru 2024-10-21 18:08:17,611 - silnlp.common.check_books - INFO - All books to check are: []

2024-10-21 18:08:17,611 - silnlp.common.check_books - INFO - No books were specified, will check all books. Traceback (most recent call last): File "C:\Python38\lib\runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "C:\Python38\lib\runpy.py", line 87, in _run_code exec(code, run_globals) File "F:\GitHub\silnlp\silnlp\common\check_books.py", line 159, in main() File "F:\GitHub\silnlp\silnlp\common\check_books.py", line 151, in main book_nums = get_chapters(books_to_check) File "C:\Users\David\AppData\Local\pypoetry\Cache\virtualenvs\silnlp-X71j0jgb-py3.8\lib\site-packages\machine\scripture\parse.py", line 153, in get_chapters selection_chapters = parse_selection(selection, versification) File "C:\Users\David\AppData\Local\pypoetry\Cache\virtualenvs\silnlp-X71j0jgb-py3.8\lib\site-packages\machine\scripture\parse.py", line 93, in parse_selection raise ValueError(f"{selection} is an invalid book ID.") ValueError: Hru is an invalid book ID.

isaac091 commented 3 weeks ago

Hi David, this looks like expected behavior to me, though I'm not sure where the [A-V]ru book names are coming from. Are you using a local projects directory? I expect this will work fine with the NRT23 project in the S3 bucket.

davidbaines commented 3 weeks ago

This is the NRT23 project in the S3 bucket I believe. My guess is that these are the last letters of the book names followed by the first two of 'post part'

GE[Nru]s EX[Oru]s etc.

isaac091 commented 3 weeks ago

Ok, I see the issue. The script doesn't account for the different book file name formats, but just uses sfm_file.name[2:5] to get the book names. It should be a quick fix using the machine settings parser.