The % sign is now accepted by French TN (this issue). This is generally implemented as part of the Measures class, which French currently lacks. As a stop-gap measure, the % was whitelisted. It should be properly implemented as part of Measures.
This PR does not address the following:
The issue with 2h not normalizing to deux heures in FR. Since the abbreviation h is subject to declension, this problem should be addressed with the implementation of the Measures class.
IMPORTANT!
This PR has not been tested for pytest or sparrowhawk as my machine doesn't have sufficient resources to run these tests for French. The the FR Normalizer has been run on a gamut of strings containing the % resulting in correct normalization. This fix needs testing before merging.
Before your PR is "Ready for review"
Pre checks:
[x] Have you signed your commits? Use git commit -s to sign.
[ ] Do all unittests finish successfully before sending PR?
1) pytest or (if your machine does not have GPU) pytest --cpu from the root folder (given you marked your test cases accordingly @pytest.mark.run_only_on('CPU')).
2) Sparrowhawk tests bash tools/text_processing_deployment/export_grammars.sh --MODE=test ...
[x] If you are adding a new feature: Have you added test cases for both pytest and Sparrowhawk here.
[x] Have you added __init__.py for every folder and subfolder, including data folder which has .TSV files?
[x] Have you followed codeQL results and removed unused variables and imports (report is at the bottom of the PR in github review box) ?
[x] Have you added the correct license header Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. to all newly added Python files?
What does this PR do ?
%
sign is now accepted by French TN (this issue). This is generally implemented as part of theMeasures
class, which French currently lacks. As a stop-gap measure, the%
was whitelisted. It should be properly implemented as part ofMeasures
.This PR does not address the following:
2
h not normalizing todeux heures
in FR. Since the abbreviationh
is subject to declension, this problem should be addressed with the implementation of theMeasures
class.IMPORTANT!
This PR has not been tested for
pytest
orsparrowhawk
as my machine doesn't have sufficient resources to run these tests for French. The the FR Normalizer has been run on a gamut of strings containing the%
resulting in correct normalization. This fix needs testing before merging.Before your PR is "Ready for review"
Pre checks:
git commit -s
to sign.pytest
or (if your machine does not have GPU)pytest --cpu
from the root folder (given you marked your test cases accordingly@pytest.mark.run_only_on('CPU')
). 2) Sparrowhawk testsbash tools/text_processing_deployment/export_grammars.sh --MODE=test ...
pytest
and Sparrowhawk here.__init__.py
for every folder and subfolder, includingdata
folder which has .TSV files?Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
to all newly added Python files?Copyright 2015 and onwards Google, Inc.
. See an example here.try import: ... except: ...
) if not already done.PR Type:
If you haven't finished some of the above items you can still open "Draft" PR.