Updated crfcut.py

[varunkatiyar819]

Crfcut creating issues for split using terminal punctuation commonly '.' (full stop) which should be treated as end of the sentence, Modified the function such that it should split using terminal punctuations and avoid any kind of empty strings.

What does this changes

Brief summary of the changes

What was wrong

Description of what was the root cause of the issue.

How this fixes it

Description of how the changes fix the issue.

Fixes #904

Your checklist for this pull request

🚨Please review the guidelines for contributing to this repository.

• [ ] Passed code styles and structures
• [ ] Passed code linting checks and unit test

[pep8speaks]

Hello @varunkatiyar819! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

• In the file pythainlp/tokenize/crfcut.py:

Line 207:35: E226 missing whitespace around arithmetic operator

Comment last updated at 2024-04-03 07:51:42 UTC

[coveralls]

coverage: 79.063% (-0.04%) from 79.103% when pulling d1b64a7f36804a1ed71159c1ea3c23f628619eec on varunkatiyar819:varunkatiyar819-patch-1 into fa0a2caf00e3ec11562a4f9c7a5d69c13eebb8b3 on PyThaiNLP:dev.

[sonarcloud[bot]]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

[varunkatiyar819]

It look the test is break. Can you fix? @varunkatiyar819

['ฉันไปโรงเรียน เธอไปโรงพยาบาล'] != ['ฉันไปโรงเรียน ', 'เธอไปโรงพยาบาล']

Logic for the Code is been modified and tested as well. Please do have a check. Thanks