Closed yjkiwilee closed 2 weeks ago
These might have been created by dwca2csv.py (#6) as it removes HTML tags, but I could also try including a line in the prompt for desc2matrix.py to remove these errors.
It looks like these are erroneous spaces in the data. If you open the begonia-desc.txt file in a regular expression capable text editor you can search for the pattern basifi[^x]
ie the term that is split is "basifixed", the regular expression looks for the the first part of the split term "basifi" followed by anything that is not "x"
Running dwca2csv.py, some whitespaces are inserted in the middle of words. For example, in begonia-desc.txt, in line 775:
These then remain in the final json file produced by desc2matrix.py.