Open atlowell-smpl opened 4 years ago
I would love this. I'd love to be able to replace something like: [?] with an empty space instead.
I think @atlowell-smpl's use case is something that falls outside of Unidecode's concern. It's easy to do such string replacements yourself before a call to Unidecode.
I agree that the characters that get replaced with a [?]
are a problem. It's something that #53 already tried to address (reading my comments there I think I initially misunderstood the goal of that pull request).
Before code for that can be added the character tables first need to be updated and unknown characters marked (probably with None instead of [?]
)
I've implemented the feature to specify custom replacement strings for characters that are unknown to Unidecode. For example, this will use ASCII space to replace characters that are not are not present in Unidecode's tables.
>>> unidecode('[\ue000]', errors='replace', replace_str=' ')
[ ]
This is a feature request to add a library function that can be called to set custom character replacement mappings. The library function could be called in a similar manner to as follows:
after which subsequent calls to unidecode would replace '™' with '(TM)' rather than the default of "(tm).
For my particular use case, this would be useful for standardizing names across data sources. For instance, if one source has the name "Acme™" and the other has the name "Acme", I could make a call to replace all instances of '™' with "", which allows me to easily compare across sources.
However, I could see a large number of other use cases for this as well. For example, in the FAQ, it is mentioned that localization is not supported and that certain german characters translate to english text rather than german phonetics. Having a call like this could allow programmers to make their own "localization" libraries, allowing them to whatever mappings they feel like.