search

voyanttools / trombone

GNU General Public License v3.0
3 stars 2 forks source link

Detect and convert ANSI texts #16

Closed pbstudent closed 1 month ago

pbstudent commented 2 years ago

bullet interpreted as ¥

Sinclair, S. & G. Rockwell. (2022). Contexts. Voyant Tools. Retrieved September 4, 2022, from https://voyant-tools.org/?query=resources&corpus=e44f6aba0a6c711b581e316dbff2ad0d&view=Contexts

services • identify human and other resources services ¥ identify human and other

ajmacdonald commented 2 years ago

That's odd. Can you please describe the file format and encoding of the source document (or email a copy to voyanttools @ gmail)?

pbstudent commented 2 years ago

Since the problem you identified was the ANSI plain text encoding instead of UTF-8, is there a way Voyant Tools could automatically recognize a plain text file and convert to UTF-8 ?

ajmacdonald commented 2 years ago

See: https://tika.apache.org/1.28/api/org/apache/tika/parser/txt/CharsetDetector.html

ajmacdonald commented 11 months ago

Note to self: add option under Processing to force a particular encoding.

pbstudent commented 11 months ago

Thanks Andrew.

As promised a long time ago I am providing my URL to the bibliographic list discussing Voyant Tools:

https://dt.athabascau.ca/jspui/handle/10791/419 https://dt.athabascau.ca/jspui/handle/10791/419

It is not a great work but sufficed under my health conditions. Fortunately Voyant Tools helped fill the gap.

Typical responses were that Voyant Tools was complicated. I disagreed. The software is not complicated to use, rather it disguises the algorithms that make the corpus visualized. Perhaps more discussion in an advanced section of the help site for each tool could provide the supporting algorithms. I found no other application on the Internet that could break down a corpus so nicely as Voyant Tools.

Thank you.

Cheers, Steve

On Dec 20, 2023, at 18:45, Andrew MacDonald @.***> wrote:

Note to self: add option under Processing to force a particular encoding.

— Reply to this email directly, view it on GitHub https://github.com/voyanttools/trombone/issues/16#issuecomment-1864886506, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFZHF65JPBCIYY7QIBARBDDYKMP4ZAVCNFSM6AAAAAAQG56Z22VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRUHA4DMNJQGY. You are receiving this because you authored the thread.

ajmacdonald commented 10 months ago

Thank you very much Steve. I will add it to the Gallery when that gets updated.

ajmacdonald commented 1 month ago

https://github.com/voyanttools/Voyant/commit/86832f37e15200b24baea21cb0c979e728d30c67