Fix bug with latin characters and ignore usernames optional

felmoltor commented 10 years ago

Not sure if you agree with them, but I added some changes to my fork that maybe are useful for you, if not, please ignore my pull request:

I've added a new option to optionally ignore all the usernames mentioned by an user. Sometimes the usernames are not useful in a wordlist.
To support non English tweets, I've added some code to avoid splitting words with accents or cyrillic characters into two words with nos sense (text.gsub!(/[^\w \s \d]/, ' ')), but instead transform to a word without the "offending" character.
Added new folder (ignore) to ignore special words non meaningful or useful to the objective of this script. This is a TODO feature.

crupler22 commented 7 years ago

I just wanted to point out that twofi is spelled twoif in the program in the help section.

digininja commented 7 years ago

@felmoltor finally got round to looking at your changes, started merging them with some changes. Doing the ignore list based on a file on the command line rather than the fixed lists, can also include the fixed lists for people to use if they want to. Check the ignorethings branch.

@crupler22 thanks for the typo comment. I managed to spell the name in two different ways when I launched the app, thought I'd got them all in sync but obviously missed some.

digininja / twofi

Fix bug with latin characters and ignore usernames optional #1