search

whitfin / runiq

An efficient way to filter duplicate lines from input, à la uniq.
MIT License
204 stars 23 forks source link

Don't allocate/process strings; just work on the bytes #2

Closed lespea closed 5 years ago

lespea commented 5 years ago

Much faster

whitfin commented 5 years ago

Hi @lespea!

Do you have any benchmarks that reflect the speed improvements here? At a glance, this looks good. But I don't have any time until the weekend to review fully!

I'm not sure about the reading yet; could you maybe strip this PR down (for now) to just the storing of Vec in the structs? Basically your changes to the filters module for now.

Once that's merged, we can take a look at actually migrating to the new reading; I have some code locally that I've been working on related to #1 which should work well with your changes.

whitfin commented 5 years ago

Hi @lespea!

I've incorporated this idea into the latest changes in master, so I'm going to close this. Thanks again for the initial PR!

lespea commented 5 years ago

Weird I never got notified about your previous question; sorry for not responding! Anyway glad to see the changes!!