search

dib-lab / kProcessor

kProcessor: kmers processing framework.
https://kprocessor.readthedocs.io
BSD 3-Clause "New" or "Revised" License
11 stars 1 forks source link

Replace Murmur with xxHash #34

Open mr-eyes opened 4 years ago

mr-eyes commented 4 years ago

According to this Benchmarking It's found that xxHash is way faster than MurmurHash 3 or any other alternative.

I think we should give it a try for speeding up the hashing.

ctb commented 4 years ago

I think options are good... is there a way to allow the hash function to be selected dynamically?

shokrof commented 4 years ago

We can let the user choose the hash function he prefers. is that what you mean?

ctb commented 4 years ago

yes

On Mon, Oct 28, 2019 at 05:16:54PM -0700, Mostafa Shokrof wrote:

We can let the user choose the hash function he prefers. is that what you mean?

-- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/dib-lab/kProcessor/issues/34#issuecomment-547201275 -- C. Titus Brown, ctbrown@ucdavis.edu

mr-eyes commented 4 years ago

@ctb There're multiple modes for hashing selected by the user in a higher abstraction level than the "hashing" terminologies.

Mode 1 (Irreversible): Will automatically select the "Murmur" or the "xxHash" after this update. Mode 2 (Reversible): Will pick the "Integer" hashing function for reversing back the hash value to kmer string. Mode 3 (No Hashing) Will just store the Two Bits Representation of the kmers for the purpose of sorting it lexicographically.

ctb commented 4 years ago

excellent