search

jermp / pthash

Fast and compact minimal perfect hash functions in C++.
MIT License
190 stars 25 forks source link

-i can't read from a non-mmap-friendly input file when using --external #18

Closed zacchiro closed 10 months ago

zacchiro commented 10 months ago

It is handy to pass "streaming" content as input to build -i, e.g., by using shell process substitution like this: build -i <(zstdcat foo.txt.zst).

Unfortunately that does not work when --external is used. Apparently in that case (and only in that case) the input file passed to -i is mmap-ed, which obviously fail when the input file is not a regular file (a FIFO, in the case of process substitution).

Would it be possible to either avoid mmap-ing the input file even when --external is used or, alternatively, to have an option to inhibit that selectively?

Thanks!

jermp commented 10 months ago

Hi @zacchiro, now it should be done, as of https://github.com/jermp/pthash/commit/0d09102915064e4ca74496a136619b9aa4e62a2e. Now, if you specify --external it does not use mmap anymore by default. To enable mmap, use --mmap.

Best, -Giulio

zacchiro commented 10 months ago

Confirmed, it works for me now. Thanks! Closing the issue.