Closed dextronomous closed 3 years ago
Thanks for trying it. I am amazed by the bPfile you created. Now there is no limit for bPfile or bloom file. It just need sufficient RAM when they will be used in the next script for searching.
I tested only 3 types of bloom writing Full, and Line by Line, and a Mix of these two
Although i think the error you get in bloom creation is due to the reading of entire bPfile. Entire bPfile Reading
I think, in such cases as yours with serious big Files, it needs to be modified to read and update everything within the function itself, line by line and not all together bPfile Reading function
Such case i did not anticipated earlier. But perhaps i will include it next time if needed.
Hi Iceland2k14, i had a bpfile 4500000000 created 250gigs it had 45000 chunks and did the 2_bloom.py got memory error, is there a limit to my bpfile, now i am at 1000 chunks just to see it that works,