search

peterGraf / pbl

PBL is an MIT License open source C-language library of functions that can be used in a C or C++ project. PBL is highly portable and compiles warning free on Linux gcc, MAC OS X and Windows Microsoft Visual C++ 2010 Express Edition.
http://www.mission-base.com/peter/source/
MIT License
40 stars 12 forks source link

PBLISAM, how to prevent automatic creation of multiple 256MB segments DB Data files.. #4

Closed mladinox closed 8 years ago

mladinox commented 8 years ago

Testing PBLISAM with 5.000.000 records, creates multiple DB files of 256MB size each..

How prevent this..? Can the entire DATA exist in just only one file..? the same for the INDEX file..?

I create one DATA file and one INDEX file with duplicates allowed, but the DATA file is splitted in multiple segment files when reach 256MB size..

Thank you..

peterGraf commented 8 years ago

When we wrote PBL, files bigger than 1GB were not possible, so we deciced to use 256MB as our split file size. Things have changed, but I still have a hard time to move files of a size of 1GB from one drive to another. So having a system that splits database files, you should treat them as opaque objects, into units of 256 MB still seems like a good choice. What problem do you have with PBL's internal split file size of 256 MB?

mladinox commented 8 years ago

Now days, moving files between devices are not an speed problem any more. But having care about many files and its sequences, is some problematic. Is required splitting files in PBL? Can i change the size of splitted files to an greater value, without impact in library design or performance?

peterGraf commented 8 years ago

I think that increasing the split file constant to any value below 7FFF FFFF should be ok, if you define it to a higher value, you should debug very carefully, no value above FFFF FFFF will be possible.