search

fstpackage / fsttable

An interface to fast on-disk data tables stored with the fst format
GNU Affero General Public License v3.0
27 stars 4 forks source link

Fsttable state should be memoryless #33

Closed martinblostein closed 6 years ago

martinblostein commented 6 years ago

In other words, if two fst tables represent the same subset of the same file, their proxy table state should be identical. (Due to internal self refs, the entire fsttable objects may not be identical.)

An example where this is currently violated is a selection, using i, where all rows are returned in their original order. I'm going to submit a patch to address that, but I think this principle is something to keep in mind as the interface is further extended and optimized.

MarcusKlik commented 6 years ago

Thanks for fixing this, a memory-less state is very important indeed! It could also help with future cases where we are comparing two table proxies or merging them (to determine beforehand if two columns are identical).