search

dhardy / pippin

a distributed-merge capable database for many small objects
ISC License
78 stars 4 forks source link

File discovery #36

Closed dhardy closed 8 years ago

dhardy commented 8 years ago

Partition file discovery should be improved. The following should succeed in discovering the partition:

cargo run --example sequences -- -d samples/part-seq-small

where samples/part-seq-small is a directory containing data files for a single Pippin partition.

dhardy commented 8 years ago

Current mode-of-operation: partitions are discovered when discovering a repository, partition files are discovered when opening the partition (more precisely, when creating the PartIO).

Possible alternatives:

  1. Discover partition files when discovering the repository; create all PartIOs at this time or store all file names
  2. Require deterministic file names; as (1) but store only numbers
  3. Require deterministic file names; PartIO checks the disk every time it is asked which files exist (more up-to-date)

It would be possible to implement multiple options and let the user pick.

dhardy commented 8 years ago

Done: 0c0bb9d833f97f97f00e63c4ba51866287271ae6