digidem / mapeo-core

Library for creating custom geo data and syncronizing via a peer to peer network
23 stars 2 forks source link

Revise database directory structure #99

Open hackergrrl opened 3 years ago

hackergrrl commented 3 years ago

Our current disk layout is:

0/
1/
2/
...
media/
storage/
index/

I propose the following:

feeds/    <--- has 0/, 1/, 2/, etc
media/   <--- same
indexes/
--- views/   <--- used to be index/ (has all kappa view leveldbs)
--- raw/  <--- used to be storage/ (has all binary data like the bkd tree)

This will make it much easier to

Open question: how will we change over existing devices? They'd need to detect the old layout and do a rename in bulk, or we support the legacy format when the new directories aren't found, but prefer to use the new layout for new devices.

hackergrrl commented 3 years ago

cc @gmaclennan @okdistribute

gmaclennan commented 3 years ago

This makes sense to me. I think prior to initialization, checking for the old format somehow and moving directories would be best, then the bulk of the code does not have to worry about legacy format support.

okdistribute commented 3 years ago

+1 on this!