search

digidem / osm-p2p-db

Peer-to-peer database for OpenStreetMap data
BSD 2-Clause "Simplified" License
236 stars 25 forks source link

Defaults - hyperlog #6

Closed gmaclennan closed 8 years ago

gmaclennan commented 8 years ago

Are there potential alternatives for hyperlog, or will this always be used? I am thinking we should make hyperlog a hard dependency and just pass the leveldb? If so we could just pass a single leveldb in the options and use sublevel? Are there performance gains from having two separate leveldbs?

Do we need opts.size or can this be read from the fd-store?

I'm thinking how we can reduce the complexity of the options.

ghost commented 8 years ago

You might want to customize a hyperlog with different settings during instantiation, such as using signing keys or encryption. If a user already has a hyperlog used for different purposes on the outside, they can also use it for osm-p2p-db, but we should probably have a sublevel-style wrapper for hyperlog to support that use-case better.

I think it's better to manage the versions of hyperlog from the outside, so this module doesn't need to be updated for every possible breaking change in hyperlog. However, I think the spirit of this issue is more that we could have a wrapper module that pulls in osm-p2p-db and some other unwritten libs like osm-p2p-media with good defaults to make it easier to get started. Perhaps an osm-p2p module that sets up everything, including signing keys for hyperlog.

ghost commented 8 years ago

This is all factored out into osm-p2p: http://github.com/digidem/osm-p2p

There were some surprisingly tricky parts to make this work without a callback, deferring the leveldb instances and the chunk store, but the resulting API is even friendlier.