search

iipc / openwayback

The OpenWayback Development
http://www.netpreserve.org/openwayback
Apache License 2.0
479 stars 272 forks source link

More scaleable CDX usage. #63

Open egh opened 10 years ago

egh commented 10 years ago

Shifting to CDX server? Use cases needed.

egh commented 10 years ago

Adding interested parties from spreadsheet. @anjackson @PsypherPunk @ikreymer

kris-sigur commented 9 years ago

Doesn't CDX server with ZipNum cluster CDXs solve this?

anjackson commented 9 years ago

IIRC from the Paris meeting, the issue was about how to manage and roll through large indexes with daily changes. But perhaps we should close this is no-one is clamouring for it.

kris-sigur commented 9 years ago

The WatchedCDXSource actually makes that scenario much easier as you can add and merge files without worrying about its effects on the configuration.

Yes, lets close this unless someone speaks up for this very soon.