search

RangerMauve / dat-store

A CLI tool for backing up hyperdrive datasets
GNU Affero General Public License v3.0
59 stars 14 forks source link

dat add resolves domain’s Dat key instead of adding the domain as-is #23

Closed da2x closed 4 years ago

da2x commented 5 years ago
  1. $ dat store run-service --storage-location dat-dir/ &
  2. $ dat store add dat://example.com
  3. $ ls dat-dir/

Actual: seeing the Dat key as it was resolved at the time the domain was added.

Expect: to see example.com in the Dat storage directory listing.

If the domain’s Dat key changes then Dat store should start using the new key. Storing the raw resolved Dat key is only expected when adding a raw Dat key directly. Humans reference domains because they’re easy to remember and they can be updated to point to new locations.

Note: the domain should be resolved by the add command to verify that it has a Dat key and provide errors (good UX). After that then the domain should be re-resolved periodically (the DNS datkey records’ TTL or the .well-known/dat TTL knows how long the key is valid before it should be refreshed – see also this issue. Updating once a day or even on startup is probably good enough for initial support if the TTL is difficult to get.)

RangerMauve commented 5 years ago

Oof, this'll be a bit of a pain to handle when there's updates to the key. 😅

da2x commented 5 years ago

The upcoming Hypercore v10 migration makes an issue like this more important. (If the transition will be going to require everyone getting new keys and dat archive addresses.)

dat-store scans the directories on startup, right? It’s probably okay to just verify that a domain’s key is up to date on startup for now. When a different key from what is stored locally is detected then delete the local directory contents and add it again using the new dat key.

I don’t think dat-store should have this behavior when adding a directory with a dat archive from outside the storage-location directly, but only when adding a dat domain URL. So the above behavior when dat store add dat://example.com but not dat store add /some/dir/with-a-.dat/.

The same issue tracked for Beaker in issue #1403.

RangerMauve commented 4 years ago

I think this is working now? DNS is currently in flux with the latest Beaker version. Not entirely sure what their plans are yet but it looks like they're gonna roll their own.