search

RangerMauve / webrun

Run Web-first ESM modules outside of the browser
https://webrun-presentation.hashbase.io/
MIT License
77 stars 8 forks source link

DatArchive API only persists to memory #2

Closed tswast closed 6 years ago

tswast commented 6 years ago

By default node-dat-archive only writes to memory. It'd be nice if the polyfill wrote to .webrun/datcache or this was somehow globally configurable, like how there's a setting when you first run Beaker Browser where to save your dat archives (maybe via an environment variable such as WEBRUN_DATARCHIVE_DIRECTORY).

tswast commented 6 years ago

Also, it'd be nice to set latest: true for my use case in dat-pin-box because I want the pinned archives to be easily browsable.

RangerMauve commented 6 years ago

I'm actually using dat-node for the Dat interaction and it should already be persisting to .webrun/datcache :D

It doesn't write the actual files to those folders, but it writes the content and metadata feeds which can be used for fetching files from the archive. The reason it doesn't copy the "current version" to the FS is that it would result in duplicated content on disk.

tswast commented 6 years ago

Cool. I see that now. I think I was confused at first because all my calls to await archive.download('/') raise a TimeoutError.

{ TimeoutError: Timed out
    at Timeout.setTimeout [as _onTimeout] (/Users/tswast/.nvm/versions/node/v10.10.0/lib/node_modules/@rangermauve/webrun/node_modules/@beaker/dat-node/lib/util.js:21:14)
    at ontimeout (timers.js:425:11)
    at tryOnTimeout (timers.js:289:5)
    at listOnTimeout (timers.js:252:5)
    at Timer.processTimers (timers.js:212:10) name: 'TimeoutError', timedOut: true }

But, if I skip the download step, the next time I run it, I am indeed able to read the /pins.json file.

tswast commented 6 years ago

Deleting my .webrun directory and running again seems to have fixed the timeout errors. I'll close this. Thanks for the help.

RangerMauve commented 6 years ago

You know what, I think I saw this happen to me in Beaker! It seems that sometimes, if you already have some content downloaded, invoking .download() won't resolve and instead timeout. I tracked it down to somewhere in hyperdrive.

The most suspicious part is here. I'll open an issue in Beaker to see if it's a bug we're encountering with Dat itself.