search

duckduckgo / p5-app-duckpan

DuckDuckHack OpenSource Development Application
http://metacpan.org/module/App::DuckPAN
Other
53 stars 47 forks source link

Offline work is possible? #321

Closed boozook closed 2 years ago

boozook commented 8 years ago

Is it possible to configure Duckpan for offline testing our great goodies & spices? Currently I always get update-error:

duckpan server
Checking for DuckPAN requirements...
Checking perl version... 
Checking for latest App::DuckPAN... 
[FATAL]  Cannot download http://duckpan.org//modules/02packages.details.txt.gz
moollaza commented 8 years ago

@fzzr- it's strongly suggested that you have an internet connection. DuckPAN downloads metadata and assets from DuckDuckGo relatively often so that everything is in sync.

We could try making that a non-fatal error but I'm not sure how well everything will work.

GuiltyDolphin commented 8 years ago

@moollaza I'm in favor of having some caching for the assets and metadata - with a reset of maybe... 1 hour or so? If we made it clear that this was happening (so people knew to reset the cache if need be) I think it would be fine. Also, I find a lot of dev time in DuckPAN is spent waiting for the downloads - which can be really painful!

Of course if it fails to download new data it could just fallback on the cache (allowing for offline use).

Edit: Hmm... Looks like it should be doing something similar to this already? Doesn't seem to do anything on mine though...

Edit 2: Weird... Sometimes the caching is fine, and it loads reasonably fast, but other times it takes up to a minute...

moollaza commented 8 years ago

Edit: Hmm... Looks like it should be doing something similar to this already? Doesn't seem to do anything on mine though...

@fzzr- @GuiltyDolphin currently DuckPAN does cache the assets, but I believe every hour or so it checks to see if the assets have updated. This requires a call to DuckDuckGo to check if the versioned assets on DuckDuckGo.com have changed. I thought 02pacakges.details.txt was also being cached, but I don't recall if it is right now.

The idea behind the caching was that previously every time the server started it would download all the necessary assets which obviously made things slower.

If you run duckpan -v server you should see it indicate when it's using the cached assets.

If you run duckpan -e -v server you can force the cache to be cleared, and you should see it download a new copy of all the required assets.

On startup I believe DuckPAN also checks to make sure you're using the newest version of DuckPAN and DDG, this is necessary because changes to the repos may require changes to DDG/DuckPAN and you could run into breakage if they aren't kept in sync.

DuckPAN does have a local .duckpan/config file. I suppose we could allow users to specify a cache time to allow DuckPAN to skip checking for updates for a longer period of time.

boozook commented 8 years ago

@moollaza I have .duckpan/config directory. It's empty. One question: how to get/export current default config into that dir (~/.duckpan/config/)?

moollaza commented 8 years ago

One question: how to get/export current default config into that dir (~/.duckpan/config/)?

@fzzr- there isn't really a default config right now. The config dir is just used to house the env.ini file that DuckPAN creates/uses when you use the duckpan keys command to add/renamed/delete API keys

boozook commented 8 years ago

:) Thanks. I see the key DDG_METADATA_TIMEOUT and will try use it soon.

GuiltyDolphin commented 8 years ago

@fzzr- You'll probably want to use NO_METADATA_DOWNLOAD if you're working off master - the DDG_METADATA_TIMEOUT is only on one of my branches AFAIK.

boozook commented 8 years ago

@GuiltyDolphin, thank you. Currently I use both of them - for case if in the future the NO_METADATA_DOWNLOAD will be legacy or deprecated (Or you will merge your branch into master).

GuiltyDolphin commented 8 years ago

@fzzr- I'll fight against deprecation of NO_METADATA_DOWNLOAD :wink:, I have connection issues too.

I don't think my branch will get merged, but if it does then sure, that var will work.

I'd recommend just having export NO_METADATA_DOWNLOAD=1 in your .bashrc, and hopefully that will have you sorted.

If we ever come back to changing the variable I'll ping you so you are in the loop.