Update registry more often

[jnordberg]

Automatic daily/weekly refreshes would be good, i think now it's something like 2 months old

[eirikb]

Hi. It seems my token for github didn't work, so the job would be killed before it finished (would take more than 2 days to run, without a valid token). Trying to fix this now

[jnordberg]

:+1:

[eirikb]

It turns out my script has suddenly become completely foobar. I'm looking at it, probably got it up and running within the weekend. Sorry for the long delay

[dandv]

+1. Registry is quite out of date by now.

[eirikb]

Ok so I just moved away from my crappy node.js-script to a whole over-engineering-delux-solution with Queues, Web Workes and Databases and God knows what. It does seem to help for the moment, but will be flaky in the time to come, but at least I get full monitoring and feedback.

I also moved the json-file out, so I don't have to rely on pushing it. I can move back to pushing, but it really makes everything a tiny bit more difficult.

I will leave the issue open until I can see that everything is working out as planned.

[eugeneware]

Hi @eirikb - I noticed that nipstr.com data hasn't been updated since 10/13. How often does the data get updated? I love your service BTW, npm inc should just use your method to help people find useful modules! Keep up the great work!

[eirikb]

Hi @eugeneware, thanks for the heads up.

It is set to update every day. It's done through Azure workers, but I ran out of credits around that date. Totally forgot about it. Although now it should be up and running again (get new credits every month), will check later today.

[eugeneware]

Hi @eirikb Thanks for the update. It's still showing the last update as 2014-10-13?

[eirikb]

Hi. Haven't had time to look at it. Or I have, and it turns out the service just wasn't running, so I wanted to find out what had happened. But no time, just re-published the project now, should be updated quite soon with older data at first. The service for fetching new data runs at the same time and takes a while longer.

[eirikb]

The new worker-solution seemed to have been working fine. But for some reason it just became very expensive, so I have shut it down.
Will have to find out why it's expensive, then take it up again.

[eugeneware]

Good luck @eirikb! If cost is becoming an issue perhaps we could run it at digital ocean (only $10 a month for a pretty decent server)?

[eirikb]

Thanks :). Maybe a VPS or similar would work fine. I'm currently hosting it on Azure, where I get ca $125 a month of usage (for free).
I have a lot of other stuff on there also consuming credits, but it was running smoothly before, and wasn't really taking up many resources. I've overstepped my limit before, but never because of nipster.

[eugeneware]

Hi @eirikb the data in nipstr.com is now over a month old. Have the updates stopped?

[eirikb]

Hi, I actually haven't gotten around to starting the worker up again after last time resource overuse. I wanted to investigate the error before firing it up, but until I can get around to actually do that perhaps it would be better to just have it running as-is.

[eugeneware]

If it's a funding issue, I'd be happy to contribute financially, or even to provide a digital ocean instance to host this if it helps?

[eirikb]

Closing, ref #21