Closed KingMarth closed 4 years ago
Yes, the server shutdown, and because I do not own it (it belong to a friend of mine) I need to reach him to understand why.
I did some investigation into cloud hosting. Unfortunately, it's a little tricky to fit the site into free hosting: NightmareJS works by starting up a browser (which is why you need xvfb), and mongo storage either requires another instance or more memory than a free VM allows, at least when using general-purpose hosting. I put together a quick temporary link using Docker images, but that consumes free trial credit.
There's a free-tier mongo option with Heroku, though Heroku doesn't support xvfb anymore since you can run Chromium in 'headless' mode without needing a display. There's a couple of options when switching from Nightmare, chromeless
matches the API closely but isn't supported anymore, while puppeteer
is the recommended package but the crawler would need some rewriting for their Promise-based API. Some tweaking will be needed either way, innerText is finicky between browsers about whether it preserves newlines.
I hacked together something that sort of works, I'll have to look into cleaning it up.
I've migrated the code from Nightmare to pupetter on the branch pupetter, you can pull it for testing purpose. I fixed the innerText issue.
For hosting, my friend has issue with the server and is not sure he want to take another one. I'll see next week if I can have a free hosting. If not the cheapest plan is to take a VPS on an hosting platform like ovh for 3 € a month. I can take it if needed but i'd like to try search first for free hosting.
I finally took a vps on OVH. The website is up again \o/
http://hexagrid.xyz isn't loading for me (with ERR_CONNECTION_TIMED_OUT) or esmecompost. Something happen?