search

sergiotapia / magnetissimo

Web application that indexes all popular torrent sites, and saves it to the local database.
MIT License
3k stars 190 forks source link

Crashes because it got 403 #68

Closed gjedeer closed 6 years ago

gjedeer commented 7 years ago

I installed Magnetissimo today by clicking Deploy to Heroku. Did not tweak any settings. Just a few hours later I'm getting Heroku's Application Error page. The log suggests that the program got a 403 Forbidden from mononova and it crashed because of that.

2017-02-14T16:43:03.665578+00:00 app[web.1]: Downloading page: https://monova.org/latest?page=1
2017-02-14T16:43:03.795084+00:00 app[web.1]: 16:43:03.794 [info] Torrent saved to database: Tuba Skinny - 2014 - Owl Call Blues [FLAC]
2017-02-14T16:43:03.806473+00:00 app[web.1]: 16:43:03.806 [info] Torrent saved to database: Traffic - 1974 - When The Eagle Flies (2003 Reissue) [[email protected]]
2017-02-14T16:43:03.827054+00:00 app[web.1]: 16:43:03.826 [info] Torrent saved to database: Traffic - 1974 - When The Eagle Flies (2003 Reissue) [FLAC]
2017-02-14T16:43:03.837873+00:00 app[web.1]: 16:43:03.837 [info] Torrent saved to database: Taken.Heart.2017.HDRip.XViD-ETRG
2017-02-14T16:43:03.851448+00:00 app[web.1]: 16:43:03.850 [info] Torrent saved to database: 30.For.30.S03E19.This.Was.The.XFL.HDTV.x264-YesTV - [SRIGGA]
2017-02-14T16:43:03.902476+00:00 app[web.1]: 16:43:03.902 [error] GenServer #PID<0.398.0> terminating
2017-02-14T16:43:03.902483+00:00 app[web.1]: ** (WithClauseError) no with clause matching: {:ok, %HTTPoison.Response{body: "<html>\n<head><title>403 Forbidden</title></head>\n<body bgcolor=\"white\">\n<center><h1>403 Forbidden</h1></center>\n<hr><center>nginx</center>\n</body>\n</html>\n", headers: [{"Date", "Tue, 14 Feb 2017 16:43:03 GMT"}, {"Content-Type", "text/html; charset=utf-8"}, {"Transfer-Encoding", "chunked"}, {"Connection", "keep-alive"}, {"Set-Cookie", "__cfduid=d366f3c916a5580007c2e1485b41c40b31487090583; expires=Wed, 14-Feb-18 16:43:03 GMT; path=/; domain=.monova.org; HttpOnly"}, {"Server", "cloudflare-nginx"}, {"CF-RAY", "3311fa14de822975-DUB"}], status_code: 403}}
2017-02-14T16:43:03.902488+00:00 app[web.1]:     (magnetissimo) lib/crawler/helper.ex:15: Magnetissimo.Crawler.Helper.download/1
2017-02-14T16:43:03.902489+00:00 app[web.1]:     (magnetissimo) lib/crawler/monova.ex:36: Magnetissimo.Crawler.Monova.process/2
2017-02-14T16:43:03.902490+00:00 app[web.1]:     (magnetissimo) lib/crawler/monova.ex:25: Magnetissimo.Crawler.Monova.handle_info/2
2017-02-14T16:43:03.902491+00:00 app[web.1]:     (stdlib) gen_server.erl:615: :gen_server.try_dispatch/4
2017-02-14T16:43:03.902491+00:00 app[web.1]:     (stdlib) gen_server.erl:681: :gen_server.handle_msg/5
2017-02-14T16:43:03.902492+00:00 app[web.1]:     (stdlib) proc_lib.erl:240: :proc_lib.init_p_do_apply/3
2017-02-14T16:43:03.971042+00:00 heroku[router]: at=error code=H13 desc="Connection closed without response" method=GET path="/summary" host=gdrsmagnetissimo.herokuapp.com request_id=f2312591-9acf-4e80-ad0a-1be2bfeb1bdc fwd="89.69.141.112" dyno=web.1 connect=0ms service=998ms status=503 bytes=0
2017-02-14T16:43:04.336193+00:00 app[web.1]: 16:43:04.332 [info] Application magnetissimo exited: shutdown
2017-02-14T16:43:05.883369+00:00 app[web.1]: {"Kernel pid terminated",application_controller,"{application_terminated,magnetissimo,shutdown}"}
tchoutri commented 6 years ago

Hi @gjedeer! Is this behaviour still observed?

gjedeer commented 6 years ago

@tchoutri No idea, I don't even have heroku toolbox installed any more.

sergiotapia commented 6 years ago

We'll re-open if this still occurs.

ThatLurker commented 6 years ago

Is there any delay on the monova crawler?

tchoutri commented 6 years ago

Oki doki :) I'll close this issue then