Added cloudflare bypass that introduces fast support for 9Anime

[justfoolingaround]

I've added an effective cloudflare bypass to the code which bypasses 9Anime's redirecting page. Now, the program can utilize faster modules like requests to scrape and decode content from their AJAX servers and reintroduce 9Anime to the supported providers list.

I've tested it using the NineAnime class with the _scrape_episodes call which seems to provide a 100% accuracy (after the validator was introduced in the second commit).

Closes #599

[AbdullahM0hamed]

1 - Uncomment https://github.com/justfoolingaround/anime-downloader/blob/bc88f35341ba496a0312aa61665673fb2ded2042/anime_downloader/sites/init.py#L36 2 - Once you've done that, you should notice that anime dl -u naruto -p 9anime gives ERROR: 401 Client Error: Unauthorized for url: https://9anime.to/search?keyword=naruto

[justfoolingaround]

1 - Done 2 - Oops, I kinda forgot about the miscellaneous calls on the programs (searching and scraping metadata) which was a massive oversight. That should be working now.

[AbdullahM0hamed]

It now errors out with ERROR: expected string or bytes-like object - this is apparently because self.url is None in _scrape_episodes for some reason that I can't figure out

[justfoolingaround]

I'm glad you didn't figure it out what was wrong or your reply would be something different. Well, that's fixed and uh, that.. error is going to haunt me for the rest of my life.

[AbdullahM0hamed]

It errored out, so I checked it - and one of the links was https://vidstreamz.online/embed/GQP495843Y93%C3%8D, which inevitably used the vidstream extractor because of:

https://github.com/justfoolingaround/anime-downloader/blob/8441962f8b6d9b491940daf2c32882e165e33a3b/anime_downloader/sites/nineanime.py#L20

I would suggest either: a) create a vidstreamz extractor, or b) pass over vidstreamz links in favour of the other sources (I mean, the link I mentioned doesn't even display anything in browser)

[justfoolingaround]

Aight, so, vidstreamz.online is just another site owned by Vidstream itself. I recently just fixed that in here because it was causing havoc in the 9Anime extractor. If the vidstream extractor can extract from Vidstream. It should be able to do so with that site.

I mean, the link I mentioned doesn't even display anything in browser

That's expected lol, accessing any embeds on server provided by 9Anime will require https://9anime.to/ as the referer.

This applies to most extractors in general. Most require the referrer to make it challenging for extractors to extract from them.

Also, the vidstream extractor in anime-dl seems like a gogo-cdn extractor (it's different than the site associated with VidStream url generated by 9Anime because that gogo-cdn has a AJAX while VidStream has a whole different info API).

Extractor class seems to have a headers option so I don't think it'll be that hard to fix that extractor and possibly the others too (Streamtape, Mp4upload, MyCloud). Can you test the url extracted from 9Anime with the StreamTape extractor? (I have no core knowledge of the code, so, I don't know how to use that "extracted" URL there) I'm asking this because StreamTape was already being yielded by the _scrape_episodes in the previous code and that might work.

If that succeeds, I'll force the yielding to just StreamTape. If not, oh boy, there's a whole lot of extractors that needs to be rewritten in the code; (dw about that though, the rewrites will possibly even fix the existing errors).

For now, I can suggest that a exception handling in the extractor call be added. If it fails, it simply switches to the other source (I mean, we have 4 yields right here). If all fail, we can raise an exception right there.

[shreyasajj]

Sorry what happened to this?

[ArjixWasTaken]

If this is not going to be worked on then it should be closed.