a4k-openproject / script.module.openscrapers

OpenScrapers Project
GNU General Public License v3.0
102 stars 40 forks source link

Move cfscrape import, update scrapers that use cfscrape, other #122

Closed Tikipeter closed 4 years ago

Tikipeter commented 4 years ago

Also, I had to come back to addons.xml and remove an 'a' I had put in the version number when testing.

bjbca commented 4 years ago

Nice work :), i have a question, not directly about your changes but about what may be an error that has been carried along from version to version, in lib/openscrapers/sources_openscrapers/_init_.py with reference to total_providers = {'en': hoster_providers, 'en_Debrid': debrid_providers, Should the 'en_Debrid' be 'en_DebridOnly' ?

Tikipeter commented 4 years ago

Whilst it should be 'en_DebridOnly' for consistency, it doesn't really matter because it looks like the only time the total_providers dictionary is used is directly under it when it is iterated through and the values are added to the 'all_providers' list. That list is then used in 'default,py' to access all the providers to toggle their settings on/off. So the key could have been called anything really, from what I can see it's never used. There were probably other plans for making the dictionaries, but they weren't implemented.

reddit-reaper commented 4 years ago

@Tikipeter been trying to test think PR but honestly havent had a chance, so far the major concern that Naze had was regarding the cookies for cf causing issues. What's your take on this before we proceed?

Tikipeter commented 4 years ago

I remember Naze saying this (the cookies). It's hard for me to talk in definites because I don't know the workings of the cfscrape modules, but I have been using this code for the over 2 weeks and I've seen no ill effects.

If I were you, I would delete this pull request and accept host505's pull request. Then wait a day (I'll be off work then) and I can make a pull request for the free hoster sources I was able to fix (I've been waiting to see what happens here before making that pull request).

Then we can wait for Naze's views on this cfscrape change, and if he thinks it's alright I can re-make this pull request with the updated files.

Does that sound good?

reddit-reaper commented 4 years ago

@Tikipeter ok cool dude that sounds good then :) thanks for the swift response