Closed Tikipeter closed 4 years ago
Nice work :), i have a question, not directly about your changes but about what may be an error that has been carried along from version to version, in lib/openscrapers/sources_openscrapers/_init_.py
with reference to
total_providers = {'en': hoster_providers, 'en_Debrid': debrid_providers,
Should the 'en_Debrid' be 'en_DebridOnly' ?
Whilst it should be 'en_DebridOnly' for consistency, it doesn't really matter because it looks like the only time the total_providers dictionary is used is directly under it when it is iterated through and the values are added to the 'all_providers' list. That list is then used in 'default,py' to access all the providers to toggle their settings on/off. So the key could have been called anything really, from what I can see it's never used. There were probably other plans for making the dictionaries, but they weren't implemented.
@Tikipeter been trying to test think PR but honestly havent had a chance, so far the major concern that Naze had was regarding the cookies for cf causing issues. What's your take on this before we proceed?
I remember Naze saying this (the cookies). It's hard for me to talk in definites because I don't know the workings of the cfscrape modules, but I have been using this code for the over 2 weeks and I've seen no ill effects.
If I were you, I would delete this pull request and accept host505's pull request. Then wait a day (I'll be off work then) and I can make a pull request for the free hoster sources I was able to fix (I've been waiting to see what happens here before making that pull request).
Then we can wait for Naze's views on this cfscrape change, and if he thinks it's alright I can re-make this pull request with the updated files.
Does that sound good?
@Tikipeter ok cool dude that sounds good then :) thanks for the swift response
Move cfscrape import to sources_openscrapers init.py file, then make an instance of it there. Sources can now import the instance directly, meaning large increases in speed when addons import all the sources (there had been a significant slowdown since cfscrape was last updated).
Update all scrapers that use cfscrape to import and use the instance from the new location.
addon.xml was missing the 'requests' dependency. Openscrapers was only working because the addons that use it as a dependency also had 'requests' as a dependency. Openscrapers needs to include this dependency itself.
Also, I had to come back to addons.xml and remove an 'a' I had put in the version number when testing.