UltimaHoarder / UltimaScraper

Scrape all the media from an OnlyFans account - Updated regularly
GNU General Public License v3.0
3.96k stars 605 forks source link

Multiple Blacklists. #1476

Closed cjemorton closed 3 years ago

cjemorton commented 3 years ago

What is the proper syntax for including multiple blacklists.

"blacklist_name": "Blacklist",

ghost commented 3 years ago

Have you even tried a few combos like "blacklist1, blackist2, blacklist3, etc...."

If that does not work just copy your scraper, edit the config to point to your first "installation" and use your second blacklist in that.

jp1151419 commented 3 years ago

Have you even tried a few combos like "blacklist1, blackist2, blacklist3, etc...."

That syntax didn't work

cjemorton commented 3 years ago

I have tried multiple syntaxes, I would not have asked if I had found the correct one.

Off the top of my head; I have tried.

"blacklist_name": "Blacklist1", "Blacklist2",

"blacklist_name": "Blacklist, Blacklist2", 

"blacklist_name": ["Blacklist", "Blacklist2"],

I've tried with single quotes, brackets, curly braces.

There's probably more that I tried and I'm forgetting - I've probably tried a few that are not valid json, as I believe the whole file is a json file. Therefore it's probably a key value pair. I don't know enough json to know the correct syntax, nor could I find anything with the limited knowledge and correct search terms to find something in a google search.

Which is why I asked.

I believe, it is probably hardcoded to be one list only.

LunarPenguin commented 3 years ago

I have had a look at the code and done some personal testing and i came to the conclusion that at the moment it seems it is impossible to use multiple blacklist due to the way blacklists are checked.

https://github.com/DIGITALCRIMINAL/OnlyFans/blob/16b5739e9a828aa6ecd817bd9c1a8c1ae72791f9/modules/onlyfans.py#L1264-L1272

as you can see in the code snippet the script only checks if there is a blacklist with a string from blacklist_name present in the response list, it then sets this value from response as a value in the new_results list. It seems there has been some thought here about using multiple blacklist so this must be a simple oversight.

But, even if it had correctly recognized the multiple blacklists it seems it only checks the first element of the new_results list as demonstrated by the line item = new_results[0]

@DIGITALCRIMINAL should be able to fix this rather quickly i think as most of the groundwork has been done already

I don't know enough json to know the correct syntax,

As usual google is your friend and a site like https://jsonlint.com/ saves you from pulling your hairs out troubleshooting

UltimaHoarder commented 3 years ago

Thanks, you can now add multiple blacklists, atm it only supports python lists https://github.com/DIGITALCRIMINAL/OnlyFans/commit/f2472c6c58a1a69ddf658b7ea0f8bb8f56afe3a0

cjemorton commented 3 years ago

Nice! Really appreciate the effort you guys put into this. Thank you.

I just pulled down the new code to test it out; my initial impressions it seems to be working.

For future reference and for anyone looking to use this: In the .config JSON file it looks like this.

"blacklists": [ "List1", "List2", "List3" ],

I will do some more testing for my own verification purposes.

Quick observation: Content from a performer that is paid for is still downloaded even if they are on one of the ignored lists. (to me this is desired behavior but something to be aware of).