search

datawhores / OF-Scraper

A completely revamped and redesigned fork, reimagined from scratch based on the original onlyfans-scraper
MIT License
699 stars 59 forks source link

# Script running but not downloading files #448

Closed realman96 closed 3 months ago

realman96 commented 3 months ago

Describe the bug

When I run the script, it scrapes the user profile and recognizes the correct number of images/videos/posts. But when the script finishes succesfully with no skips, the files are not found and nothing is downloaded. I get a 'precheck failed' message at the top of the log.

To Reproduce

ofscraper --perform actions -- download content from a user-- profile, archived, messages --

Expected behavior

Profile posts, archived posts, and messages would all be downloaded

Screenshots/Logs

https://privatebin.io/?1a31c8e055fc826e#AgwxvorYCsjf4GVLzNF3VFuzXVTTWUFLDm9tE2iSuPDU

Config

{ "main_profile": "main_profile", "metadata": "{configpath}/{profile}/.data/{model_id}", "discord": "", "file_options": { "save_location": "C:\Users\-----\Data\ofscraper", "dir_format": "{model_username}/{responsetype}/{mediatype}/", "file_format": "{filename}.{ext}", "textlength": 0, "space_replacer": " ", "date": "MM-DD-YYYY", "text_type_default": "letter", "truncation_default": true }, "download_options": { "filter": [ "Images", "Audios", "Videos" ], "auto_resume": true, "system_free_min": 0, "max_post_count": 0 }, "binary_options": { "ffmpeg": "C:\Users\------\.config\ofscraper\bin\ffmpeg.exe" }, "cdm_options": { "private-key": null, "client-id": null, "key-mode-default": "cdrm", "keydb_api": "" }, "performance_options": { "download_sems": 6, "thread_count": 2, "download_limit": 0 }, "content_filter_options": { "block_ads": false, "file_size_max": 0, "file_size_min": 0, "length_max": null, "length_min": null }, "advanced_options": { "code-execution": false, "dynamic-mode-default": "digitalcriminals", "backend": "aio", "downloadbars": false, "cache-mode": "sqlite", "appendlog": false, "custom_values": null, "sanitize_text": false, "temp_dir": null, "remove_hash_match": null, "infinite_loop_action_mode": false, "enable_auto_after": true, "default_user_list": "main", "default_black_list": "" }, "scripts": { "post_download_script": null, "post_script": null }, "responsetype": { "timeline": "Posts", "message": "Messages", "archived": "Archived", "paid": "Messages", "stories": "Stories", "highlights": "Stories", "profile": "Profile", "pinned": "Posts", "streams": "Streams" }, "overwrites": { "audios": {}, "videos": {}, "images": {}, "text": {} }, "ffmpeg": "C:\Users\------\.config\ofscraper\bin\ffmpeg.exe" }

System Info

Additional context

Add any other context about the problem here. Apologies if this is a noob question, I'm unfamiliar with this kind of stuff

RoboSquc commented 3 months ago

I'm having the same issue. Even when using manual mode to download a single purchase, it doesn't download anything.

Windoze345 commented 3 months ago

Got the same issue. Probably OF has changed something on their end...

RoboSquc commented 3 months ago

Just updated OFscraper to 3.11.4 and it seems to be working again

realman96 commented 3 months ago

Still getting this issue even after update to 3.11.4 :(

kazin8 commented 3 months ago

Hi! I don’t use this library for scraping, but I found a solution. OnlyFans now has a new header for CDN2 called “X-Hash.” You can generate this hash here: https://cdn2.onlyfans.com/hash/. After generating the hash, you need to add this header to all your requests: X-Hash: xxxxxx (where xxxxxx is the result from the /hash/ request). You can generate the hash without any authentication.

For now, the behavior seems a bit strange, but at least it works. I’m not sure about its reliability, so I’ll keep an eye on it.

kazin8 commented 3 months ago

Oh, and the full URL is now located in media.files.full.url, but it seems that this has already been addressed in release 3.11.4.

realman96 commented 3 months ago

Assuming I'm a total idiot when it comes to Python...how would I actually add that header to requests? 😅I have the hash, but not sure how to include it when using the tool

datawhores commented 3 months ago

Most issues should be fixed in 3.11.6 if not please send a log so I can see what exactly your issue is

realman96 commented 3 months ago

Upgraded to 3.11.6, but still seeing issue- script runs, and it can see the amount of files available, but 0 files downloaded. Pasting PB log here. I'm sure I'm doing something wrong 😂

https://privatebin.io/?24e0dca1f5828fa7#FjNRuf8W1ga1a65HwH7ubyn9SecszUZse7mt9BqgNzaK

BathingGorilla commented 3 months ago

Same issue here

datawhores commented 3 months ago

seems like it just getting caught by the dupe filter can you do ofscraper --after 2000 --dupe

realman96 commented 3 months ago

No dice. Tried appending, it only downloads the profile picture and header picture and no posts/photos/content. Wonder if it's something to do with the profile itself, but it has lots of posts...?

datawhores commented 3 months ago

Can you show a new log file

realman96 commented 3 months ago

New log file here https://privatebin.net/?a679c1d76e8b9b0f#GVbrpexWUDUZCx1bLFqXvvM9hDiWvqZ1cFRRNEmmdaEw

datawhores commented 3 months ago

It seems like it did download based on the log

Just not sure why the count is wrong at the moment

realman96 commented 3 months ago

Not sure what the issue is- the directory to where those files save (C:\Users\your_username\Data\ofscraper -----\Profile\Images) is empty, save for the profile picture and cover photo from the account. No other files have been downloaded despite there being 100+ on the account. It succesfully downloads those two files, but the hundred others aren't coming through 🤔

datawhores commented 3 months ago

Are you subscribed to this person ,and is there content unlocked for you

datawhores commented 3 months ago

I'm getting signals that the answer is no

datawhores commented 3 months ago

From checking the profile You can not scrape without being subscribed

It is not possible to download media anonymously

realman96 commented 3 months ago

Was previously subscribed, content not currently unlocked. I thought that was a functionality- to scrape previously unlocked/subscribed content, but I may have misunderstood. Apologies if I've wasted your time debugging a non-issue!!

datawhores commented 3 months ago

The download links are provided by s3 and change maybe once a day, and have randomly generated keys for security

realman96 commented 3 months ago

Got it. Thanks for looking into it regardless

original-copy commented 3 months ago

Version: 3.11.6+.d0aa160b

Same problem here. Tried "ofscraper --after 2000 --dupe" & "ofscraper --after 2000 --force-all" but apart from the profile picture, nothing else gets downloaded.

datawhores commented 3 months ago

Version: 3.11.6+.d0aa160b

Same problem here. Tried "ofscraper --after 2000 --dupe" & "ofscraper --after 2000 --force-all" but apart from the profile picture, nothing else gets downloaded.

Did you read the other parts of this discussion

You can send a log, but it unlikely to be the script

koolaidflavas commented 3 months ago

same issue here. only downloads profile picture and header. and this is with models that i'm currently subscribed. updated and issue persists.

m4rc1nt0sch commented 3 months ago

Issue seems resolved when forcing the script to reinstall to 3.11.7 -->

pip install git+https://github.com/datawhores/OF-Scraper.git@3.11.7 --force

datawhores commented 3 months ago

Please use https://github.com/datawhores/OF-Scraper/issues/455