search

UltimaHoarder / UltimaScraper

Scrape all the media from an OnlyFans account - Updated regularly
GNU General Public License v3.0
3.97k stars 607 forks source link

Only scraping up to a certain date, not grabbing all videos #2049

Open jlandgrebe-guardian opened 10 months ago

jlandgrebe-guardian commented 10 months ago

I tried a fresh install by deleting, re-cloning the repo and installing dependencies (seemed they were already installed, not sure how to remove for a full re-install).

Added my auth information again (was working previously) and re-ran script. Certain videos of new subscriptions were missing, and some videos of old subs were also missing from the scrape.

Occasionally received an error TypeError: 'ClientConnectionError' object is not iterable which I believe was mitigated by unblocking subscriptions I didn't want to scrape and dealing with large downloads anyways.

I've gone through and tried scraping individual models with the config file, trying "all" and "video". I'll see that it's attempting to download X number of files, but completes the process in about a few seconds, when I expect at least a minute or so for the given download (based on historical experience).

jlandgrebe-guardian commented 10 months ago

UPDATE:

This bug is only occurring for a few subscriptions. The majority of my OF subscriptions are scraping up to the latest post date.

For the affected subscriptions, not only will the tool not scrape the most recent posts, but it will also avoid scraping messages for additional paid content.

I can confirm that the issue persists after a fresh install of python 3.11, and after re-cloning the repo

tedy02 commented 9 months ago

If I'm not mistaken, the date it is stopping at is when the sub switched to copyright encryption which I don't think this can download.

InPlotITrust commented 9 months ago

Experiences the same here.

I cleared out the entire metadata and redownload everything and it works fine for old posts, but newer videos from after 25 November 2023 don't get downloaded. There's a video post from 27 November that it doesn't download and any videos from December also don't get downloaded. Images are fine though.

EDIT: Based on this reddit post from 8 months ago, I can only assume that the videos we can't download have the DRM protection enabled.