DIGITALCRIMINAL / ArchivedUltimaScraper

Scrape content from OnlyFans and Fansly
GNU General Public License v3.0
948 stars 39 forks source link

Not Downloading Any Files #663

Closed ghost closed 1 year ago

ghost commented 2 years ago

Keeping it simple, it's not downloading any content from OnlyFans. It'll download the avatar and header, it'll create folders and what not, but no images or videos.

Dudeman456 commented 2 years ago

I having the same issue, but when I choose a model, it will download that model's stuff then it will pick another model and ask me what I want to download from the new pick.

cyberkoala commented 2 years ago

So I'm seeing the same issue, but the 'pick a single model' option doesn't work for me, i seem to get the same behavior either way. No error output, i see bars as if content should be downloaded, but nothing ends up in the folders.

I had a much older version, but deleted everything in the master folder except .sites, renamed .sites-old , then unpacked the latest version.. Only filled out my profiles. Can confirm that fansly works fine, it's only OF where it's broken.

Ran with -v flag to get more outpu, and here's an example:

Type: Stories Processing Scraped Stories 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 4005.06it/s] Processing metadata. Finished processing metadata. I:\Scrappers\OnlyFans-master\database\databases\user_data\alembic\__pycache__\env.cpython-310.pyc matches I:\Scrappers\OnlyFans-master\database\databases\user_data\alembic\env.py code object from 'I:\\Scrappers\\OnlyFans-master\\database\\databases\\user_data\\alembic\\__pycache__\\env.cpython-310.pyc' I:\Scrappers\OnlyFans-master\database\databases\user_data\alembic\versions\__pycache__\0d4d92c0498e_content.cpython-310.pyc matches I:\Scrappers\OnlyFans-master\database\databases\user_data\alembic\versions\0d4d92c0498e_content.py code object from 'I:\\Scrappers\\OnlyFans-master\\database\\databases\\user_data\\alembic\\versions\\__pycache__\\0d4d92c0498e_content.cpython-310.pyc' I:\Scrappers\OnlyFans-master\database\databases\user_data\alembic\versions\__pycache__\37c4f2719d65_content.cpython-310.pyc matches I:\Scrappers\OnlyFans-master\database\databases\user_data\alembic\versions\37c4f2719d65_content.py code object from 'I:\\Scrappers\\OnlyFans-master\\database\\databases\\user_data\\alembic\\versions\\__pycache__\\37c4f2719d65_content.cpython-310.pyc' I:\Scrappers\OnlyFans-master\database\databases\user_data\alembic\versions\__pycache__\5493253cc03c_content.cpython-310.pyc matches I:\Scrappers\OnlyFans-master\database\databases\user_data\alembic\versions\5493253cc03c_content.py code object from 'I:\\Scrappers\\OnlyFans-master\\database\\databases\\user_data\\alembic\\versions\\__pycache__\\5493253cc03c_content.cpython-310.pyc' I:\Scrappers\OnlyFans-master\database\databases\user_data\alembic\versions\__pycache__\b791cf213df9_content.cpython-310.pyc matches I:\Scrappers\OnlyFans-master\database\databases\user_data\alembic\versions\b791cf213df9_content.py code object from 'I:\\Scrappers\\OnlyFans-master\\database\\databases\\user_data\\alembic\\versions\\__pycache__\\b791cf213df9_content.cpython-310.pyc' destroy env_py Renaming files. 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 4001.24it/s] Type: Posts Scrape Attempt: 1/100 destroy 0d4d92c0498e_content_py destroy 37c4f2719d65_content_py destroy 5493253cc03c_content_py destroy b791cf213df9_content_py Type: Archived Posts Processing Scraped Posts 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 141/141 [00:00<00:00, 2105.21it/s] Processing metadata. Finished processing metadata. I:\Scrappers\OnlyFans-master\database\databases\user_data\alembic\__pycache__\env.cpython-310.pyc matches I:\Scrappers\OnlyFans-master\database\databases\user_data\alembic\env.py code object from 'I:\\Scrappers\\OnlyFans-master\\database\\databases\\user_data\\alembic\\__pycache__\\env.cpython-310.pyc' I:\Scrappers\OnlyFans-master\database\databases\user_data\alembic\versions\__pycache__\0d4d92c0498e_content.cpython-310.pyc matches I:\Scrappers\OnlyFans-master\database\databases\user_data\alembic\versions\0d4d92c0498e_content.py code object from 'I:\\Scrappers\\OnlyFans-master\\database\\databases\\user_data\\alembic\\versions\\__pycache__\\0d4d92c0498e_content.cpython-310.pyc' I:\Scrappers\OnlyFans-master\database\databases\user_data\alembic\versions\__pycache__\37c4f2719d65_content.cpython-310.pyc matches I:\Scrappers\OnlyFans-master\database\databases\user_data\alembic\versions\37c4f2719d65_content.py code object from 'I:\\Scrappers\\OnlyFans-master\\database\\databases\\user_data\\alembic\\versions\\__pycache__\\37c4f2719d65_content.cpython-310.pyc' I:\Scrappers\OnlyFans-master\database\databases\user_data\alembic\versions\__pycache__\5493253cc03c_content.cpython-310.pyc matches I:\Scrappers\OnlyFans-master\database\databases\user_data\alembic\versions\5493253cc03c_content.py code object from 'I:\\Scrappers\\OnlyFans-master\\database\\databases\\user_data\\alembic\\versions\\__pycache__\\5493253cc03c_content.cpython-310.pyc' I:\Scrappers\OnlyFans-master\database\databases\user_data\alembic\versions\__pycache__\b791cf213df9_content.cpython-310.pyc matches I:\Scrappers\OnlyFans-master\database\databases\user_data\alembic\versions\b791cf213df9_content.py code object from 'I:\\Scrappers\\OnlyFans-master\\database\\databases\\user_data\\alembic\\versions\\__pycache__\\b791cf213df9_content.cpython-310.pyc' destroy env_py destroy 0d4d92c0498e_content_py destroy 37c4f2719d65_content_py destroy 5493253cc03c_content_py destroy b791cf213df9_content_py Renaming files. 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 141/141 [00:00<00:00, 2168.41it/s] Type: Archived Type: Chats Type: Messages Processing Scraped Messages 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<?, ?it/s] Processing metadata. Finished processing metadata. I:\Scrappers\OnlyFans-master\database\databases\user_data\alembic\__pycache__\env.cpython-310.pyc matches I:\Scrappers\OnlyFans-master\database\databases\user_data\alembic\env.py code object from 'I:\\Scrappers\\OnlyFans-master\\database\\databases\\user_data\\alembic\\__pycache__\\env.cpython-310.pyc' I:\Scrappers\OnlyFans-master\database\databases\user_data\alembic\versions\__pycache__\0d4d92c0498e_content.cpython-310.pyc matches I:\Scrappers\OnlyFans-master\database\databases\user_data\alembic\versions\0d4d92c0498e_content.py# code object from 'I:\\Scrappers\\OnlyFans-master\\database\\databases\\user_data\\alembic\\versions\\__pycache__\\0d4d92c0498e_content.cpython-310.pyc' I:\Scrappers\OnlyFans-master\database\databases\user_data\alembic\versions\__pycache__\37c4f2719d65_content.cpython-310.pyc matches I:\Scrappers\OnlyFans-master\database\databases\user_data\alembic\versions\37c4f2719d65_content.py code object from 'I:\\Scrappers\\OnlyFans-master\\database\\databases\\user_data\\alembic\\versions\\__pycache__\\37c4f2719d65_content.cpython-310.pyc' I:\Scrappers\OnlyFans-master\database\databases\user_data\alembic\versions\__pycache__\5493253cc03c_content.cpython-310.pyc matches I:\Scrappers\OnlyFans-master\database\databases\user_data\alembic\versions\5493253cc03c_content.py code object from 'I:\\Scrappers\\OnlyFans-master\\database\\databases\\user_data\\alembic\\versions\\__pycache__\\5493253cc03c_content.cpython-310.pyc' I:\Scrappers\OnlyFans-master\database\databases\user_data\alembic\versions\__pycache__\b791cf213df9_content.cpython-310.pyc matches I:\Scrappers\OnlyFans-master\database\databases\user_data\alembic\versions\b791cf213df9_content.py code object from 'I:\\Scrappers\\OnlyFans-master\\database\\databases\\user_data\\alembic\\versions\\__pycache__\\b791cf213df9_content.cpython-310.pyc' destroy env_py destroy 0d4d92c0498e_content_py destroy 37c4f2719d65_content_py destroy 5493253cc03c_content_py destroy b791cf213df9_content_py Renaming files. 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<?, ?it/s] Type: Highlights Type: MassMessages Scrape Completed

So no errors jump out at me there, but it looks like it's still failing to get any content. I also know it can download the avatar, so it's not a weird folder permission. it's getting valid info for the scrapes , like number of messages- it just seems to not actually be downloading the content. I've checked other recent common/open issues, nothing else seems relevant- it has the fix from #604 already.

Elende commented 2 years ago

I had the same issue after grabbing the master build. Setting messages and paid_content to false under scrape in .settings\config.json seemed to help.

cyberkoala commented 2 years ago

I had the same issue after grabbing the master build. Setting messages and paid_content to false under scrape in .settings\config.json seemed to help.

That's interesting.. i know that in the 604 thread, there were thoughts it was tied to messages... only problem is there's some paid content in messages i would like to scrape, but that's not the bulk of content i am worried about now, will try tha tthis afternoon.

Elende commented 2 years ago

I had the same issue after grabbing the master build. Setting messages and paid_content to false under scrape in .settings\config.json seemed to help.

That's interesting.. i know that in the 604 thread, there were thoughts it was tied to messages... only problem is there's some paid content in messages i would like to scrape, but that's not the bulk of content i am worried about now, will try tha tthis afternoon.

Yeah it's unfortunate that paid messaging content will be ignored but at least you'll be able to grab just about everything else. I haven't found a proper solution and the edits in 604 just loops through messages without actually downloading.

Alternatively you could just revert to 7.6.1 and apply the fix in #601. The fansly side of that build doesn't work but the OF scrapes everything just fine.

lmilt13 commented 2 years ago

Try this: poetry run python .\start_ofd.py Worked for me :)

ghost commented 2 years ago

@lmilt13 no luck with that.

@Elende thank you for the tip, that seems to work for now, but definitely leaves something to be desired 😅

masteraaran commented 2 years ago

@dogedudefa @Dudeman456 I ran into this issue before and wrote up some information on WHY I believe it's doing this and what it's doing. You can check out what I thought was the problem in issue #397

The basic overview: The script creates a background download queue, and asks you a bunch of questions about what you want to download before it actually starts downloading. The problem you're seeing, where it seemingly asks about every model, comes from a change that the Dev made to message scraping. Now instead of getting only the messages for the model you select, it now asks about EVERY chat you have ever been part of, and there is no 'none' option for what to download.

You've got three options to work around it:

  1. Follow along with the script and select something for every model it asks you about.
  2. Disable message scraping (this is what I did, but I still get the content..... method below ;) )
  3. as @Elende said, downgrade.

So, how do I get the paid content in the messages? - (Caveat: I only pay for one model, and she sends a video out a few times a month in DM's. That's the only paid content she sends and so I don't have to scrape lots of content.) - Basically I open the browser Developer Panel to the Network tab, click 'play' on the video, wait for it to start downloading, grab the URL of the video from the browser, open it in a new tab, and I can save it from there.