DIGITALCRIMINAL / ArchivedUltimaScraper

Scrape content from OnlyFans and Fansly
GNU General Public License v3.0
932 stars 39 forks source link

Endless loop, and I can't get anything #1011

Open djmarioka opened 1 year ago

djmarioka commented 1 year ago

Hi,

when starting, it gets this endless loop, and I can't get any subscriptions:

image

Could Digitalcriminals, or anyone, help me?

Thanks.

TheJoe49 commented 1 year ago

Does your auth file look like this? I ran in to that and just had to change the "active" from null to true.

image

djmarioka commented 1 year ago

I did it, and now it gives these errors:

image

Could anybody help me?

Thanks

geoff10e commented 1 year ago

I was getting the same error, and it looks like there's a display issue, and maybe a selections issue. If you just spam 0, it'll eventually go through and scrape everything from everyone. I don't think you can be selective in the current build, but maybe I'm wrong and there's something I've screwed up.

NeeFoo commented 1 year ago

I was getting the same error, and it looks like there's a display issue, and maybe a selections issue. If you just spam 0, it'll eventually go through and scrape everything from everyone. I don't think you can be selective in the current build, but maybe I'm wrong and there's something I've screwed up.

this happens to me too, no way to be selective.

ezam0000 commented 1 year ago

It works but how do i prevent it from downloading everything from everyone. Ive tried twice and it just downloads everything from all my subscriptions.

(base) ez@EZ-MacBook-Pro-M2-Max UltimaScraper % poetry run python start_us.py Choose Sites: 0 = All | 1 = OnlyFans | 2 = Fansly 1 Choose Profiles: 0 = All | 1 = default 1 Choose Subscriptions: 0 = All | 1 = witch_vanity | 2 = burdsnest | 3 = suicidegirls | 4 = tslilixxx | 5 = vanity_witch | 6 = lovesunbaby | 7 = sinzor | 8 = anacheri | 9 = amandaparis | 10 = kjanecaron | 11 = british_joanna 2 [2023-07-15 00:54:23] Assigning Jobs Choose Medias: 0 = All | 1 = Images | 2 = Videos | 3 = Audios | 4 = Texts 1 Choose Medias: 0 = All | 1 = Images | 2 = Videos | 3 = Audios | 4 = Texts 1 Choose Medias: 0 = All | 1 = Images | 2 = Videos | 3 = Audios | 4 = Texts 1 Choose Medias: 0 = All | 1 = Images | 2 = Videos | 3 = Audios | 4 = Texts 1 Choose Medias: 0 = All | 1 = Images | 2 = Videos | 3 = Audios | 4 = Texts 1 Processing Scraped Messages 100%|█████████████████████████████████████████████████████████████████████████████| 59/59 [00:00<00:00, 4972.65it/s] skylarmaexo: Renaming files. Processing Download: Name: skylarmaexo | Type: Messages | Downloading: 247 | Total: 247 | Duplicates: 0 | Directory: __user_data__/sites/OnlyFans/skylarmaexo

Processing Scraped Messages 100%|█████████████████████████████████████████████████████████████████████████████| 74/74 [00:00<00:00, 6731.99it/s] bbyanni: Renaming files. Processing Download: Name: bbyanni | Type: Messages | Downloading: 223 | Total: 237 | Duplicates: 14 | Directory: __user_data__/sites/OnlyFans/bbyanni

Processing Scraped Messages 0%| | 0/169 [00:00<?, ?it/s]Traceback (most recent call last): File "/Users/ez/Documents/PROJECTS/UltimaScraper/start_us.py", line 62, in asyncio.run(main()) File "/Users/ez/anaconda3/lib/python3.10/asyncio/runners.py", line 44, in run return loop.run_until_complete(main) File "/Users/ez/anaconda3/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete return future.result() File "/Users/ez/Documents/PROJECTS/UltimaScraper/start_us.py", line 44, in main _api = await USR.start( File "/Users/ez/Documents/PROJECTS/UltimaScraper/ultima_scraper/ultima_scraper.py", line 50, in start await self.start_datascraper(datascraper) File "/Users/ez/Documents/PROJECTS/UltimaScraper/ultima_scraper/ultima_scraper.py", line 137, in start_datascraper await datascraper.datascraper.api.job_manager.process_jobs() File "/Users/ez/Library/Caches/pypoetry/virtualenvs/ultima-scraper-mpMxmBBI-py3.10/lib/python3.10/site-packages/ultima_scraper_api/managers/job_manager/job_manager.py", line 45, in process_jobs await asyncio.create_task(self.worker()) File "/Users/ez/Library/Caches/pypoetry/virtualenvs/ultima-scraper-mpMxmBBI-py3.10/lib/python3.10/site-packages/ultima_scraper_api/managers/job_manager/job_manager.py", line 53, in worker await job.task File "/Users/ez/Library/Caches/pypoetry/virtualenvs/ultima-scraper-mpMxmBBI-py3.10/lib/python3.10/site-packages/ultima_scraper_collection/modules/module_streamliner.py", line 202, in prepare_scraper await self.process_scraped_content( File "/Users/ez/Library/Caches/pypoetry/virtualenvs/ultima-scraper-mpMxmBBI-py3.10/lib/python3.10/site-packages/ultima_scraper_collection/modules/module_streamliner.py", line 237, in process_scraped_content unrefined_set: list[dict[str, Any]] = await tqdm_asyncio.gather( File "/Users/ez/Library/Caches/pypoetry/virtualenvs/ultima-scraper-mpMxmBBI-py3.10/lib/python3.10/site-packages/tqdm/asyncio.py", line 79, in gather res = [await f for f in cls.as_completed(ifs, loop=loop, timeout=timeout, File "/Users/ez/Library/Caches/pypoetry/virtualenvs/ultima-scraper-mpMxmBBI-py3.10/lib/python3.10/site-packages/tqdm/asyncio.py", line 79, in res = [await f for f in cls.as_completed(ifs, loop=loop, timeout=timeout, File "/Users/ez/anaconda3/lib/python3.10/asyncio/tasks.py", line 571, in _wait_for_one return f.result() # May raise f.exception(). File "/Users/ez/Library/Caches/pypoetry/virtualenvs/ultima-scraper-mpMxmBBI-py3.10/lib/python3.10/site-packages/tqdm/asyncio.py", line 76, in wrap_awaitable return i, await f File "/Users/ez/Library/Caches/pypoetry/virtualenvs/ultima-scraper-mpMxmBBI-py3.10/lib/python3.10/site-packages/ultima_scraper_collection/managers/datascraper_manager/datascrapers/onlyfans.py", line 61, in media_scraper reformat_item = reformat_manager.prepare_reformat(asset) File "/Users/ez/Library/Caches/pypoetry/virtualenvs/ultima-scraper-mpMxmBBI-py3.10/lib/python3.10/site-packages/ultima_scraper_renamer/reformat.py", line 372, in prepare_reformat download_path = self.filesystem_manager.get_directory_manager( File "/Users/ez/Library/Caches/pypoetry/virtualenvs/ultima-scraper-mpMxmBBI-py3.10/lib/python3.10/site-packages/ultima_scraper_collection/managers/filesystem_manager.py", line 55, in get_directory_manager return self.directory_manager_users[user_id] KeyError: 34867161 22%|█████████████████▎ | 38/169 [00:00<00:00, 583.77it/s] (base) ez@EZ-MacBook-Pro-M2-Max UltimaScraper %