UltimaHoarder / UltimaScraper

Scrape all the media from an OnlyFans account - Updated regularly
GNU General Public License v3.0
4k stars 608 forks source link

Scraper starts but crashes after a few models #2072

Open SirJohnCard opened 10 months ago

SirJohnCard commented 10 months ago

Scraper starts, authenticates and begins scraping, but then throws an error after 3-4 models.

Traceback (most recent call last): File "G:\UltimaScraper-main\UltimaScraper-main\start_us.py", line 62, in asyncio.run(main()) File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\asyncio\runners.py", line 190, in run return runner.run(main) ^^^^^^^^^^^^^^^^ File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\asyncio\runners.py", line 118, in run return self._loop.run_until_complete(task) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\asyncio\base_events.py", line 653, in run_until_complete return future.result() ^^^^^^^^^^^^^^^ File "G:\UltimaScraper-main\UltimaScraper-main\start_us.py", line 44, in main _api = await USR.start( ^^^^^^^^^^^^^^^^ File "G:\UltimaScraper-main\UltimaScraper-main\ultima_scraper\ultima_scraper.py", line 50, in start await self.start_datascraper(datascraper) File "G:\UltimaScraper-main\UltimaScraper-main\ultima_scraper\ultima_scraper.py", line 137, in start_datascraper await datascraper.datascraper.api.job_manager.process_jobs() File "C:\Users\USER\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-hZTlC-0s-py3.11\Lib\site-packages\ultima_scraper_api\managers\job_manager\job_manager.py", line 45, in process_jobs await asyncio.create_task(self.worker()) File "C:\Users\USER\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-hZTlC-0s-py3.11\Lib\site-packages\ultima_scraper_api\managers\job_manager\job_manager.py", line 53, in worker await job.task File "C:\Users\USER\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-hZTlC-0s-py3.11\Lib\site-packages\ultima_scraper_collection\modules\module_streamliner.py", line 202, in prepare_scraper await self.process_scraped_content( File "C:\Users\USER\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-hZTlC-0s-py3.11\Lib\site-packages\ultima_scraper_collection\modules\module_streamliner.py", line 237, in process_scraped_content unrefined_set: list[dict[str, Any]] = await tqdm_asyncio.gather( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-hZTlC-0s-py3.11\Lib\site-packages\tqdm\asyncio.py", line 79, in gather res = [await f for f in cls.as_completed(ifs, loop=loop, timeout=timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-hZTlC-0s-py3.11\Lib\site-packages\tqdm\asyncio.py", line 79, in res = [await f for f in cls.as_completed(ifs, loop=loop, timeout=timeout, ^^^^^^^ File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\asyncio\tasks.py", line 605, in _wait_for_one return f.result() # May raise f.exception(). ^^^^^^^^^^ File "C:\Users\USER\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-hZTlC-0s-py3.11\Lib\site-packages\tqdm\asyncio.py", line 76, in wrap_awaitable return i, await f ^^^^^^^ File "C:\Users\USER\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-hZTlC-0s-py3.11\Lib\site-packages\ultima_scraper_collection\managers\datascraper_manager\datascrapers\onlyfans.py", line 61, in media_scraper reformat_item = reformat_manager.prepare_reformat(asset) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-hZTlC-0s-py3.11\Lib\site-packages\ultima_scraper_renamer\reformat.py", line 372, in prepare_reformat download_path = self.filesystem_manager.get_directory_manager( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-hZTlC-0s-py3.11\Lib\site-packages\ultima_scraper_collection\managers\filesystem_manager.py", line 55, in get_directory_manager return self.directory_manager_users[user_id]


KeyError: 402542
basketballhead6 commented 8 months ago

Anyone got a fix for this? I am getting this error as well