search

UltimaHoarder / UltimaScraper

Scrape all the media from an OnlyFans account - Updated regularly
GNU General Public License v3.0
3.88k stars 610 forks source link

KeyError: 'videoSources' #2046

Open MortenCB opened 7 months ago

MortenCB commented 7 months ago

I get the following on every run. Any idea what's wrong?

Traceback (most recent call last):
  File "/home/user/UltimaScraper"/UltimaScraper/start_us.py", line 62, in <module>
    asyncio.run(main())
  File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "/home/user/UltimaScraper"/UltimaScraper/start_us.py", line 44, in main
    _api = await USR.start(
  File "/home/user/UltimaScraper"/UltimaScraper/ultima_scraper/ultima_scraper.py", line 50, in start
    await self.start_datascraper(datascraper)
  File "/home/user/UltimaScraper"/UltimaScraper/ultima_scraper/ultima_scraper.py", line 137, in start_datascraper
    await datascraper.datascraper.api.job_manager.process_jobs()
  File "/home/user/UltimaScraper"/UltimaScraper/venv/lib/python3.10/site-packages/ultima_scraper_api/managers/job_manager/job_manager.py", line 45, in process_jobs
    await asyncio.create_task(self.__worker())
  File "/home/user/UltimaScraper"/UltimaScraper/venv/lib/python3.10/site-packages/ultima_scraper_api/managers/job_manager/job_manager.py", line 53, in __worker
    await job.task
  File "/home/user/UltimaScraper"/UltimaScraper/venv/lib/python3.10/site-packages/ultima_scraper_collection/modules/module_streamliner.py", line 202, in prepare_scraper
    await self.process_scraped_content(
  File "/home/user/UltimaScraper"/UltimaScraper/venv/lib/python3.10/site-packages/ultima_scraper_collection/modules/module_streamliner.py", line 237, in process_scraped_content
    unrefined_set: list[dict[str, Any]] = await tqdm_asyncio.gather(
  File "/home/user/UltimaScraper"/UltimaScraper/venv/lib/python3.10/site-packages/tqdm/asyncio.py", line 79, in gather
    res = [await f for f in cls.as_completed(ifs, loop=loop, timeout=timeout,
  File "/home/user/UltimaScraper"/UltimaScraper/venv/lib/python3.10/site-packages/tqdm/asyncio.py", line 79, in <listcomp>
    res = [await f for f in cls.as_completed(ifs, loop=loop, timeout=timeout,
  File "/usr/lib/python3.10/asyncio/tasks.py", line 571, in _wait_for_one
    return f.result()  # May raise f.exception().
  File "/home/user/UltimaScraper"/UltimaScraper/venv/lib/python3.10/site-packages/tqdm/asyncio.py", line 76, in wrap_awaitable
    return i, await f
  File "/home/user/UltimaScraper"/UltimaScraper/venv/lib/python3.10/site-packages/ultima_scraper_collection/managers/datascraper_manager/datascrapers/onlyfans.py", line 51, in media_scraper
    content_metadata.resolve_extractor(Extractor(post_result))
  File "/home/user/UltimaScraper"/UltimaScraper/venv/lib/python3.10/site-packages/ultima_scraper_collection/managers/metadata_manager/metadata_manager.py", line 216, in resolve_extractor
    self.medias: list[MediaMetadata] = result.get_medias(self)
  File "/home/user/UltimaScraper"/UltimaScraper/venv/lib/python3.10/site-packages/ultima_scraper_collection/managers/metadata_manager/metadata_manager.py", line 147, in get_medias
    main_url = self.item.url_picker(asset_metadata)
  File "/home/user/UltimaScraper"/UltimaScraper/venv/lib/python3.10/site-packages/ultima_scraper_api/apis/onlyfans/__init__.py", line 35, in url_picker
    video_qualities = media_item["videoSources"]
KeyError: 'videoSources'
Egon099 commented 5 months ago

get sess, user agent and x-bc from firefox. Chromium ones seem to fail for some reason