Latest git pull gives blank {post_id}

[valdearg]

Not sure if this is something which can be replicated, but I've just updated to the latest git but when downloading I've run into some problems which may be related to the issue in question.

Just to document everything.

Yesterday I updated to the latest version via git pull.

I then ran the scraper via poetry run python start_us.py

I then got a variety of very similar errors to below:

  File "C:\Python\Python311\Lib\shutil.py", line 825, in move
    os.rename(src, real_dst)
FileNotFoundError: [WinError 2] The system cannot find the file specified: '.sites\\OnlyFans-new2\\lysaretta\\Metadata\\__legacy_metadata__\\Messages.db' -> '.sites\\OnlyFans-new2\\lysaretta\\Metadata\\Messages_0.db'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "G:\Scripts\OnlyFans\start_us.py", line 62, in <module>
    asyncio.run(main())
  File "C:\Python\Python311\Lib\asyncio\runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "C:\Python\Python311\Lib\asyncio\runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python\Python311\Lib\asyncio\base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "G:\Scripts\OnlyFans\start_us.py", line 44, in main
    _api = await USR.start(
           ^^^^^^^^^^^^^^^^
  File "G:\Scripts\OnlyFans\ultima_scraper\ultima_scraper.py", line 50, in start
    await self.start_datascraper(datascraper)
  File "G:\Scripts\OnlyFans\ultima_scraper\ultima_scraper.py", line 115, in start_datascraper
    await self.assign_jobs(final_job_user_list)
  File "G:\Scripts\OnlyFans\ultima_scraper\ultima_scraper.py", line 173, in assign_jobs
    metadata_manager = MetadataManager(user, filesystem_manager)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\valdearg\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-FY1q5lm3-py3.11\Lib\site-packages\ultima_scraper_collection\managers\metadata_manager\metadata_manager.py", line 290, in __init__
    self.update_metadata_filepaths()
  File "C:\Users\valdearg\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-FY1q5lm3-py3.11\Lib\site-packages\ultima_scraper_collection\managers\metadata_manager\metadata_manager.py", line 645, in update_metadata_filepaths
    self.filesystem_manager.move(metadata_filepath, new_m_f)
  File "C:\Users\valdearg\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-FY1q5lm3-py3.11\Lib\site-packages\ultima_scraper_collection\managers\filesystem_manager.py", line 44, in move
    shutil.move(src, trg)
  File "C:\Python\Python311\Lib\shutil.py", line 845, in move
    copy_function(src, real_dst)
  File "C:\Python\Python311\Lib\shutil.py", line 436, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "C:\Python\Python311\Lib\shutil.py", line 256, in copyfile
    with open(src, 'rb') as fsrc:
         ^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '.sites\\OnlyFans-new2\\lysaretta\\Metadata\\__legacy_metadata__\\Messages.db'

When checking the __legacy_metadata__ folder, this turned out to be empty. If I checked another model that it hadn't errored on, there would be items in there.

If I ran the command again it'd fail on the same folder. If I removed that empty __legacy_metadata__ it would continue onto the next model.

My thoughts were that it was processing the legacy metadata into the new format? But re-processing the Messages.db somewhere.

Once I'd done this for all subscribed models, the scraper ran through.

It successfully downloaded items, however the folders it was downloading to were missing the post_id and just using the date.

So for example:

G:\Scripts\OnlyFans\.sites\OnlyFans-new2\{model}\Posts\Images\ (30-04-2023)

In the config, I'm using:

"file_directory_format": "OnlyFans-new2/{model_username}/{api_type}/{media_type}/{post_id} ({date})",

So looks like the date is being pulled through, but not the post_id.

Checking the user_data.db, this seems to contain the post_id correctly, e.g.

Is anyone else able to replicate? If so is there a way to go back and rename the files out?

[Anonymouse1746]

I am experiencing this as well - {date} works in folder name but {post_id} is left blank.

[therizzryder]

Happened to me too. DRM Protection is getting rolled out. Many models are starting to activate it. So far it's only applying to any newer videos.

[valdearg]

Slightly interestingly, the "medias" table in the user_data.db database is populated correctly.

directory: .sites/OnlyFans-new2/{model}/Posts/Images/567312733 (30-04-2023) filename: 1024x1280_d1a45f0cea8502d4804856160ef8d69b.jpg

I wrote a very quick script to move the files for the moment.