DIGITALCRIMINAL / ArchivedUltimaScraper

Scrape content from OnlyFans and Fansly
GNU General Public License v3.0
932 stars 39 forks source link

Only downloading "Metadata" and "Messages" folders. #285

Closed HistoricTheater closed 2 years ago

HistoricTheater commented 2 years ago

Using 9ef5fd4 I renamed my active models just to see how everything was with a fresh pull. Only OF "messages" and "metadata" folders are being downloaded, posts/archives/profile are not being downloaded at all,even running multiple times. Further checking, the messages folder is not downloading "Free" content either

The "No subscriptions to scrape" dialog still shows but no python error

sintaxx commented 2 years ago

Are you trying to select multiple models at once to download? like "1,2,3,4" ? I noticed just a few minutes ago I tried to download multiple models and i got the "no subscriptions to scrape" error but when I do a model one at a time it seemed to work fine. Either way, it's still an issue but that's how I temporarily fixed it.

DIGITALCRIMINAL commented 2 years ago

Numbers don't work for config, gotta be their usernames

offf1989 commented 2 years ago

I did the same thing and I'm getting a similar bug. New messages are not being downloaded. I tried just running it for one of the models like normal and noticed that messages after Jan 1 weren't being downloaded, so then I tried a fresh pull and I'm still not getting messages newer than Jan 6... I tried both with numbers and names of the models with the same result. Update: there are some models that have only messages missing and other models that have both messages and posts missing... I'm not seeing a clear pattern as to why though

prockerxoxo commented 2 years ago

Have you tried using 'python -v start_ofd.py' to see what the issue may be?

offf1989 commented 2 years ago

I had not tried that, I just ran it with the -v option, although nothing in the logs looks particularly interesting. Pasting in case I'm missing something though:

Type: Stories Processing Scraped Stories 0%| | 0/1 [00:00<?, ?it/s]C:\Users\user1\AppData\Local\Programs\Python\Python310\lib__pycache_strptime.cpython-310.pyc matches C:\Users\user1\AppData\Local\Programs\Python\Python310\lib_strptime.py code object from 'C:\Users\user1\AppData\Local\Programs\Python\Python310\lib\pycache\_strptime.cpython-310.pyc' import '_strptime' <_frozen_importlib_external.SourceFileLoader object at 0x000000CD5A5CFFA0> 100%|███████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 332.43it/s] Processing metadata. Finished processing metadata. possible namespace for S:\Transfer\Downloads\ofdownloader\OnlyFans\extras import 'extras' <_frozen_importlib_external._NamespaceLoader object at 0x000000CD5A5CFB50> possible namespace for S:\Transfer\Downloads\ofdownloader\OnlyFans\extras\OFRenamer import 'extras.OFRenamer' <_frozen_importlib_external._NamespaceLoader object at 0x000000CD5A569810> S:\Transfer\Downloads\ofdownloader\OnlyFans\extras\OFRenamer__pycache\start_ofr.cpython-310.pyc matches S:\Transfer\Downloads\ofdownloader\OnlyFans\extras\OFRenamer\start_ofr.py code object from 'S:\Transfer\Downloads\ofdownloader\OnlyFans\extras\OFRenamer\pycache\start_ofr.cpython-310.pyc' import 'extras.OFRenamer.start_ofr' <_frozen_importlib_external.SourceFileLoader object at 0x000000CD5A5CF3D0> S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic__pycache\env.cpython-310.pyc matches S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\env.py code object from 'S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\pycache\env.cpython-310.pyc' C:\Users\user1\AppData\Local\Programs\Python\Python310\lib\logging__pycache\config.cpython-310.pyc matches C:\Users\user1\AppData\Local\Programs\Python\Python310\lib\logging\config.py code object from 'C:\Users\user1\AppData\Local\Programs\Python\Python310\lib\logging\pycache\config.cpython-310.pyc' C:\Users\user1\AppData\Local\Programs\Python\Python310\lib\logging__pycache\handlers.cpython-310.pyc matches C:\Users\user1\AppData\Local\Programs\Python\Python310\lib\logging\handlers.py code object from 'C:\Users\user1\AppData\Local\Programs\Python\Python310\lib\logging\pycache\handlers.cpython-310.pyc' import 'logging.handlers' <_frozen_importlib_external.SourceFileLoader object at 0x000000CD5C87CFA0> import 'logging.config' <_frozen_importlib_external.SourceFileLoader object at 0x000000CD5C87C880> C:\Users\user1\AppData\Roaming\Python\Python310\site-packages\sqlalchemy\dialects\sqlite__pycache__init.cpython-310.pyc matches C:\Users\user1\AppData\Roaming\Python\Python310\site-packages\sqlalchemy\dialects\sqlite\init.py code object from 'C:\Users\user1\AppData\Roaming\Python\Python310\site-packages\sqlalchemy\dialects\sqlite\pycache\init.cpython-310.pyc' C:\Users\user1\AppData\Roaming\Python\Python310\site-packages\sqlalchemy\dialects\sqlite\pycache\base.cpython-310.pyc matches C:\Users\user1\AppData\Roaming\Python\Python310\site-packages\sqlalchemy\dialects\sqlite\base.py code object from 'C:\Users\user1\AppData\Roaming\Python\Python310\site-packages\sqlalchemy\dialects\sqlite\pycache\base.cpython-310.pyc' C:\Users\user1\AppData\Roaming\Python\Python310\site-packages\sqlalchemy\dialects\sqlite\pycache\json.cpython-310.pyc matches C:\Users\user1\AppData\Roaming\Python\Python310\site-packages\sqlalchemy\dialects\sqlite\json.py code object from 'C:\Users\user1\AppData\Roaming\Python\Python310\site-packages\sqlalchemy\dialects\sqlite\pycache\json.cpython-310.pyc' import 'sqlalchemy.dialects.sqlite.json' <_frozen_importlib_external.SourceFileLoader object at 0x000000CD5CA1C100> import 'sqlalchemy.dialects.sqlite.base' <_frozen_importlib_external.SourceFileLoader object at 0x000000CD5C87E9E0> C:\Users\user1\AppData\Roaming\Python\Python310\site-packages\sqlalchemy\dialects\sqlite\pycache\pysqlcipher.cpython-310.pyc matches C:\Users\user1\AppData\Roaming\Python\Python310\site-packages\sqlalchemy\dialects\sqlite\pysqlcipher.py code object from 'C:\Users\user1\AppData\Roaming\Python\Python310\site-packages\sqlalchemy\dialects\sqlite\pycache\pysqlcipher.cpython-310.pyc' C:\Users\user1\AppData\Roaming\Python\Python310\site-packages\sqlalchemy\dialects\sqlite\pycache\pysqlite.cpython-310.pyc matches C:\Users\user1\AppData\Roaming\Python\Python310\site-packages\sqlalchemy\dialects\sqlite\pysqlite.py code object from 'C:\Users\user1\AppData\Roaming\Python\Python310\site-packages\sqlalchemy\dialects\sqlite\pycache\pysqlite.cpython-310.pyc' import 'sqlalchemy.dialects.sqlite.pysqlite' <_frozen_importlib_external.SourceFileLoader object at 0x000000CD5CA1DB70> import 'sqlalchemy.dialects.sqlite.pysqlcipher' <_frozen_importlib_external.SourceFileLoader object at 0x000000CD5C87F220> C:\Users\user1\AppData\Roaming\Python\Python310\site-packages\sqlalchemy\dialects\sqlite\pycache\dml.cpython-310.pyc matches C:\Users\user1\AppData\Roaming\Python\Python310\site-packages\sqlalchemy\dialects\sqlite\dml.py code object from 'C:\Users\user1\AppData\Roaming\Python\Python310\site-packages\sqlalchemy\dialects\sqlite\pycache__\dml.cpython-310.pyc' import 'sqlalchemy.dialects.sqlite.dml' <_frozen_importlib_external.SourceFileLoader object at 0x000000CD5CA1D330> C:\Users\user1\AppData\Roaming\Python\Python310\site-packages\sqlalchemy\dialects\sqlite\pycache\aiosqlite.cpython-310.pyc matches C:\Users\user1\AppData\Roaming\Python\Python310\site-packages\sqlalchemy\dialects\sqlite\aiosqlite.py code object from 'C:\Users\user1\AppData\Roaming\Python\Python310\site-packages\sqlalchemy\dialects\sqlite\pycache\aiosqlite.cpython-310.pyc' import 'sqlalchemy.dialects.sqlite.aiosqlite' <_frozen_importlib_external.SourceFileLoader object at 0x000000CD5CA1D990> import 'sqlalchemy.dialects.sqlite' <_frozen_importlib_external.SourceFileLoader object at 0x000000CD5C87EB00> C:\Users\user1\AppData\Local\Programs\Python\Python310\lib\sqlite3\pycache\init.cpython-310.pyc matches C:\Users\user1\AppData\Local\Programs\Python\Python310\lib\sqlite3\init.py code object from 'C:\Users\user1\AppData\Local\Programs\Python\Python310\lib\sqlite3\pycache\init.cpython-310.pyc' C:\Users\user1\AppData\Local\Programs\Python\Python310\lib\sqlite3\pycache\dbapi2.cpython-310.pyc matches C:\Users\user1\AppData\Local\Programs\Python\Python310\lib\sqlite3\dbapi2.py code object from 'C:\Users\user1\AppData\Local\Programs\Python\Python310\lib\sqlite3\pycache__\dbapi2.cpython-310.pyc' extension module '_sqlite3' loaded from 'C:\Users\user1\AppData\Local\Programs\Python\Python310\DLLs\_sqlite3.pyd' extension module '_sqlite3' executed from 'C:\Users\user1\AppData\Local\Programs\Python\Python310\DLLs\_sqlite3.pyd' import '_sqlite3' <_frozen_importlib_external.ExtensionFileLoader object at 0x000000CD5CA1E9E0> import 'sqlite3.dbapi2' <_frozen_importlib_external.SourceFileLoader object at 0x000000CD5CA1ED10> import 'sqlite3' <_frozen_importlib_external.SourceFileLoader object at 0x000000CD5CA1EEF0> S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\versions\pycache\0d4d92c0498e_content.cpython-310.pyc matches S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\versions\0d4d92c0498e_content.py code object from 'S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\versions\pycache\0d4d92c0498e_content.cpython-310.pyc' S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\versions\pycache\5493253cc03c_content.cpython-310.pyc matches S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\versions\5493253cc03c_content.py code object from 'S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\versions\pycache\5493253cc03c_content.cpython-310.pyc' S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\versions\pycache\b791cf213df9_content.cpython-310.pyc matches S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\versions\b791cf213df9_content.py code object from 'S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\versions\pycache__\b791cf213df9_content.cpython-310.pyc' destroy env_py Renaming files. destroy 0d4d92c0498e_content_py destroy 5493253cc03c_content_py destroy b791cf213df9_content_py 100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 14/14 [00:00<00:00, 324.71it/s] Type: Posts Scrape Attempt: 1/100 Type: Archived Posts Scrape Attempt: 1/100 Processing Scraped Posts 100%|██████████████████████████████████████████████████████████████████████████████████████████████| 899/899 [00:00<00:00, 2456.47it/s] Processing metadata. Finished processing metadata. S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\pycache\env.cpython-310.pyc matches S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\env.py code object from 'S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\pycache__\env.cpython-310.pyc' S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\versions\pycache\0d4d92c0498e_content.cpython-310.pyc matches S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\versions\0d4d92c0498e_content.py code object from 'S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\versions\pycache\0d4d92c0498e_content.cpython-310.pyc' S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\versions\pycache\5493253cc03c_content.cpython-310.pyc matches S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\versions\5493253cc03c_content.py code object from 'S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\versions\pycache\5493253cc03c_content.cpython-310.pyc' S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\versions\pycache\b791cf213df9_content.cpython-310.pyc matches S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\versions\b791cf213df9_content.py code object from 'S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\versions\pycache__\b791cf213df9_content.cpython-310.pyc' destroy env_py Renaming files. destroy 0d4d92c0498e_content_py destroy 5493253cc03c_content_py destroy b791cf213df9_content_py 100%|█████████████████████████████████████████████████████████████████████████████████████████████| 1016/1016 [00:04<00:00, 239.32it/s] Type: Archived Type: Chats Type: Messages Processing Scraped Messages 100%|██████████████████████████████████████████████████████████████████████████████████████████████| 402/402 [00:00<00:00, 1293.32it/s] Processing metadata. Finished processing metadata. S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\pycache\env.cpython-310.pyc matches S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\env.py code object from 'S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\pycache__\env.cpython-310.pyc' S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\versions\pycache\0d4d92c0498e_content.cpython-310.pyc matches S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\versions\0d4d92c0498e_content.py code object from 'S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\versions\pycache\0d4d92c0498e_content.cpython-310.pyc' S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\versions\pycache\5493253cc03c_content.cpython-310.pyc matches S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\versions\5493253cc03c_content.py code object from 'S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\versions\pycache\5493253cc03c_content.cpython-310.pyc' S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\versions\pycache\b791cf213df9_content.cpython-310.pyc matches S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\versions\b791cf213df9_content.py code object from 'S:\Transfer\Downloads\ofdownloader\OnlyFans\database\databases\user_data\alembic\versions\pycache__\b791cf213df9_content.cpython-310.pyc' destroy env_py destroy 0d4d92c0498e_content_py destroy 5493253cc03c_content_py destroy b791cf213df9_content_py Renaming files. 100%|███████████████████████████████████████████████████████████████████████████████████████████████| 422/422 [00:01<00:00, 317.89it/s] Type: Highlights Type: MassMessages Scrape Completed

Archive Completed in 1.11 Minutes Pausing scraper for 0 seconds.

Removed the # signs from the output to avoid them being treated as headers

PsikhotykEvil commented 2 years ago

Tried with just the model name and still having the same issue.

sintaxx commented 2 years ago

Mine was working fine just with the numbers somehow

magentabang commented 2 years ago

There's definitely a bug with paid content, it's passing the wrong config value to the SubscriptionManager, I've fixed it locally and after I clean up I should be able to share.

HistoricTheater commented 2 years ago

This is still present with commit f403fee

outsdr01 commented 2 years ago

I too experience this issue.

DIGITALCRIMINAL commented 2 years ago

Try latest commit

HistoricTheater commented 2 years ago

Same problem with 4eb18c7 ,only "Messages" and "Metadata" are being populated, this is with the model folder not existing too,it redownloaded the messages and nothing else,but also not the "Free" folder in messages

88bells commented 2 years ago

Im getting this too. auto_model_choice is set to true for me.

HistoricTheater commented 2 years ago

Commit ffccfe8 does not resolve this issue.

DIGITALCRIMINAL commented 2 years ago

What is your config

HistoricTheater commented 2 years ago
{
  "info": {
    "version": 8.0
  },
  "settings": {
    "auto_site_choice": "onlyfans",
    "export_type": "json",
    "profile_directories": [
      ".profiles"
    ],
    "max_threads": -1,
    "min_drive_space": 0,
    "helpers": {
      "renamer": true,
      "reformat_media": true,
      "downloader": true,
      "delete_empty_directories": false
    },
    "webhooks": {
      "global_webhooks": [],
      "global_status": true,
      "auth_webhook": {
        "succeeded": {
          "webhooks": [],
          "status": null,
          "hide_sensitive_info": true
        },
        "failed": {
          "webhooks": [],
          "status": null,
          "hide_sensitive_info": true
        }
      },
      "download_webhook": {
        "succeeded": {
          "webhooks": [],
          "status": null,
          "hide_sensitive_info": true
        },
        "failed": {
          "webhooks": [],
          "status": null,
          "hide_sensitive_info": true
        }
      }
    },
    "exit_on_completion": true,
    "infinite_loop": false,
    "loop_timeout": 0,
    "dynamic_rules_link": "https://raw.githubusercontent.com/DATAHOARDERS/dynamic-rules/main/onlyfans.json",
    "proxies": [],
    "cert": "",
    "random_string": "3c28fa90731311ec9067a85e45a229e9"
  },
  "supported": {
    "onlyfans": {
      "settings": {
        "auto_profile_choice": "Default",
        "auto_model_choice": "All",
        "auto_media_choice": "All",
        "auto_api_choice": true,
        "browser": {
          "auth": false
        },
        "jobs": {
          "scrape": {
            "subscriptions": true,
            "paid_content": true
          },
          "metadata": {
            "posts": true,
            "comments": true
          }
        },
        "download_directories": [
          ".sites"
        ],
        "file_directory_format": "{site_name}/{model_username}/{api_type}/{value}/{media_type}",
        "filename_format": "{date}_{filename}.{ext}",
        "metadata_directories": [
          ".sites"
        ],
        "metadata_directory_format": "{site_name}/{model_username}/Metadata",
        "delete_legacy_metadata": false,
        "text_length": 255,
        "video_quality": "source",
        "overwrite_files": false,
        "date_format": "%Y-%m-%d",
        "ignored_keywords": [],
        "ignore_type": "",
        "blacklists": [],
        "webhook": false
      }
    },
    "fansly": {
      "settings": {
        "auto_profile_choice": [],
        "auto_model_choice": false,
        "auto_media_choice": "",
        "auto_api_choice": true,
        "browser": {
          "auth": true
        },
        "jobs": {
          "scrape": {
            "subscriptions": true,
            "paid_content": true
          },
          "metadata": {
            "posts": true,
            "comments": true
          }
        },
        "download_directories": [
          ".sites"
        ],
        "file_directory_format": "{site_name}/{model_username}/{api_type}/{value}/{media_type}",
        "filename_format": "{filename}.{ext}",
        "metadata_directories": [
          ".sites"
        ],
        "metadata_directory_format": "{site_name}/{model_username}/Metadata",
        "delete_legacy_metadata": false,
        "text_length": 255,
        "video_quality": "source",
        "overwrite_files": false,
        "date_format": "%d-%m-%Y",
        "ignored_keywords": [],
        "ignore_type": "",
        "blacklists": [],
        "webhook": false
      }
    },
    "starsavn": {
      "settings": {
        "auto_profile_choice": [],
        "auto_model_choice": false,
        "auto_media_choice": "",
        "auto_api_choice": true,
        "browser": {
          "auth": true
        },
        "jobs": {
          "scrape": {
            "subscriptions": true,
            "paid_content": true
          },
          "metadata": {
            "posts": true,
            "comments": true
          }
        },
        "download_directories": [
          ".sites"
        ],
        "file_directory_format": "{site_name}/{model_username}/{api_type}/{value}/{media_type}",
        "filename_format": "{filename}.{ext}",
        "metadata_directories": [
          ".sites"
        ],
        "metadata_directory_format": "{site_name}/{model_username}/Metadata",
        "delete_legacy_metadata": false,
        "text_length": 255,
        "video_quality": "source",
        "overwrite_files": false,
        "date_format": "%d-%m-%Y",
        "ignored_keywords": [],
        "ignore_type": "",
        "blacklists": [],
        "webhook": true
      }
    }
  }
}
DIGITALCRIMINAL commented 2 years ago

Okay, I know the problem now.

"All" doesn't work. It needs to be set to true

Example:


"auto_model_choice": true
HistoricTheater commented 2 years ago

That seems to be it.

88bells commented 2 years ago

I'm getting the same thing.

  "info": {
    "version": 8.0
  },
  "settings": {
    "auto_site_choice": "onlyfans",
    "export_type": "json",
    "profile_directories": [
      ".profiles"
    ],
    "max_threads": -1,
    "min_drive_space": 0,
    "helpers": {
      "renamer": true,
      "reformat_media": true,
      "downloader": true,
      "delete_empty_directories": false
    },
    "webhooks": {
      "global_webhooks": [],
      "global_status": true,
      "auth_webhook": {
        "succeeded": {
          "webhooks": [],
          "status": null,
          "hide_sensitive_info": true
        },
        "failed": {
          "webhooks": [],
          "status": null,
          "hide_sensitive_info": true
        }
      },
      "download_webhook": {
        "succeeded": {
          "webhooks": [],
          "status": null,
          "hide_sensitive_info": true
        },
        "failed": {
          "webhooks": [],
          "status": null,
          "hide_sensitive_info": true
        }
      }
    },
    "exit_on_completion": false,
    "infinite_loop": false,
    "loop_timeout": "0",
    "dynamic_rules_link": "https://raw.githubusercontent.com/DATAHOARDERS/dynamic-rules/main/onlyfans.json",
    "proxies": [],
    "cert": "",
    "random_string": "53d59d50250a11ecbd1340b0344bbe34"
  },
  "supported": {
    "onlyfans": {
      "settings": {
        "auto_profile_choice": [
          "default"
        ],
        "auto_model_choice": true,
        "auto_media_choice": "0",
        "auto_api_choice": true,
        "browser": {
          "auth": true
        },
        "jobs": {
          "scrape": {
            "subscriptions": true,
            "paid_content": true
          },
          "metadata": {
            "posts": true,
            "comments": true
          }
        },
        "download_directories": [
          ".sites"
        ],
        "file_directory_format": "{site_name}/{model_username}/{api_type}/{value}/{media_type}",
        "filename_format": "{filename}.{ext}",
        "metadata_directories": [
          ".sites"
        ],
        "metadata_directory_format": "{site_name}/{model_username}/Metadata",
        "delete_legacy_metadata": false,
        "text_length": 255,
        "video_quality": "source",
        "overwrite_files": false,
        "date_format": "%d-%m-%Y",
        "ignored_keywords": [],
        "ignore_type": "",
        "blacklists": "",
        "webhook": true
      }
    },
    "fansly": {
      "settings": {
        "auto_profile_choice": [],
        "auto_model_choice": false,
        "auto_media_choice": "",
        "auto_api_choice": true,
        "browser": {
          "auth": true
        },
        "jobs": {
          "scrape": {
            "subscriptions": true,
            "paid_content": true
          },
          "metadata": {
            "posts": true,
            "comments": true
          }
        },
        "download_directories": [
          ".sites"
        ],
        "file_directory_format": "{site_name}/{model_username}/{api_type}/{value}/{media_type}",
        "filename_format": "{filename}.{ext}",
        "metadata_directories": [
          ".sites"
        ],
        "metadata_directory_format": "{site_name}/{model_username}/Metadata",
        "delete_legacy_metadata": false,
        "text_length": 255,
        "video_quality": "source",
        "overwrite_files": false,
        "date_format": "%d-%m-%Y",
        "ignored_keywords": [],
        "ignore_type": "",
        "blacklists": "",
        "webhook": true
      }
    },
    "starsavn": {
      "settings": {
        "auto_profile_choice": "",
        "auto_model_choice": false,
        "auto_media_choice": "",
        "auto_api_choice": true,
        "browser": {
          "auth": true
        },
        "jobs": {
          "scrape": {
            "subscriptions": true,
            "paid_content": true
          },
          "metadata": {
            "posts": true,
            "comments": true
          }
        },
        "download_directories": [
          ".sites"
        ],
        "file_directory_format": "{site_name}/{model_username}/{api_type}/{value}/{media_type}",
        "filename_format": "{filename}.{ext}",
        "metadata_directories": [
          ".sites"
        ],
        "metadata_directory_format": "{site_name}/{model_username}/Metadata",
        "delete_legacy_metadata": false,
        "text_length": 255,
        "video_quality": "source",
        "overwrite_files": false,
        "date_format": "%d-%m-%Y",
        "ignored_keywords": [],
        "ignore_type": "",
        "blacklists": [
          ""
        ],
        "webhook": true
      }
    }
  }
}
DIGITALCRIMINAL commented 2 years ago

"All" for auto choices should be fixed, but the preferred way is to just set it to true

outsdr01 commented 2 years ago

Thank you for the fixes, but unfortunately, I'm still having issues. I created an entirely new install and left the download folder as the default value; the only media being downloaded seems to be avatars and headers. The only other data collected is the metadata.

That's when I select all models, which I do manually by selecting 0.

I tried downloading a single person I subscribe to, and it appears that it is downloading images as well now, which is more than I was getting before. Comma selecting multiple profiles gives the same results as when All are selected.

I don't know Python, so I hope I'm not making a simple mistake somewhere.

I've attached two files, one with the output when I select 0, and the other is when I select a single profile (minus all the output that comes before that, which is already in the other file.)

All.txt Single.txt