datawhores / OF-Scraper

A completely revamped and redesigned fork, reimagined from scratch based on the original onlyfans-scraper
MIT License
674 stars 56 forks source link

Not scrapping #47

Closed Troll180 closed 1 year ago

Troll180 commented 1 year ago

Hey, I'm new to this, using the last version (1.95.1) I made the auth.json and config but when I launch the script, it download nothing, here's what I got. I probably missed something but I don't know what, if you could help, thanks!

Using profile: main_profile
Status - UP
? Modify filters for your accounts list?
Some usage examples are scraping free accounts only or paid accounts without renewal No
Welcome, Xxxx | xxxxx
? Which models do you want to scrape: xxxx
                                                                                                          scraper.py:516
===========================
Script Finished
Run Time:  0:00:09
===========================
datawhores commented 1 year ago

Can you rerun the same command but with --output debug append

1 Did you select all models 2 what type of media purchased timeline where you trying to scrape

  1. You need to have access to the timeline on onlyfans.com for the scraper to work.

Otherwise only messages and paid content can be scraped

Troll180 commented 1 year ago

To answer you 1 -> I selected one model only 2 -> It doesnt offer me to chose what I want to scrape 3 -> It's a free model actually, I tried with a paid one but it's the same thing

Here's what I got with the --output debug : Thanks!

PS D:\Program Files (x86)\Python\OF-Scraper-1.95.1> ofscraper --output debug
DEBUG    Namespace(username=None, daemon=None, log='OFF', discord='OFF', output='DEBUG',     scraper.py:59
         dupe=False, posts=[], letter_count=False, action=None, before=None, after=None,
         account_type=None, renewal=None, sub_status=None)
? What would you like to do? Download content from a user
DEBUG    final mp4decrypt path bin/mp4decrypt.exe                                           scraper.py:434
DEBUG    final ffmpeg path bin/ffmpeg.exe                                                   scraper.py:435
                                                                                            scraper.py:505
==============================
starting script
==============================

Using profile: main_profile
Status - UP
? Modify filters for your accounts list?
Some usage examples are scraping free accounts only or paid accounts without renewal No
Welcome, Xxxxx | xxxxx
? Which models do you want to scrape: ailurful
TRACEBACK failed with exception: 'moder_username'                                           scraper.py:347
TRACEBACK Traceback (most recent call last):                                                scraper.py:348
            File "D:\Program Files (x86)\Python\lib\site-packages\ofscraper\scraper.py",
          line 337, in process_post
              create_tables(model_id,ele['name'])
            File "D:\Program Files (x86)\Python\lib\site-packages\ofscraper\scraper.py",
          line 489, in create_tables
              operations.create_post_table(model_id,username)
            File "D:\Program Files
          (x86)\Python\lib\site-packages\ofscraper\db\operations.py", line 56, in
          create_post_table
              datebase_path =databasePathHelper(model_id,username)
            File "D:\Program Files
          (x86)\Python\lib\site-packages\ofscraper\utils\paths.py", line 48, in
          databasePathHelper
              return pathlib.Path(formatStr.format(configpath=homeDir /
          constants.configPath,profile=profiles.get_current_profile(),model_username=userna
          me,username=username,model_id=model_id,sitename="Onlyfans",site_name="Onlyfans",f
          irst_letter=username[0],save_location=config_.get_save_location(config_.read_conf
          ig())),"user_data.db")
          KeyError: 'moder_username'

                                                                                            scraper.py:516
===========================
Script Finished
Run Time:  0:00:26
===========================
datawhores commented 1 year ago

I think it is your config file can you show it?

Troll180 commented 1 year ago

Here you go, I come from the DC scrapper, tried to put similar stuff but I probably did something wrong somewhere then I guess :p

{
    "config": {
        "main_profile": "main_profile",
        "save_location": "/OF Rip",
        "file_size_limit": 0,
        "dir_format": "{model_username}/{responsetype}/{mediatype}/",
        "file_format": "{filename} [{date}] {media_id}.{ext}",
        "textlength": 10,
        "date": "YYYY-MM-DD",
        "metadata": "{configpath}/{moder_username}/Metadata",
        "filter": [
            "Images",
            "Audios",
            "Videos"
        ],
        "mp4decrypt": "bin/mp4decrypt.exe",
        "ffmpeg": "bin/ffmpeg.exe",
        "discord": "",
        "responsetype": {
            "timeline": "posts",
            "message": "messages",
            "archived": "archived",
            "paid": "messages",
            "stories": "stories",
            "highlights": "stories",
            "profile": "profile",
            "pinned": "posts"
        }
    }
}
datawhores commented 1 year ago

moder_username Should be model_username

Spelling is important for the parser

Troll180 commented 1 year ago

I don't know how I could've missed that one, ugh, sorry and thanks for your help, it work fine now. Do you plan on making it work for Fansly too in future? :)

datawhores commented 1 year ago

Most likely I don't have a timeline of when