UltimaHoarder / UltimaScraper

Scrape all the media from an OnlyFans account - Updated regularly
GNU General Public License v3.0
3.98k stars 608 forks source link

Subscriptions Not Showing #621

Closed pcormier01 closed 3 years ago

pcormier01 commented 3 years ago

Hello,

Everything seems to connect properly and run smoothly to AUTH, but I get those lines after running the PY:

Scraping Paid Content Scraping Subscriptions There's nothing to scrape. Archive Completed in 0.0 Minutes Pausing scraper for 0 seconds.

I have a few subscriptions active.

If you need any help on the documentation I can support you on that side. Love doing it!

Ryki1 commented 3 years ago

hi ! i have the same problem please we need help!

UltimaHoarder commented 3 years ago

@pcormier01 @Ryki1 what does your OnlyFans config.json look like? Specifically the settings key.

pcormier01 commented 3 years ago

@DIGITALCRIMINAL Didn't change anything Config

Ryki1 commented 3 years ago

Capture d’écran 2020-12-22 193909

this ?

UltimaHoarder commented 3 years ago

Try turning scrape_paid content to false

Ryki1 commented 3 years ago

Is works thanks bro ! Wow !

Ryki1 commented 3 years ago

Is only work for onlyfan profile who have chat with me ?

UltimaHoarder commented 3 years ago

There's a bug with my previous commit. Download the latest commit and you'll see their names

Ryki1 commented 3 years ago

How to install ? Should I reinstall all the zip ( Only fans DataScrapter v.6.1)?

UltimaHoarder commented 3 years ago

How to install ? Should I reinstall all the zip ( Only fans DataScrapter v.6.1)?

Download here https://github.com/DIGITALCRIMINAL/OnlyFans/archive/master.zip

You can overwrite the old files with the new

Ryki1 commented 3 years ago

Thanks bro I finish to download so it the same process ? I'll to try

Ryki1 commented 3 years ago

I install but now I don't know how to do for unlock another Onlyfan profile without a chat

onlyfans123 commented 3 years ago

I am seconding this issue. Multiple subscribed accounts, paid and free combo. Did not change config at all but here it is anyhow. I am using the docker image on an OS X.

https://pastebin.pl/view/241bd4ba

zetaomegagon commented 3 years ago

I'm also haveing this issue since updating to latest master. I have both paid and free subscriptions.

Nothing has worked so far.

Current config.json:

{
  "settings": {
    "auto_site_choice": "",
    "export_type": "json",
    "profile_directories": [
      ".profiles"
    ],
    "max_threads": -1,
    "min_drive_space": 0,
    "webhooks": [],
    "exit_on_completion": false,
    "infinite_loop": true,
    "loop_timeout": "0",
    "proxies": [],
    "cert": "",
    "random_string": "f5a73c52463211eb9d00f0189849c53e"
  },
  "supported": {
    "onlyfans": {
      "settings": {
        "auto_profile_choice": "",
        "auto_scrape_names": false,
        "auto_choice": "",
        "auto_scrape_apis": true,
        "jobs": {
          "scrape_names": true,
          "scrape_paid_content": true
        },
        "download_directories": [
          ".sites"
        ],
        "file_directory_format": "{site_name}/{username}/{api_type}/{value}/{date}/{media_type}/{post_id}",
        "filename_format": "{username}_{date}_{value}_{post_id}_{media_id}_{filename}.{ext}",
        "metadata_directories": [
          ".sites"
        ],
        "metadata_directory_format": "{site_name}/{username}/Metadata",
        "text_length": 255,
        "overwrite_files": false,
        "date_format": "%d-%m-%Y",
        "ignored_keywords": [],
        "ignore_type": "",
        "export_metadata": true,
        "delete_legacy_metadata": false,
        "blacklist_name": "",
        "webhook": true
      }
    },
    "patreon": {
      "settings": {
        "auto_profile_choice": "",
        "auto_scrape_names": false,
        "auto_choice": "",
        "auto_scrape_apis": true,
        "download_directories": [
          ".sites"
        ],
        "file_directory_format": "{site_name}/{username}/{api_type}/{value}/{media_type}",
        "filename_format": "{filename}.{ext}",
        "metadata_directories": [
          ".sites"
        ],
        "metadata_directory_format": "{site_name}/{username}/Metadata",
        "text_length": 255,
        "overwrite_files": false,
        "date_format": "%d-%m-%Y",
        "ignored_keywords": [],
        "ignore_type": "",
        "export_metadata": true,
        "delete_legacy_metadata": false,
        "blacklist_name": "",
        "webhook": true
      }
    },
    "starsavn": {
      "settings": {
        "auto_profile_choice": "",
        "auto_scrape_names": false,
        "auto_choice": "",
        "auto_scrape_apis": true,
        "jobs": {
          "scrape_names": true,
          "scrape_paid_content": true
        },
        "download_directories": [
          ".sites"
        ],
        "file_directory_format": "{site_name}/{username}/{api_type}/{value}/{media_type}",
        "filename_format": "{filename}.{ext}",
        "metadata_directories": [
          ".sites"
        ],
        "metadata_directory_format": "{site_name}/{username}/Metadata",
        "text_length": 255,
        "overwrite_files": false,
        "date_format": "%d-%m-%Y",
        "ignored_keywords": [],
        "ignore_type": "",
        "export_metadata": true,
        "delete_legacy_metadata": false,
        "blacklist_name": "",
        "webhook": true
      }
    }
  }
}
zetaomegagon commented 3 years ago

@DIGITALCRIMINAL is there any way to enable any debug logging other than download and error logging?

UltimaHoarder commented 3 years ago

@zetaomegagon I can't debug this because I don't have access to the account

zetaomegagon commented 3 years ago

@DIGITALCRIMINAL When I get some time, I'll try to repro the issue on a dummy account.

RE: debug logging-- I was referencing enabling it locally on my end.

UltimaHoarder commented 3 years ago

It wouldn't change anything since I have to check the API requests

zetaomegagon commented 3 years ago

I mean, I'd give you access to the account 🤷‍♀️

UltimaHoarder commented 3 years ago

I mean, I'd give you access to the account 🤷‍♀️

You can email me the auth details at datahoarders@protonmail.com

@zetaomegagon Do you have access to a VSCode?

zetaomegagon commented 3 years ago

You can email me the auth details at datahoarders@protonmail.com

Sounds good. Will likely get around to it after the Holiday tomorrow (EST).

@zetaomegagon Do you have access to a VSCode?

It's the other editor that I use.

UltimaHoarder commented 3 years ago

You can email me the auth details at datahoarders@protonmail.com

Sounds good. Will likely get around to it after the Holiday tomorrow (EST).

@zetaomegagon Do you have access to a VSCode?

It's the other editor that I use.

Oh ok, you need need to check if offset_array is returning anything api/onlyfans/onlyfans.py image

onlyfans123 commented 3 years ago

I have the exact same code above from the latest master and it still gives me no subs :/ I used to use this tool a while back on windows and had no issues with the same account. Maybe it is related with the docker image?

zetaomegagon commented 3 years ago

@onlyfans123 erm...did you check if offset_array returns anything?

TheSafecracker commented 3 years ago

I came from thread #627 , and my onlyfans.py looks exactly like above. Tried the solutions listed above in this thread, but no luck. Worked fine last night with the newest commit, but this morning, no luck.

onlyfans123 commented 3 years ago

@onlyfans123 erm...did you check if offset_array returns anything?

I can't debug but I inserted these print statements and they don't get printed which means this code is not reached:

def get_subscriptions(self, resume=None, refresh=True, identifiers: list = [], extra_info=True, limit=20, offset=0) -> list[Union[create_subscription, None]]:
        authed = self.auth
        print("here")
        if not authed:
            return []
        if not refresh:
            subscriptions = authed.subscriptions
            return subscriptions
        link = links(global_limit=limit, global_offset=offset).subscriptions
        session = self.sessions[0]
        ceil = math.ceil(authed.subscribesCount / limit)
        a = list(range(ceil))
        offset_array = []
        for b in a:
            b = b * limit
            link = links(global_limit=limit, global_offset=b).subscriptions
            offset_array.append(link)

        print("here too")
        print(offset_array)
TheSafecracker commented 3 years ago

What I found is that if I delete the auth.json and the config.json and run start_ofd.py, I now get "user_agent required for: default" after I select 0 for OnlyFans. So I nuked the auth.json again, and manually filled it in again (I used the OnlyFans Cookie Helper) and changed the support_2fa back to false. Once it was started, it asked for profile/ choose username, so I chose 0 for all - that's when it found my profile and started up. (I had added a line to the script to print the offset array, and it never returned anything; once I saw it was now, I knew things were looking up.) I think that might have fixed the issue. It looks like it's merging profile data now.

ph6sor commented 3 years ago

For people who have used the CookieHelper, you need to either update that extension and run it again, or just add this to your auth.json:

"active": true

I had the same problem as OP and found that after re-initializing the script, the original auth.json has this line in it but then I hastily pasted over it with the cookie info from the CookieHelper which, until the recently updated version 1.0.2, didn't include this line.

TL;DR make sure your auth.json looks like this

Screen Shot 2020-12-27 at 2 14 10 AM

James-MF-Thomas commented 3 years ago

I have that active although I'm not running 2fa; I'll give 2fa try and see if that fixes but as now, I'm still stuck in the paid loop that doesn't download paid content.

Screen Shot 2020-12-27 at 2 46 06 AM

ph6sor commented 3 years ago

@James-MF-Thomas the 2fa value is irrelevant. There's a big difference between active: true and active: false so you might try changing that first.

James-MF-Thomas commented 3 years ago

"active" was set to true, I on;y set it to "false" to expierment

James-MF-Thomas commented 3 years ago

well, it must have been set to something like true. After switching it back, were good to go;

UltimaHoarder commented 3 years ago

Lol, you guys are wasting time. If someone just let me use their account, I could have fixed this day one.

anon767346 commented 3 years ago

how can i help without giving you my credentials? any steps to follow? stuff to check?

muggsyd commented 3 years ago

I resolved mine by making the following change to my auth.json. Previously I did not need this value but since upgrading to 6.1.1 I ran into issues. I reset both my config.json and auth.json and then recreated both

I don't believe I was required to use the "auth_hash" value. This was always blank. However I have used the "fp" value from my exported cookie to use as my "auth_hash" and this has worked correctly

I use "Export Cookies" addin for firefox to get my onlyfans cookie

This is my current auth.json after configuring and confirming it works correct

{ "auth": { "username": "default", "auth_id": "auth_id", "auth_hash": "fp", "authuniq": "", "sess": "sess", "app_token": "33d57ade8c02dbc5a333db99ff9ae26a", "user_agent": "Mozilla\/5.0 (Windows NT 10.0; Win64; x64; rv:84.0) Gecko\/20100101 Firefox\/84.0", "support_2fa": true, "active": true } }.

zetaomegagon commented 3 years ago

@muggsyd tried your fix in two parts:

  1. added fp value to auth_hash
  2. added "active": true

Only after adding "active": true did the issue resolve.

EDIT: I kept the fp value, so both auth_hash and active have values in my auth.json

UltimaHoarder commented 3 years ago

how can i help without giving you my credentials? any steps to follow? stuff to check?

Everything is working for me, so I wouldn't know.

I resolved mine by making the following change to my auth.json. Previously I did not need this value but since upgrading to 6.1.1 I ran into issues. I reset both my config.json and auth.json and then recreated both

I don't believe I was required to use the "auth_hash" value. This was always blank. However I have used the "fp" value from my exported cookie to use as my "auth_hash" and this has worked correctly

I use "Export Cookies" addin for firefox to get my onlyfans cookie

This is my current auth.json after configuring and confirming it works correct

{ "auth": { "username": "default", "auth_id": "auth_id", "auth_hash": "fp", "authuniq": "", "sess": "sess", "app_token": "33d57ade8c02dbc5a333db99ff9ae26a", "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:84.0) Gecko/20100101 Firefox/84.0", "support_2fa": true, "active": true } }.

FP token doesn't do anything, you might as well just be putting random symbols in auth_hash.

- What I learned from this thread, is that it's most likely just a user error.

anon767346 commented 3 years ago

I managed to solve the issue. what i did:

The issue occurred because of a misconfiguration of config.json: setting the auto_profile_choice to the wrong value would give the "nothing to scrape" problem.

to automatically scrape all it's now set to "0" (in quotes, json value is a string)

zetaomegagon commented 3 years ago

@DIGITALCRIMINAL while the orig issue in this thread is user error, when upgrading-- where the auth config.json object is pulled out and placed into auth.json-- the fact that the user-agent is not populated does cause some confusion.

i.e. for a lot of users this wouldn't have been a problem if the user-agent string was copied into auth.json along with the rest of the auth object.

UltimaHoarder commented 3 years ago

@DIGITALCRIMINAL while the orig issue in this thread is user error, when upgrading-- where the auth config.json object is pulled out and placed into auth.json-- the fact that the user-agent is not populated does cause some confusion.

i.e. for a lot of users this wouldn't have been a problem if the user-agent string was copied into auth.json along with the reset of the auth object.

I didn't allow the script to import the previous user agent because it would potentially log people out of their accounts.

UltimaHoarder commented 3 years ago

@DIGITALCRIMINAL while the orig issue in this thread is user error, when upgrading-- where the auth config.json object is pulled out and placed into auth.json-- the fact that the user-agent is not populated does cause some confusion. i.e. for a lot of users this wouldn't have been a problem if the user-agent string was copied into auth.json along with the reset of the auth object.

I didn't allow the script to import the previous user agent because it would potentially log people out of their accounts.

Also just wanted to say, the script doesn't log you into the account UNLESS there's a user-agent, so it forces you to add it.

zetaomegagon commented 3 years ago

@DIGITALCRIMINAL sounds like upgrade docs to me

UltimaHoarder commented 3 years ago

@DIGITALCRIMINAL sounds like upgrade docs to me

I updated the readme with an example of the auth.

UltimaHoarder commented 3 years ago

I helped @James-MF-Thomas and the solution to this problem is to literally just set active = True and update your credentials as others were saying. That's all.