DIGITALCRIMINAL / ArchivedUltimaScraper

Scrape content from OnlyFans and Fansly
GNU General Public License v3.0
932 stars 39 forks source link

[OnlyFans] 'AuthModel' object has no attribute 'auth_details' #1008

Open RayRho93 opened 1 year ago

RayRho93 commented 1 year ago

Hello, I am getting the following error after updating to the latest:

File "...\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-D7yaGwKQ-py3.11\Lib\site-packages\ultima_scraper_collection\modules\module_streamliner.py", line 415, in account_setup auth.auth_details.email ^^^^^^^^^^^^^^^^^ AttributeError: 'AuthModel' object has no attribute 'auth_details'. Did you mean: 'get_auth_details'?

Javier-bat commented 1 year ago
1. poetry cache clear pypi --all
2. python updater.py
3. poetry run python start_us.py
kwangsing3 commented 1 year ago
1. poetry cache clear pypi --all
2. python updater.py
3. poetry run python start_us.py

image

Same issue, and no luck

kwangsing3 commented 1 year ago

more details

Choose Sites: 0 = All | 1 = OnlyFans | 2 = Fansly
1
Choose Profiles: 0 = All | 1 = default
1
Traceback (most recent call last):
  File "/home//UltimaScraper/start_us.py", line 62, in <module>
    asyncio.run(main())
  File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/home//UltimaScraper/start_us.py", line 44, in main
    _api = await USR.start(
  File "/home//UltimaScraper/ultima_scraper/ultima_scraper.py", line 50, in start
    await self.start_datascraper(datascraper)
  File "/home//UltimaScraper/ultima_scraper/ultima_scraper.py", line 88, in start_datascraper
    setup, _subscriptions = await datascraper.account_setup(
  File "/root/.cache/pypoetry/virtualenvs/ultima-scraper-QjxTFIwQ-py3.10/lib/python3.10/site-packages/ultima_scraper_collection/modules/module_streamliner.py", line 415, in account_setup
    auth.auth_details.email
AttributeError: 'AuthModel' object has no attribute 'auth_details'. Did you mean: 'get_auth_details'?
Javier-bat commented 1 year ago

more details

Choose Sites: 0 = All | 1 = OnlyFans | 2 = Fansly
1
Choose Profiles: 0 = All | 1 = default
1
Traceback (most recent call last):
  File "/home//UltimaScraper/start_us.py", line 62, in <module>
    asyncio.run(main())
  File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/home//UltimaScraper/start_us.py", line 44, in main
    _api = await USR.start(
  File "/home//UltimaScraper/ultima_scraper/ultima_scraper.py", line 50, in start
    await self.start_datascraper(datascraper)
  File "/home//UltimaScraper/ultima_scraper/ultima_scraper.py", line 88, in start_datascraper
    setup, _subscriptions = await datascraper.account_setup(
  File "/root/.cache/pypoetry/virtualenvs/ultima-scraper-QjxTFIwQ-py3.10/lib/python3.10/site-packages/ultima_scraper_collection/modules/module_streamliner.py", line 415, in account_setup
    auth.auth_details.email
AttributeError: 'AuthModel' object has no attribute 'auth_details'. Did you mean: 'get_auth_details'?

Try this

sudo rm -rf /root/.cache/pypoetry
python updater.py
poetry run python start_us.py
kwangsing3 commented 1 year ago

Try this

sudo rm -rf /root/.cache/pypoetry
python updater.py
poetry run python start_us.py

@RayRho93 @Javier-bat

Sorry still failed, but I do read into the api code and found the solution.

Change my auth.json

"Cookie": "csrf=*; auth_id=*; cookiesAccepted=all; __stripe_mid=*; fp=*; sess=*; st=*; ref_src=",

into

"Cookie": "csrf=*; auth_id=*; cookiesAccepted=all; __stripe_mid=*; fp=*; sess=*; st=*;",

I found that because the origin module code says I failed in login, after that I look into my session. That was the cookie I just copy all from the website, and noticed that wired end.

Remove that no end key, and login successfully good luck and hope this works : )

Prophaniti86 commented 1 year ago

Try this

sudo rm -rf /root/.cache/pypoetry
python updater.py
poetry run python start_us.py

@RayRho93 @Javier-bat

Sorry still failed, but I do read into the api code and found the solution.

Change my auth.json

"Cookie": "csrf=*; auth_id=*; cookiesAccepted=all; __stripe_mid=*; fp=*; sess=*; st=*; ref_src=",

into

"Cookie": "csrf=*; auth_id=*; cookiesAccepted=all; __stripe_mid=*; fp=*; sess=*; st=*;",

I found that because the origin module code says I failed in login, after that I look into my session. That was the cookie I just copy all from the website, and noticed that wired end.

Remove that no end key, and login successfully good luck and hope this works : )

Tried that, just takes me back to the original error and ends up re-writing the auth.json to the original one

kwangsing3 commented 1 year ago

Tried that, just takes me back to the original error and ends up re-writing the auth.json to the original one

@Prophaniti86 How may I help you? What's your original error?

About the original auth.json, I met two case, of that:

  1. Return to empty auth.json The default auth.json permissions it created is --------- (even it depends on OS, but my case is) And it may unable to read when process started.

    sudo chmod 777 __user_data__/profiles/default/auth.json
  2. Cookie I set that exists but no use: just simply edit active key in JSON, auth.json will be disable and re-write when hit the error, so I've to re-edit auth.json every time I hit errors.

    {
    ...
    "Cookie": "....",
    "active":  true,
    ...
    }
Prophaniti86 commented 1 year ago

O after making those changes, it goes back to AttributeError: 'AuthModel' object has no attribute 'auth_details'. Did you mean: 'get_auth_details'?

and then it re-rites the auth.json to be the original version before the changes you suggested

kwangsing3 commented 1 year ago

try

ls -l   __user_data__/profiles/default/auth.json

What's it say ?

I had your problem because my auth.json didn't had right permissions. After I changed it into chmod 777 then problem fixed.

make sure

cat  __user_data__/profiles/default/auth.json

is the right content you feed before launch anything.

or try remove auth.json, and generated new one ? perhaps it has wrong permissions.

Prophaniti86 commented 1 year ago

Hey I'm doing all this in command prompt and i honestly know nothing. Sorry, but I need like.. really basic instructions lol

RayRho93 commented 1 year ago

My original issue is fixed but after a while of the scraper running, I am getting this error now

Traceback (most recent call last): File "C:\Users\Raul_\Documents\GitHub\OnlyFans\start_us.py", line 62, in asyncio.run(main()) File "C:\Program Files\Python311\Lib\asyncio\runners.py", line 190, in run return runner.run(main) ^^^^^^^^^^^^^^^^ File "C:\Program Files\Python311\Lib\asyncio\runners.py", line 118, in run return self._loop.run_until_complete(task) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Program Files\Python311\Lib\asyncio\base_events.py", line 653, in run_untilcomplete return future.result() ^^^^^^^^^^^^^^^ File "C:\Users\Raul\Documents\GitHub\OnlyFans\start_us.py", line 44, in main api = await USR.start( ^^^^^^^^^^^^^^^^ File "C:\Users\Raul\Documents\GitHub\OnlyFans\ultima_scraper\ultima_scraper.py", line 50, in start await self.startdatascraper(datascraper) File "C:\Users\Raul\Documents\GitHub\OnlyFans\ultima_scraper\ultima_scraper.py", line 137, in start_datascraper await datascraper.datascraper.api.job_manager.processjobs() File "C:\Users\Raul\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-D7yaGwKQ-py3.11\Lib\site-packages\ultima_scraper_api\managers\job_manager\job_manager.py", line 45, in process_jobs await asyncio.create_task(self._worker()) File "C:\Users\Raul\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-D7yaGwKQ-py3.11\Lib\site-packages\ultima_scraper_api\managers\job_manager\job_manager.py", line 53, in _worker await job.task File "C:\Users\Raul\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-D7yaGwKQ-py3.11\Lib\site-packages\ultima_scraper_collection\modules\module_streamliner.py", line 174, in prepare_scraper unrefined_set: list[message_types | Any] = await user.getmessages() ^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Raul\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-D7yaGwKQ-py3.11\Lib\site-packages\ultima_scraper_api\apis\onlyfans\classes\user_model.py", line 359, in get_messages has_more = results[-1]["hasMore"] if results else False


KeyError: 'hasMore'
kwangsing3 commented 1 year ago

My original issue is fixed but after a while of the scraper running, I am getting this error now

Have you tried only fetch one instead of fetch all at one round?

This repo author said may some post content would break the scipt, and I've no idea how you get there : (

Did you get some part of file to downdload?

Prophaniti86 commented 1 year ago

I got it all working and I have no idea how lol

Prophaniti86 commented 1 year ago

I got it all working and I have no idea how lol

Nevermind, I got it scraping everyone, then it gets to processing and crashes. Here's the traceback

EDIT: I turned off paid content and messages and it works now. I think it misses items though

EDIT 2: Crashes on a specific profile

Traceback (most recent call last): File "C:\Users\Matt\Desktop\UltimaScraper-master\start_us.py", line 62, in asyncio.run(main()) File "C:\Program Files\Python310\lib\asyncio\runners.py", line 44, in run return loop.run_until_complete(main) File "C:\Program Files\Python310\lib\asyncio\base_events.py", line 649, in run_until_complete return future.result() File "C:\Users\Matt\Desktop\UltimaScraper-master\start_us.py", line 44, in main _api = await USR.start( File "C:\Users\Matt\Desktop\UltimaScraper-master\ultima_scraper\ultima_scraper.py", line 50, in start await self.start_datascraper(datascraper) File "C:\Users\Matt\Desktop\UltimaScraper-master\ultima_scraper\ultima_scraper.py", line 137, in start_datascraper await datascraper.datascraper.api.job_manager.process_jobs() File "C:\Users\Matt\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-yfGR4bq4-py3.10\lib\site-packages\ultima_scraper_api\managers\job_manager\job_manager.py", line 45, in process_jobs await asyncio.create_task(self.worker()) File "C:\Users\Matt\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-yfGR4bq4-py3.10\lib\site-packages\ultima_scraper_api\managers\job_manager\job_manager.py", line 53, in worker await job.task File "C:\Users\Matt\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-yfGR4bq4-py3.10\lib\site-packages\ultima_scraper_collection\modules\module_streamliner.py", line 202, in prepare_scraper await self.process_scraped_content( File "C:\Users\Matt\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-yfGR4bq4-py3.10\lib\site-packages\ultima_scraper_collection\modules\module_streamliner.py", line 237, in process_scraped_content unrefined_set: list[dict[str, Any]] = await tqdm_asyncio.gather( File "C:\Users\Matt\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-yfGR4bq4-py3.10\lib\site-packages\tqdm\asyncio.py", line 79, in gather res = [await f for f in cls.as_completed(ifs, loop=loop, timeout=timeout, File "C:\Users\Matt\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-yfGR4bq4-py3.10\lib\site-packages\tqdm\asyncio.py", line 79, in res = [await f for f in cls.as_completed(ifs, loop=loop, timeout=timeout, File "C:\Program Files\Python310\lib\asyncio\tasks.py", line 571, in _wait_for_one return f.result() # May raise f.exception(). File "C:\Users\Matt\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-yfGR4bq4-py3.10\lib\site-packages\tqdm\asyncio.py", line 76, in wrap_awaitable return i, await f File "C:\Users\Matt\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-yfGR4bq4-py3.10\lib\site-packages\ultima_scraper_collection\managers\datascraper_manager\datascrapers\onlyfans.py", line 61, in media_scraper reformat_item = reformat_manager.prepare_reformat(asset) File "C:\Users\Matt\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-yfGR4bq4-py3.10\lib\site-packages\ultima_scraper_renamer\reformat.py", line 372, in prepare_reformat download_path = self.filesystem_manager.get_directory_manager( File "C:\Users\Matt\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-yfGR4bq4-py3.10\lib\site-packages\ultima_scraper_collection\managers\filesystem_manager.py", line 55, in get_directory_manager return self.directory_manager_users[user_id] Keyerror:

kwangsing3 commented 1 year ago

Looks like some content on specific post crash the script. Are there emojis ? that's known issues write on README.

vinsh3nt commented 1 year ago

Running into the same AuthModel issue on latest commit

RayRho93 commented 1 year ago

Updated to the latest commit and I am having the same AuthModel issue again. I made sure to redo the auth.json and I am still running into the error.

anon767346 commented 1 year ago

Same issue with the docker container, any workarounds yet?

kotori2 commented 1 year ago

This error means the user was not successfully authenticated. For me, my config.json file for dynamic rules link was not automatically updated from

"dynamic_rules_link": "https://raw.githubusercontent.com/DATAHOARDERS/dynamic-rules/main/onlyfans.json",

to

"dynamic_rules_link": "https://raw.githubusercontent.com/DIGITALCRIMINALS/dynamic-rules/main/onlyfans.json",

I have no idea why this rule works for half a year without any update. Manually updating this make everything works.

TheAEllis commented 1 year ago

I got a little farther by going into D:\Users\<USERNAME>\AppData\Local\pypoetry\Cache\virtualenvs\ultima-scraper-vH07-f3y-py3.10\Lib\site-packages\ultima_scraper_collection\modules\module_streamliner.py and commenting out lines 415, 416, and 417 as such:

elif (

"""             auth.auth_details.email
            and auth.auth_details.password
            and site_settings.browser.auth """
        ):

Alas, while that allows me to pass by the auth_details bug, the script gets stuck on the Choose Medias prompt, and just asks over and over without beginning any downloads.

TheAEllis commented 1 year ago

I get the same behavior when I edit lines 414-418 to be

elif(
     authed.get_auth_details.email
     and authed.auth_details.password
     and site_settings.browser.auth
):

which may be closer to the intended fix.