DIGITALCRIMINAL / ArchivedUltimaScraper

Scrape content from OnlyFans and Fansly
GNU General Public License v3.0
948 stars 39 forks source link

Traceback Error: AttributeError: 'str' object has no attribute 'get' #518

Closed Nostang3 closed 1 year ago

Nostang3 commented 2 years ago

Just started this morning. Tried updating all modules in requirements. Running 7.6.1. Tried changing the auto_api_choice to "2,3,4,5,6,7" that I saw in another thread with no change.

0
Scrape Processing
Name: enjinight
Type: Profile
Type: Stories
Traceback (most recent call last):
  File "D:\OnlyFans\start_ofd.py", line 104, in <module>
    asyncio.run(main())
  File "C:\Users\chris\AppData\Local\Programs\Python\Python310\lib\asyncio\runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "C:\Users\chris\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 641, in run_until_complete
    return future.result()
  File "D:\OnlyFans\start_ofd.py", line 86, in main
    api = await main_datascraper.start_datascraper(
  File "D:\OnlyFans\datascraper\main_datascraper.py", line 94, in start_datascraper
    await main_helper.process_names(
  File "D:\OnlyFans\helpers\main_helper.py", line 966, in process_names
    result = await module.start_datascraper(authed, username, site_name)
  File "D:\OnlyFans\modules\onlyfans.py", line 161, in start_datascraper
    await prepare_scraper(authed, site_name, item)
  File "D:\OnlyFans\modules\onlyfans.py", line 784, in prepare_scraper
    highlights = await subscription.get_highlights()
  File "D:\OnlyFans\apis\onlyfans\classes\create_user.py", line 270, in get_highlights
    results = [create_highlight(x) for x in results]
  File "D:\OnlyFans\apis\onlyfans\classes\create_user.py", line 270, in <listcomp>
    results = [create_highlight(x) for x in results]
  File "D:\OnlyFans\apis\onlyfans\classes\create_highlight.py", line 3, in __init__
    self.id: int = option.get("id")
AttributeError: 'str' object has no attribute 'get'
lordoffools commented 2 years ago

Getting this error today, too.

No changes in the version or code I've been running. It was working last night.

Did something change on the OF side, or perhaps on the shared dynamic library side on our end?

ethanhen commented 2 years ago

i changed the config to not scrape stories and it works

Nostang3 commented 2 years ago

@ethanhen If changing the auto_api_choice I mentioned in my post then it didn't work for me. If it's not then please let know what to change.

ethanhen commented 2 years ago

did you also change auto_media_choice?

Nostang3 commented 2 years ago

I did not. What should it be?

ethanhen commented 2 years ago

i have both set to 2,3,4,5,6,7

Nostang3 commented 2 years ago

Now I get this error:

0
Scrape Processing
Name: enjinight
Traceback (most recent call last):
  File "D:\OnlyFans\start_ofd.py", line 104, in <module>
    asyncio.run(main())
  File "C:\Users\chris\AppData\Local\Programs\Python\Python310\lib\asyncio\runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "C:\Users\chris\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 641, in run_until_complete
    return future.result()
  File "D:\OnlyFans\start_ofd.py", line 86, in main
    api = await main_datascraper.start_datascraper(
  File "D:\OnlyFans\datascraper\main_datascraper.py", line 94, in start_datascraper
    await main_helper.process_names(
  File "D:\OnlyFans\helpers\main_helper.py", line 966, in process_names
    result = await module.start_datascraper(authed, username, site_name)
  File "D:\OnlyFans\modules\onlyfans.py", line 144, in start_datascraper
    api_array = scrape_choice(authed, subscription)
  File "D:\OnlyFans\modules\onlyfans.py", line 176, in scrape_choice
    choice_list = main_helper.choose_option(media_types, auto_media_choice)
  File "D:\OnlyFans\helpers\main_helper.py", line 906, in choose_option
    new_name = names[int(value)]
IndexError: list index out of range

Relevant Section:


"settings": {
        "auto_profile_choice": "1",
        "auto_model_choice": false,
        "auto_media_choice": "2,3,4,5,6,7",
        "auto_api_choice": "2,3,4,5,6,7",
        "browser": {
          "auth": true
        },
ethanhen commented 2 years ago

7 might not be applicable then, i guess i have a higher range? try only going up to 6

Nostang3 commented 2 years ago

Changing it to 6 produced the same error.

ethanhen commented 2 years ago

keep going down, i dunno. what are options you have when you set both to false? i.e. what options do you have for both

naodrej commented 2 years ago

In the file apis\onlyfans\classes\create_user.py change the line 270 from: results = [create_highlight(x) for x in results] to results = [create_highlight(x) for x in results.get("data")]

Don't need to change your auto_media_choice or auto_api_choice

RonnieBlaze commented 2 years ago

results = [create_highlight(x) for x in results.get("data")]

This fix seems to be working so far.. I am not sure what version i am on settings say 7.2 but i think thats wrong... I can not get the newest 7.6.1 to work right.

PurpleDrank21 commented 2 years ago

In the file apis\onlyfans\classes\create_user.py change the line 270 from: results = [create_highlight(x) for x in results] to results = [create_highlight(x) for x in results.get("data")]

Don't need to change your auto_media_choice or auto_api_choice

This solved the issue for me.

naodrej commented 2 years ago

results = [create_highlight(x) for x in results.get("data")]

This fix seems to be working so far.. I am not sure what version i am on settings say 7.2 but i think thats wrong... I can not get the newest 7.6.1 to work right.

7.2 is correct. Don't worry about that. It still says 7.2 there on the release version 7.6.1

Nostang3 commented 2 years ago

I didn't think of setting them to false to check, good idea.

0
Scrape Processing
Name: enjinight
Scrape: 0 = All | 1 = Images | 2 = Videos | 3 = Audios | 4 = Texts
1
Apis: 0 = All | 1 = Profile | 2 = Stories | 3 = Posts | 4 = Archived | 5 = Messages

I set it to the following settings and it still tries to search for stories. It's like it's not reading the config file.

"settings": {
        "auto_profile_choice": "1",
        "auto_model_choice": false,
        "auto_media_choice": "0",
        "auto_api_choice": "1,3,4,5",
        "browser": {
          "auth": true
        }

However changing line 270 has worked for me as it's now scraping with changing media_choice and api back to auto.

RonnieBlaze commented 2 years ago

results = [create_highlight(x) for x in results.get("data")]

This fix seems to be working so far.. I am not sure what version i am on settings say 7.2 but i think thats wrong... I can not get the newest 7.6.1 to work right.

7.2 is correct. Don't worry about that. It still says 7.2 there on the release version 7.6.1

for what ever reason i can not get 7.6.1 to work. I updated my python to 3.10.5, ran the the new poetry commands to install and set it up, updated the requirements.txt (which then broke my old version) I had to restore a system back up from yesterday to atleast get me old version working.

lordoffools commented 2 years ago

In the file apis\onlyfans\classes\create_user.py change the line 270 from: results = [create_highlight(x) for x in results] to results = [create_highlight(x) for x in results.get("data")]

Don't need to change your auto_media_choice or auto_api_choice

This worked for me, as well!

Thanks for this!

dpsbrdi commented 2 years ago

In the file apis\onlyfans\classes\create_user.py change the line 270 from: results = [create_highlight(x) for x in results] to results = [create_highlight(x) for x in results.get("data")]

Don't need to change your auto_media_choice or auto_api_choice

for those using latest commit apis\onlyfans\classes\user_model.py line 283 is where you need to look for this instead

Proutbedaine commented 2 years ago

In the file apis\onlyfans\classes\create_user.py change the line 270 from: results = [create_highlight(x) for x in results] to results = [create_highlight(x) for x in results.get("data")] Don't need to change your auto_media_choice or auto_api_choice

for those using latest commit apis\onlyfans\classes\user_model.py line 283 is where you need to look for this instead

This one worked for me I can confirm.

Fall0n commented 2 years ago

In the file apis\onlyfans\classes\create_user.py change the line 270 from: results = [create_highlight(x) for x in results] to results = [create_highlight(x) for x in results.get("data")] Don't need to change your auto_media_choice or auto_api_choice

for those using latest commit apis\onlyfans\classes\user_model.py line 283 is where you need to look for this instead

This one worked for me I can confirm.

the line for me is 282 not 283 work fine did change it.. Your post did help me. Thanks

simonchin513 commented 2 years ago

In the file apis\onlyfans\classes\create_user.py change the line 270 from: results = [create_highlight(x) for x in results] to results = [create_highlight(x) for x in results.get("data")]

Don't need to change your auto_media_choice or auto_api_choice

Ayy it worked

otisrush commented 2 years ago

In the file apis\onlyfans\classes\create_user.py change the line 270 from: results = [create_highlight(x) for x in results] to results = [create_highlight(x) for x in results.get("data")] Don't need to change your auto_media_choice or auto_api_choice

for those using latest commit apis\onlyfans\classes\user_model.py line 283 is where you need to look for this instead

Did the trick for me!

cuenta111 commented 2 years ago

In the file apis\onlyfans\classes\create_user.py change the line 270 from: results = [create_highlight(x) for x in results] to results = [create_highlight(x) for x in results.get("data")]

Don't need to change your auto_media_choice or auto_api_choice

after doing this , when i enter the profile in scraping, it says finished scraping but doesnt scrape a single content

cuenta111 commented 2 years ago

GETTING THIS ERROR WHEN I TRY TO CREATE .JOHN AND CONFIG FILE

c:\of>python start_ofd.py Traceback (most recent call last): File "c:\of\start_ofd.py", line 9, in from helpers.main_helper import OptionsFormat File "c:\of\helpers\main_helper.py", line 21, in import classes.make_settings as make_settings File "c:\of\classes\make_settings.py", line 6, in from yarl import URL ModuleNotFoundError: No module named 'yarl'

PLEASE HELP ME SOLVE THIS ISSUE

stardock commented 2 years ago

You need to have the module installed:
pip install yarl

misterscraper commented 2 years ago

I now get this error

Traceback (most recent call last): File "/Users/thegoat/Downloads/OnlyFans-7.6.1/start_ofd.py", line 66, in <module> asyncio.run(main()) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/runners.py", line 44, in run return loop.run_until_complete(main) File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete return future.result() File "/Users/thegoat/Downloads/OnlyFans-7.6.1/start_ofd.py", line 52, in main api = await main_datascraper.start_datascraper(config, site_name) File "/Users/thegoat/Downloads/OnlyFans-7.6.1/datascraper/main_datascraper.py", line 131, in start_datascraper await default(datascraper) File "/Users/thegoat/Downloads/OnlyFans-7.6.1/datascraper/main_datascraper.py", line 55, in default await main_helper.process_profiles(api, global_settings) File "/Users/thegoat/Downloads/OnlyFans-7.6.1/helpers/main_helper.py", line 963, in process_profiles json_auth = temp_json_auth["auth"] KeyError: 'auth'

darkplace77 commented 2 years ago

In the file apis\onlyfans\classes\create_user.py change the line 270 from: results = [create_highlight(x) for x in results] to results = [create_highlight(x) for x in results.get("data")]

Don't need to change your auto_media_choice or auto_api_choice

I was getting that error. 0 Type: Profile Type: Stories Traceback (most recent call last): File "F:\scrp\ofs\OnlyFans-7.6.1\start_ofd.py", line 104, in <module> asyncio.run(main()) File "C:\Users\xxx\AppData\Local\Programs\Python\Python310\lib\asyncio\runners.py", line 44, in run return loop.run_until_complete(main) File "C:\Users\xxx\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 646, in run_until_complete return future.result() File "F:\scrp\ofs\OnlyFans-7.6.1\start_ofd.py", line 86, in main api = await main_datascraper.start_datascraper( File "F:\scrp\ofs\OnlyFans-7.6.1\datascraper\main_datascraper.py", line 94, in start_datascraper await main_helper.process_names( File "F:\scrp\ofs\OnlyFans-7.6.1\helpers\main_helper.py", line 966, in process_names result = await module.start_datascraper(authed, username, site_name) File "F:\scrp\ofs\OnlyFans-7.6.1\modules\onlyfans.py", line 161, in start_datascraper await prepare_scraper(authed, site_name, item) File "F:\scrp\ofs\OnlyFans-7.6.1\modules\onlyfans.py", line 784, in prepare_scraper highlights = await subscription.get_highlights() File "F:\scrp\ofs\OnlyFans-7.6.1\apis\onlyfans\classes\create_user.py", line 270, in get_highlights results = [create_highlight(x) for x in results] File "F:\scrp\ofs\OnlyFans-7.6.1\apis\onlyfans\classes\create_user.py", line 270, in <listcomp> results = [create_highlight(x) for x in results] File "F:\scrp\ofs\OnlyFans-7.6.1\apis\onlyfans\classes\create_highlight.py", line 3, in __init__ self.id: int = option.get("id") AttributeError: 'str' object has no attribute 'get'

and your solution works for me! thanks.

jp1151419 commented 2 years ago

In the file apis\onlyfans\classes\create_user.py change the line 270 from: results = [create_highlight(x) for x in results] to results = [create_highlight(x) for x in results.get("data")]

Don't need to change your auto_media_choice or auto_api_choice

That didn't fix it for me :( I'm currently on the latest build and found this on line 270 (not 283 as others said) but now I get a new error:

AttributeError: 'str' object has no attribute 'get'
Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x000001AE653DA320>
Traceback (most recent call last):
  File "C:\Users\user1\AppData\Local\Programs\Python\Python310\lib\asyncio\proactor_events.py", line 116, in __del__
    self.close()
  File "C:\Users\user1\AppData\Local\Programs\Python\Python310\lib\asyncio\proactor_events.py", line 108, in close
    self._loop.call_soon(self._call_connection_lost, None)
  File "C:\Users\user1\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 745, in call_soon
    self._check_closed()
  File "C:\Users\user1\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 510, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
twist3dimages commented 2 years ago

i don't seem to have the createuser.py file.. and I don't see it in the repo either

edit: it's user_model.py not create_user.py

naodrej commented 2 years ago

i don't seem to have the createuser.py file.. and I don't see it in the repo either

edit: it's user_model.py not create_user.py

it's create_user.py for 7.6.1 and user_model.py for later versions

aroundtheblockchainer commented 2 years ago

In the file apis\onlyfans\classes\create_user.py change the line 270 from: results = [create_highlight(x) for x in results] to results = [create_highlight(x) for x in results.get("data")]

Don't need to change your auto_media_choice or auto_api_choice

This worked for me too..

bigfranko commented 2 years ago

Ok, so I did changed the line and ran it. It worked for a couple of accounts then I got the error again.

aroundtheblockchainer commented 2 years ago

Ok, so I did changed the line and ran it. It worked for a couple of accounts then I got the error again.

Don't know what to tell you bro. It worked for me with every account I used after changing it.

JamiDEV commented 2 years ago

Ok, so I did changed the line and ran it. It worked for a couple of accounts then I got the error again.

You will have to try an older or newer version if youre still getting the errors. I have 3 versions of scripts I run. Also what IDE or command line are you using? I use pycharm to run the scripts.

aroundtheblockchainer commented 2 years ago

You will have to try an older or newer version if youre still getting the errors. I have 3 versions of scripts I run. Also what IDE or command line are you using? I use pycharm to run the scripts.

How does running through pycharm make a difference? I just run the scripts through normal multiple cmd windows if I need to run multiple rips. Does pycharm offer any functionality that would be more useful/efficient?

RB1875 commented 2 years ago

In the file apis\onlyfans\classes\create_user.py change the line 270 from: results = [create_highlight(x) for x in results] to results = [create_highlight(x) for x in results.get("data")] Don't need to change your auto_media_choice or auto_api_choice

for those using latest commit apis\onlyfans\classes\user_model.py line 283 is where you need to look for this instead

Giving it a run now, and it appears that it's working for now, it's late where I am and I need sleep will report back later on this to confirm.

JamiDEV commented 2 years ago

You will have to try an older or newer version if youre still getting the errors. I have 3 versions of scripts I run. Also what IDE or command line are you using? I use pycharm to run the scripts.

How does running through pycharm make a difference? I just run the scripts through normal multiple cmd windows if I need to run multiple rips. Does pycharm offer any functionality that would be more useful/efficient?

Editing and maintaining the script is much easier to do as is debugging since pycharm is for specialized in python dev. I'm on mac and ran into no stop issues using the command line. Pycharm lets you do things like switch python interpreter e.g., 3.9, 3.10 with ease. I can run multiple rips and trouble shoot all in one IDE.

xxx-uploaded commented 2 years ago

Loads not only active subscriptions How to disable EXPIRED download? I don't need the profiles I unsubscribed from

MonkeyKingViper commented 2 years ago

In the file apis\onlyfans\classes\create_user.py change the line 270 from: results = [create_highlight(x) for x in results] to results = [create_highlight(x) for x in results.get("data")] Don't need to change your auto_media_choice or auto_api_choice

for those using latest commit apis\onlyfans\classes\user_model.py line 283 is where you need to look for this instead

Confirming also that this fixed the errors on OnlyFans-85aec02

naodrej commented 2 years ago

Loads not only active subscriptions How to disable EXPIRED download? I don't need the profiles I unsubscribed from

Maybe create a list on OF and add those profiles to that list, then blacklist it in the config file.

aroundtheblockchainer commented 2 years ago

Loads not only active subscriptions

How to disable EXPIRED download?

I don't need the profiles I unsubscribed from

Wait so you're saying it's allowing you to scrape from those unsubbed profiles??

bigfranko commented 2 years ago

Okay so fresh install run updater, change line 283 in user_model and it worked. Thanks everyone for the help.

DeepSeaOne commented 2 years ago

Loads not only active subscriptions

How to disable EXPIRED download?

I don't need the profiles I unsubscribed from

Wait so you're saying it's allowing you to scrape from those unsubbed profiles??

If you have purchased content from a profile, it will still download the content even if you unsubscribe from the profile.

aroundtheblockchainer commented 2 years ago

Wait so you're saying it's allowing you to scrape from those unsubbed profiles??

If you have purchased content from a profile, it will still download the content even if you unsubscribe from the profile.

Hmm that's interesting. All my expired subs are gone from my list whether they expired on their own or I manually unsubbed early before it expired.

xxx-uploaded commented 2 years ago

Hmm that's interesting. All my expired subs are gone from my list whether they expired on their own or I manually unsubbed early before it expired.

ACTIVE (142 FANS ONLINE) [All 2861] [Active 1] [Expired 2860] [Attention required 0]

fightcancer2 commented 2 years ago

In the file apis\onlyfans\classes\create_user.py change the line 270 from: results = [create_highlight(x) for x in results] to results = [create_highlight(x) for x in results.get("data")]

Don't need to change your auto_media_choice or auto_api_choice

Thank you so very much!!!

aroundtheblockchainer commented 2 years ago

Hmm that's interesting. All my expired subs are gone from my list whether they expired on their own or I manually unsubbed early before it expired.

ACTIVE (142 FANS ONLINE)

[All 2861]

[Active 1]

[Expired 2860]

[Attention required 0]

I'm talking about the cmd tool not the website. Of course they will still be there in the site.

Smackdab69 commented 2 years ago

I am running on the latest build and made the suggested change and it still doesn't work :( @DIGITALCRIMINALS

In the file apis\onlyfans\classes\create_user.py change the line 270 from: results = [create_highlight(x) for x in results] to results = [create_highlight(x) for x in results.get("data")]

Don't need to change your auto_media_choice or auto_api_choice

fightcancer2 commented 2 years ago

I am running on the latest build and made the suggested change and it still doesn't work :( @DIGITALCRIMINALS

In the file apis\onlyfans\classes\create_user.py change the line 270 from: results = [create_highlight(x) for x in results] to results = [create_highlight(x) for x in results.get("data")] Don't need to change your auto_media_choice or auto_api_choice

Did you save the file? Did you verify later that it was saved? Are you getting the exact same error as above?

cuenta111 commented 2 years ago

You need to have the module installed: pip install yarl

i did that and it shows me some other module not found, i do pip install for that and still everytime it keeps showing some or other module missing