DIGITALCRIMINAL / ArchivedUltimaScraper

Scrape content from OnlyFans and Fansly
GNU General Public License v3.0
948 stars 39 forks source link

Unable to get past initial site selection step #581

Closed lordoffools closed 2 years ago

lordoffools commented 2 years ago

This just started happening this afternoon in v7.6.1, so I'm assuming either it's a strange issue on my end (no libs or anything have changed on my end), or it's got something to do with the dynamically loaded code?

python start_ofd.py 
Select Site: 0 = onlyfans | 1 = fansly | 2 = starsavn
0
Traceback (most recent call last):
  File "/OnlyFans-7.6.1/start_ofd.py", line 108, in <module>
    asyncio.run(main())
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/base_events.py", line 641, in run_until_complete
    return future.result()
  File "/OnlyFans-7.6.1/start_ofd.py", line 93, in main
    api = await main_datascraper.start_datascraper(
  File "/OnlyFans-7.6.1/datascraper/main_datascraper.py", line 49, in start_datascraper
    api = main_helper.process_profiles(json_settings, proxies, site_name, api)
  File "/OnlyFans-7.6.1/helpers/main_helper.py", line 992, in process_profiles
    auth = api.add_auth(json_auth)
  File "/OnlyFans-7.6.1/apis/onlyfans/onlyfans.py", line 42, in add_auth
    auth = create_auth(pool=self.pool, max_threads=self.max_threads)
  File "/OnlyFans-7.6.1/apis/onlyfans/classes/create_auth.py", line 52, in __init__
    self.session_manager = api_helper.session_manager(self, max_threads=max_threads)
  File "/OnlyFans-7.6.1/apis/api_helper.py", line 97, in __init__
    dynamic_rules = requests.get(dr_link).json()  # type: ignore
  File "/Library/Python/3.10/lib/python/site-packages/requests/models.py", line 910, in json
    return complexjson.loads(self.text, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

I can reproduce regardless of which site I select at the intro (either 0 for OF or 1 for F or 2 for S), so it can't be any changes on the site's part itself.

god43 commented 2 years ago

same thing

File "D:\OnlyFans-master\start_ofd.py", line 107, in asyncio.run(main()) File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1776.0_x64qbz5n2kfra8p0\lib\asyncio\runners.py", line 44, in run return loop.run_until_complete(main) File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1776.0_x64qbz5n2kfra8p0\lib\asyncio\base_events.py", line 646, in run_until_complete return future.result() File "D:\OnlyFans-master\start_ofd.py", line 89, in main api = await main_datascraper.start_datascraper( File "D:\OnlyFans-master\datascraper\main_datascraper.py", line 49, in start_datascraper api = main_helper.process_profiles(json_settings, proxies, site_name, api) File "D:\OnlyFans-master\helpers\main_helper.py", line 942, in process_profiles auth = api.add_auth(json_auth) File "D:\OnlyFans-master\apis\onlyfans\onlyfans.py", line 42, in add_auth auth = create_auth(pool=self.pool, max_threads=self.max_threads) File "D:\OnlyFans-master\apis\onlyfans\classes\create_auth.py", line 44, in init self.session_manager = api_helper.session_manager(self, max_threads=max_threads) File "D:\OnlyFans-master\apis\api_helper.py", line 95, in init dynamic_rules = requests.get(dr_link).json() # type: ignore File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\requests\models.py", line 910, in json return complexjson.loads(self.text, **kwargs) File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1776.0_x64qbz5n2kfra8p0\lib\json\init.py", line 346, in loads return _default_decoder.decode(s) File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1776.0_x64qbz5n2kfra8p0\lib\json\decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1776.0_x64__qbz5n2kfra8p0\lib\json\decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

meanjoep92 commented 2 years ago

It's nice to know that I'm not the only one suddenly experiencing these errors.

Having the EXACT same problem when the program was running fine earlier today.

To preface this, it couldn't possibly be my JSON files (I did not alter them in any way prior to this happening)

nunga commented 2 years ago

Same here. My window was open from last night, so I was able to scrape an account successfully this morning, then it came up with the error when it tried to get the subscription list.

_PS C:\OF_scraper> poetry run python start_ofd.py Traceback (most recent call last): File "C:\Users\user\AppData\Local\pypoetry\Cache\virtualenvs\ultimascraper-9En8Lxug-py3.10\lib\site-packages\requests\models.py", line 910, in json return complexjson.loads(self.text, **kwargs) File "C:\Users\user\AppData\Local\Programs\Python\Python310\lib\json__init__.py", line 346, in loads return _default_decoder.decode(s) File "C:\Users\user\AppData\Local\Programs\Python\Python310\lib\json\decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "C:\Users\user\AppData\Local\Programs\Python\Python310\lib\json\decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\OF_scraper\start_ofd.py", line 66, in asyncio.run(main()) File "C:\Users\user\AppData\Local\Programs\Python\Python310\lib\asyncio\runners.py", line 44, in run return loop.run_until_complete(main) File "C:\Users\user\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 641, in run_until_complete return future.result() File "C:\OF_scraper\start_ofd.py", line 52, in main api = await main_datascraper.start_datascraper(config, site_name) File "C:\OF_scraper\datascraper\main_datascraper.py", line 131, in start_datascraper await default(datascraper) File "C:\OF_scraper\datascraper\main_datascraper.py", line 55, in default await main_helper.process_profiles(api, global_settings) File "C:\OF_scraper\helpers\main_helper.py", line 967, in process_profiles auth = api.add_auth(json_auth) File "C:\OF_scraper\apis\onlyfans\onlyfans.py", line 31, in add_auth auth = create_auth(self, pool=self.pool, max_threads=self.max_threads) File "C:\OF_scraper\apis\onlyfans\classes\auth_model.py", line 52, in init self.session_manager = self._session_manager(self, max_threads=max_threads) File "C:\OF_scraper\apis\onlyfans\classes\auth_model.py", line 68, in init api_helper.session_manager.init( File "C:\OF_scraper\apis\api_helper.py", line 129, in init dynamic_rules = requests.get(dr_link).json() # type: ignore File "C:\Users\user\AppData\Local\pypoetry\Cache\virtualenvs\ultimascraper-9En8Lxug-py3.10\lib\site-packages\requests\models.py", line 917, in json raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) requests.exceptions.JSONDecodeError: [Errno Expecting value] : 0 PS C:\OFscraper>

tyrion-the-imp commented 2 years ago

I'm getting this too. It appears that api_helper.py looks for the following file: https://raw.githubusercontent.com/DATAHOARDERS/dynamic-rules/main/onlyfans.json

This file doesn't appear to exist any more?

meanjoep92 commented 2 years ago

I'm getting this too. It appears that api_helper.py looks for the following file: https://raw.githubusercontent.com/DATAHOARDERS/dynamic-rules/main/onlyfans.json

This file doesn't appear to exist any more?

I also suspected that this was possibly the issue, but not sure if any of the devs could possibly confirm this.

tyrion-the-imp commented 2 years ago

Looks like someone was working on it as recently as 2 hours ago. The issue appears to have been reported. Probably just need to wait until they're done with it and get a new version put up there.

coltonshipley commented 2 years ago

It looks like something probably broke with the automation that generates that dynamic value json value. Hopefully it gets fixed. Kinda appears something changed and the process automatically nuked the file.

firemanchampion commented 2 years ago

If you look at the history for today: https://github.com/DATAHOARDERS/dynamic-rules/commits/main They pushed out two bad updates. Both don't work (one doesn't allow login/busted, other is just blank) Use the one from Aug 8th: https://raw.githubusercontent.com/DATAHOARDERS/dynamic-rules/84ca30c8e1df3f35c3ad58e340685f876c655542/onlyfans.json

Use that URL for dynamic_rules_link in .settings/config.json

lordoffools commented 2 years ago

Hmm, strangely, that doesn't work for me. Is there a need to toggle something else for that older one to work?

naodrej commented 2 years ago

yeah that's not working for me either.

firemanchampion commented 2 years ago

Can't help you there, worked for me just now. You can try the Aug 9th one I guess.

lordoffools commented 2 years ago

I've tried multiple past ones. Still doesn't work. So either there's a setting that I'm not toggling correctly, or that's not sufficient to fix it (for v7.6.1 and/or just for me).

RonnieBlaze commented 2 years ago

If you look at the history for today: https://github.com/DATAHOARDERS/dynamic-rules/commits/main They pushed out two bad updates. Both don't work (one doesn't allow login/busted, other is just blank) Use the one from Aug 8th: https://raw.githubusercontent.com/DATAHOARDERS/dynamic-rules/84ca30c8e1df3f35c3ad58e340685f876c655542/onlyfans.json

Use that URL for dynamic_rules_link in .settings/config.json

this worked for me, Thanks!

naodrej commented 2 years ago

For those that it worked for, what version are you on? Are you on 7.6.1?

firemanchampion commented 2 years ago

I'm using the master branch if that helps. What isn't working for you exactly? Provide some feedback and people can help direct you what the issue may be.

naodrej commented 2 years ago

I'm on 7.6.1, I just tried replacing the links. The result is the same as the error above

lordoffools commented 2 years ago

That's the difference, indeed. Master / latest vs. v7.6.1 (the last "official" release).

GooseWhoIsLoose commented 2 years ago

I'm using the master branch if that helps. What isn't working for you exactly? Provide some feedback and people can help direct you what the issue may be.

How do you use the master branch? I'm on the latest official release (7.6.1)

Having the same issue as the original poster, btw.

MonkeyKingViper commented 2 years ago

If you look at the history for today: https://github.com/DATAHOARDERS/dynamic-rules/commits/main They pushed out two bad updates. Both don't work (one doesn't allow login/busted, other is just blank) Use the one from Aug 8th: https://raw.githubusercontent.com/DATAHOARDERS/dynamic-rules/84ca30c8e1df3f35c3ad58e340685f876c655542/onlyfans.json

Use that URL for dynamic_rules_link in .settings/config.json

Confirming this worked for me. Many thanks! Also, I'm on 85aec02 if anyone needs a version this is fixed with

nunga commented 2 years ago

I'm not sure which version I'm using but updating .settings/config.json as suggested by @firemanchampion worked for me. Thanks!

_"dynamic_ruleslink": "https://raw.githubusercontent.com/DATAHOARDERS/dynamic-rules/84ca30c8e1df3f35c3ad58e340685f876c655542/onlyfans.json",

GooseWhoIsLoose commented 2 years ago

I'm not sure which version I'm using but updating .settings/config.json as suggested by @firemanchampion worked for me. Thanks!

I'm not having any luck... so to clarify, you changed

"dynamic_rules_link": "https://raw.githubusercontent.com/DATAHOARDERS/dynamic-rules/main/onlyfans.json",

on line 45 to

"dynamic_rules_link": "https://raw.githubusercontent.com/DATAHOARDERS/dynamic-rules/84ca30c8e1df3f35c3ad58e340685f876c655542/onlyfans.json",

Is that right? Just save the file and run?

hollowsmart commented 2 years ago

Just confirming myself on latest release 7.6.1 - changing the dynamic_rules_link works.

GooseWhoIsLoose commented 2 years ago

Just confirming myself on latest release 7.6.1 - changing the dynamic_rules_link works.

Hollowsmart, can you elaborate on what exactly you did? Just swapping the URL out isn't working for me

lucky1luc commented 2 years ago

I downloaded and replaced the config.json (first I saved my version - 7.6.1 - not modiffied since 2-may-2022 to config.json.saved) That did not work (gave me some versioning error), so I removed the new config.json and restored the old one... and now it works????

maxcom99 commented 2 years ago

Looks like the original URL works now

GooseWhoIsLoose commented 2 years ago

Yeah... seems to be working for me now too, with the original URL. Weird.

lordoffools commented 2 years ago

Interestingly, I didn't even switch back to the original URL, and it just works now.

So it leads me to wonder if even changing dynamic_rules_link mattered in v7.6.1, or if there was something else also affecting this (in addition to dynamic_rules_link) which has now been resolved?

Leaving this bug open for one of the devs to review and reply accordingly (especially if there's no further action needed), and then close it.

Thank you all.

OFfriend commented 2 years ago

There was bug in dynamic_rules which was fixed. You can close ths issue.