Open AyotoCorp opened 1 year ago
So, uh, weird update but it works again. I have no idea what caused that behavior though, and finding out the cause and solution would probably be really handy if/when it happens again.
Hi! Thank you for the kind words, I really appreciate you found the bot useful!
I can see reading your config that you're using Guest Tokens.
Bad news, it looks like Twitter has been changing and restricting the search endpoint we use to scrape tweets as a guest since yesterday:
https://twitter.com/i/api/2/search/adaptive.json
The change may be related to the removal of the "Latest" tab on the search for non-logged in users:
They also seem to have blacklisted the TLS fingerprint that the Python package requests
uses and returns HTTP 404
if you try to use the endpoint in any way.
I'll investigate if we can get around this by setting custom ciphers or some other way.
I have an in-progress draft version with custom ciphers you can try to install to confirm if that's the issue you're facing:
pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple pleroma-bot==1.2.1rc3
But even after doing that, the pagination also seems to be broken (so it only retrieves 47 tweets at most) and I need to probably fix the pinned tweet endpoint too.
Unfortunately, the only fully-working alternative at the moment is using Twitter Tokens or an RSS feed with the bot, which are unaffected and are still working as expected after Twitter's changes to the search.
I'll keep trying to see if we can circumvent the restrictions somehow.
I'll keep an eye on it, the next time it goes down I'll see if that change will fix things. "Unfortunately" right now it seems to be functioning as intended, and I'm not sure why. The only thing I'm noticing is that the tweets that were made during the time it wasn't working aren't being picked up by the bot when I run it, even when using --forcedate 2023-01-01
to make sure it's not just grabbing things from afterwards.
Alright, the issue is back. Running that install didn't fix it, so I'll just have to wait and see if it'll go away on its own or if this is the end of the auto-tweet pulls
I've been using the bot to mirror my Twitter (thanks for the work put into it btw, greatly appreciated) and for some reason between roughly 30 to 12 hours ago the bot started requiring OAuth keys, despite not pulling from a protected account.
And the config file
Apologies if this is something stupid I'm missing, but I have no idea what would have changed between when I got it running several days ago and now