micahflee / semiphemeral

Automatically delete your old tweets, except for the ones you want to keep
MIT License
885 stars 85 forks source link

Feature Request: delete older tweets using the Twitter Archive #109

Closed dippedrusk closed 1 year ago

dippedrusk commented 2 years ago

I was one of the users who got hit by the weird Twitter API thing where only ~3000 tweets got downloaded and deleted.

I have my Twitter archive with all my tweet IDs and since you do a similar thing on the hosted version with DMs I thought maybe it would be fairly easy to set up a similar thing to delete all old tweets from the data/tweet.js file of an archive.

Update: meanwhile (since i didn't want to give twitter my phone number to change my API access level and then play with this repository) I wrote a horrible ugly selenium script to go through a list of tweet IDs from my archive and delete things for me in a browser, slowly, as i watch :P

JackSzwergold commented 2 years ago

Yes! This can be done right now if you uninstall the default PIP package version that is out there right now and instead install the latest version that is right here in this GitHub repo. This GitHub repo has an import function that doesn’t exist in the standard pip3 install version.

I followed this process and in my case I had 11,000+ Tweets a few days ago and now I trimmed it down to 1,800-ish Tweets. Works like a charm!

First, uninstall the version of semiphemeral you initially installed via PIP like this:

pip3 uninstall semiphemeral

Then reinstall it via this GitHub repo like this:

pip3 install git+ssh://git@github.com/micahflee/semiphemeral.git

This will install the current master branch in this repo. To check and see if it was properly installed, just run the semiphemeral without anything else and the help text will read as follows; note the new import option:

The output should look like this:


  Automatically delete your old tweets, except for the ones you want to keep

Options:
  --help  Show this message and exit.

Commands:
  configure        Start the web server to configure semiphemeral
  delete           Delete tweets that aren't automatically or manually
                   excluded, likes, and DMs
  delete_dms       Delete DMs that aren't available through the Twitter API
  excluded_export  Export tweets excluded that are excluded from deletion
  excluded_import  Import tweets excluded that are excluded from deletion
  fetch            Download all tweets/DMs
  import           Import tweets from a Twitter data export
  stats            Show stats about tweets in the database
  unlike           Delete old likes that aren't available through the Twitter
                   API

Now if that all worked well and import is in there, you are ready to do the next step… Actually import your Tweets from the Twitter archive.

First, unZip the archive and now run the command as follows:

 semiphemeral import /path/to/twitter/archive

Note that this might fail because semiphemeral is looking for the file data/tweet.js when it seems like the current Twitter archive format has that set as data/tweets.js.

The quick and dirty solution is to copy or rename data/tweets.js to data/tweet.js and run that command again.

If all goes well, the semiphemeral import process will show data just like the semiphemeral fetch process. Just note that while the import grinds through data, it will still be live-checking the Twitter API to fetch data for each Tweet. So it might take a while.

But once it is done, all of the Tweets will be stored locally on your system and ready for you to trim and cull them as you see fit.

dippedrusk commented 2 years ago

I noticed this! And I did get set up to try it out with the latest version of the code but then things didn't work because I have the lower level of API access (the one you got if you created your developer account after November 2021). And to get Elevated access I'd have had to give Twitter my phone number and fill out a form telling them what I wanted to do and wait for their approval and... I didn't want to do any of that. So (the hosted version of) semiphemeral took care of about 6500 tweets across two accounts and for the remaining 16000 I just hacked together some Selenium code that will pretend to be me manually deleting and un-RTing individual tweets in detached Chrome windows over the next 24 hours while I do other fun things.

I agree with you though that the import functionality should be in a new release and hopefully integrated into the hosted version as well because obviously semiphemeral is a much much better tool than my kludgey use of selenium because I don't want to play twitter's API access game.

JackSzwergold commented 2 years ago

Ohhh! So what you are pointing as an “issue” is about the hosted version of Semiphemeral and not the command line tool that is also available here. Understood. And respect your choices!

Alphakilo commented 1 year ago

I followed this process and in my case I had 11,000+ Tweets a few days ago and now I trimmed it down to 1,800-ish Tweets. Works like a charm!

@JackSzwergold Could you explain how your "Twitter App" is configured? I followed your process, but I end up with this during the import:

tweepy.error.TweepError: [{'message': 'You currently have Essential access which includes access to Twitter API v2 endpoints only. If you need access to this endpoint, you’ll need to apply for Elevated access via the Developer Portal. You can learn more here: https://developer.twitter.com/en/docs/twitter-api/getting-started/about-twitter-api#v2-access-leve', 'code': 453}]
JackSzwergold commented 1 year ago

@Alphakilo The error you have states word-for-word exactly what the issue is. The error is not an issue with Semiphemeral, but rather your Twitter API access level. Please read the error again; Essential access is the most basic level of access and you you need to apply for Elevated access which is a fairly simple process. Just login to your Twitter developer account and request Elevated access. You might need to regenerate your application keys, but that’s it.

Here is that error in a more human readable form.

“You currently have Essential access which includes access to Twitter API v2 endpoints only. If you need access to this endpoint, you’ll need to apply for Elevated access via the Developer Portal. You can learn more here:”

https://developer.twitter.com/en/docs/twitter-api/getting-started/about-twitter-api#v2-access-leve

But as @dippedrusk states:

“And to get Elevated access I'd have had to give Twitter my phone number and fill out a form telling them what I wanted to do and wait for their approval and... I didn't want to do any of that.”

So if you don’t want to share your phone number with Twitter when you apply for Elevated access, you will only ever have Essential access. Your choice.