Closed dippedrusk closed 1 year ago
Yes! This can be done right now if you uninstall the default PIP package version that is out there right now and instead install the latest version that is right here in this GitHub repo. This GitHub repo has an import
function that doesn’t exist in the standard pip3 install
version.
I followed this process and in my case I had 11,000+ Tweets a few days ago and now I trimmed it down to 1,800-ish Tweets. Works like a charm!
First, uninstall the version of semiphemeral
you initially installed via PIP like this:
pip3 uninstall semiphemeral
Then reinstall it via this GitHub repo like this:
pip3 install git+ssh://git@github.com/micahflee/semiphemeral.git
This will install the current master
branch in this repo. To check and see if it was properly installed, just run the semiphemeral
without anything else and the help text will read as follows; note the new import
option:
The output should look like this:
Automatically delete your old tweets, except for the ones you want to keep
Options:
--help Show this message and exit.
Commands:
configure Start the web server to configure semiphemeral
delete Delete tweets that aren't automatically or manually
excluded, likes, and DMs
delete_dms Delete DMs that aren't available through the Twitter API
excluded_export Export tweets excluded that are excluded from deletion
excluded_import Import tweets excluded that are excluded from deletion
fetch Download all tweets/DMs
import Import tweets from a Twitter data export
stats Show stats about tweets in the database
unlike Delete old likes that aren't available through the Twitter
API
Now if that all worked well and import
is in there, you are ready to do the next step… Actually import your Tweets from the Twitter archive.
First, unZip the archive and now run the command as follows:
semiphemeral import /path/to/twitter/archive
Note that this might fail because semiphemeral
is looking for the file data/tweet.js
when it seems like the current Twitter archive format has that set as data/tweets.js
.
The quick and dirty solution is to copy or rename data/tweets.js
to data/tweet.js
and run that command again.
If all goes well, the semiphemeral import
process will show data just like the semiphemeral fetch
process. Just note that while the import grinds through data, it will still be live-checking the Twitter API to fetch data for each Tweet. So it might take a while.
But once it is done, all of the Tweets will be stored locally on your system and ready for you to trim and cull them as you see fit.
I noticed this! And I did get set up to try it out with the latest version of the code but then things didn't work because I have the lower level of API access (the one you got if you created your developer account after November 2021). And to get Elevated access I'd have had to give Twitter my phone number and fill out a form telling them what I wanted to do and wait for their approval and... I didn't want to do any of that. So (the hosted version of) semiphemeral took care of about 6500 tweets across two accounts and for the remaining 16000 I just hacked together some Selenium code that will pretend to be me manually deleting and un-RTing individual tweets in detached Chrome windows over the next 24 hours while I do other fun things.
I agree with you though that the import functionality should be in a new release and hopefully integrated into the hosted version as well because obviously semiphemeral is a much much better tool than my kludgey use of selenium because I don't want to play twitter's API access game.
Ohhh! So what you are pointing as an “issue” is about the hosted version of Semiphemeral and not the command line tool that is also available here. Understood. And respect your choices!
I followed this process and in my case I had 11,000+ Tweets a few days ago and now I trimmed it down to 1,800-ish Tweets. Works like a charm!
@JackSzwergold Could you explain how your "Twitter App" is configured? I followed your process, but I end up with this during the import:
tweepy.error.TweepError: [{'message': 'You currently have Essential access which includes access to Twitter API v2 endpoints only. If you need access to this endpoint, you’ll need to apply for Elevated access via the Developer Portal. You can learn more here: https://developer.twitter.com/en/docs/twitter-api/getting-started/about-twitter-api#v2-access-leve', 'code': 453}]
@Alphakilo The error you have states word-for-word exactly what the issue is. The error is not an issue with Semiphemeral, but rather your Twitter API access level. Please read the error again; Essential access is the most basic level of access and you you need to apply for Elevated access which is a fairly simple process. Just login to your Twitter developer account and request Elevated access. You might need to regenerate your application keys, but that’s it.
Here is that error in a more human readable form.
“You currently have Essential access which includes access to Twitter API v2 endpoints only. If you need access to this endpoint, you’ll need to apply for Elevated access via the Developer Portal. You can learn more here:”
https://developer.twitter.com/en/docs/twitter-api/getting-started/about-twitter-api#v2-access-leve
But as @dippedrusk states:
“And to get Elevated access I'd have had to give Twitter my phone number and fill out a form telling them what I wanted to do and wait for their approval and... I didn't want to do any of that.”
So if you don’t want to share your phone number with Twitter when you apply for Elevated access, you will only ever have Essential access. Your choice.
I was one of the users who got hit by the weird Twitter API thing where only ~3000 tweets got downloaded and deleted.
I have my Twitter archive with all my tweet IDs and since you do a similar thing on the hosted version with DMs I thought maybe it would be fairly easy to set up a similar thing to delete all old tweets from the
data/tweet.js
file of an archive.Update: meanwhile (since i didn't want to give twitter my phone number to change my API access level and then play with this repository) I wrote a horrible ugly selenium script to go through a list of tweet IDs from my archive and delete things for me in a browser, slowly, as i watch :P