Mirroring several accounts

[Didou09]

Hi,

Thanks for your tutorial, it's very interesting, I've tried it to mirror an account for an association, with their explicit approval. Now another asociation would be interested if I could do it for them too.

Could you describe how to modify the script so that the bot can check several independant accounts (each corresponding to a different host / bot) every time it is run ? Would the best way be to add several account to the config.json ? or instead to add a config.json file for each account ?

Thanks for your help, I am not very familiar with json files.

Also I noticed your tutorial does not seem to be available online anymore, any idea why ? Could you put it here in html format maybe if it's not possible to put it back in its original place ? https://dev.to/ayushsharma/tweet-toot-building-a-bot-for-mastodon-using-python-5a6d

[ayush-sharma]

Hi @Didou09,

Sorry about the dev.to link. You can find the original article here: Tweet-Toot: Building a bot for Mastodon using Python

For running multiple versions of the app, you can check out the README of this repo. The most effective way might be to run separate containers of the app and passing different environment variables to Docker, like this:

docker run --rm -e TT_APP_SECURE_TOKEN="token1" -e TT_SOURCE_TWITTER_URL="account-1" -v /tmp:/tmp tweet-toot:latest

docker run --rm -e TT_APP_SECURE_TOKEN="token2" -e TT_SOURCE_TWITTER_URL="account-2" -v /tmp:/tmp tweet-toot:latest

This should keep things nice and isolated and allow you to run as many parallel relays as you want.

Let me know if this works.

[Didou09]

Thanks very much for your fast answer !

However, I'm planning to run this on a raspberry pi (low-cost, low-tech solution) and I can't install Docker on it, any way to specify a list of accounts to check directly in the code ?

I'd like to have this running with as low power consumption as possible, actually I'm even thinking about trying to port it to a smaller device, like an Pyboard for lower power consumption, using micropython for power saving (programming cycle of one check per hour or so), but I can't find a version of BeautifulSoup that would be usable in Micropython, any idea ? That could be a project for students (low-power, low cost minimal media communication, easy to unplug / replug), but I'd like to do it myself first.

[ayush-sharma]

I don't really have much experience with micropython. I've only tried BS4 for web scraping, but if you need something even more light-weight, you can try fetching pages using requests and parsing the HTML in code using regex. If the HTML is straight-forward the parsing might be easy, but you won't get the object-oriented parsing functionality of BS4.

For supporting multiple twitter URLs, the reason I never added it because the requirements around relationships were not really clear. For example, let's say I even define a list for the source twitter accounts and another list for the Mastodon accounts. Which Twitter is relayed to which Mastodon? Do we keep the relationship one-to-one, one-to-many, or many-to-many? I mean, I may want 5 Twitter accounts to post to the same Mastodon, or one account to post to 5 Mastodons, etc.

If your use-case is the first, that is relay multiple Twitter accounts to a single Mastodon host, I can work on adding it since it should be trivial. Might take me a day or two depending on my day job. I can take a semicolon-separated list of Twitter accounts and parse them all for relaying. This will ensure the syntax will work from the config.json file and also when specified in the env, which Tweet-Toot supports.

What say?

[Didou09]

Thank again,

I'll try to go for the option you proposed regarding micropython: home-made parsing using requests

For the multi-accounts aspect: My use-case is the first one you mention, I need to read several twitter accounts and post them to several mastodon accounts, one-to-one.

This is the least optimized but the most general use-case, do you think you could tweak the code so that it allows for this ?

Thanks for your help

[ayush-sharma]

Hey @Didou09,

Sorry for the delay, but this feature was a bit more complex. I have a develop build ready to go. I'm doing some testing on my end and working out some final kinks, so if you want to test out your use-case and how it would work, you can check out the develop branch.

https://github.com/ayush-sharma/tweet-toot/tree/develop

Let me know in case you think we need to make other changes. Keep in mind, this is a dev build only.

[ayush-sharma]

Hi @Didou09, did you get a chance to check this out?

Hey @Didou09,

Sorry for the delay, but this feature was a bit more complex. I have a develop build ready to go. I'm doing some testing on my end and working out some final kinks, so if you want to test out your use-case and how it would work, you can check out the develop branch.

https://github.com/ayush-sharma/tweet-toot/tree/develop

Let me know in case you think we need to make other changes. Keep in mind, this is a dev build only.

[Didou09]

Hi @ayush-sharma , not yet sorry, but I'll try to do it this afternoon (sorry, work took over)

[Didou09]

Hi @ayush-sharma , so I forked your repo and checked out to the develop branch, and tested it with 2 twitter accounts to 2 mastodon accounts.

I had to tweak it a bit because the str formatting f"{variable}" of Python 3.7 is not available on raspberry pi 4 which only has python 3.4. It's ok now.

I tested it with one account first (Euroscipy), it worked but it seems to have posted the twitter messages in a non-chronological order (the last toot is not the last tweet): https://twitter.com/EuroSciPy https://botsin.space/web/accounts/180022

What do you think?

Here is the terminal stdout:

pi@raspberrypi:~/Twitter2Mastodon/tweet-toot/tweet-toot (develop) $ python run.py 
2020-05-08 00:05:05,639 - __main__ - INFO - __main__ => Mode: one-to-one
2020-05-08 00:05:05,639 - tweettoot - INFO - relay() => Init relay from https://twitter.com/EuroSciPy to https://botsin.space. State file /tmp/tt_b67df4aa5c823c5c3d3c22aa28e4e1c73a86dbe5
2020-05-08 00:05:05,663 - requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): twitter.com
2020-05-08 00:05:07,488 - tweettoot - INFO - get_tweets() => Fetched 0 new tweets for https://twitter.com/EuroSciPy.
2020-05-08 00:05:07,916 - tweettoot - INFO - relay() => Tweeting 1230060371114954752 to https://botsin.space
2020-05-08 00:05:07,919 - requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): botsin.space
2020-05-08 00:05:09,146 - tweettoot - INFO - toot_the_tweet() => OK. Tooted 1230060371114954752 to https://botsin.space.
2020-05-08 00:05:09,214 - tweettoot - INFO - relay() => Tweeting 1234538966906527752 to https://botsin.space
2020-05-08 00:05:09,225 - requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): botsin.space
2020-05-08 00:05:10,815 - tweettoot - INFO - toot_the_tweet() => OK. Tooted 1234538966906527752 to https://botsin.space.
2020-05-08 00:05:10,820 - tweettoot - INFO - relay() => Tweeting 1251268988912443395 to https://botsin.space
2020-05-08 00:05:10,829 - requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): botsin.space
2020-05-08 00:05:11,939 - tweettoot - INFO - toot_the_tweet() => OK. Tooted 1251268988912443395 to https://botsin.space.
2020-05-08 00:05:11,946 - tweettoot - INFO - relay() => Tweeting 1199413419931291648 to https://botsin.space
2020-05-08 00:05:11,962 - requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): botsin.space
2020-05-08 00:05:12,641 - tweettoot - INFO - toot_the_tweet() => OK. Tooted 1199413419931291648 to https://botsin.space.
2020-05-08 00:05:12,643 - tweettoot - INFO - relay() => Tweeting 1235112746988875777 to https://botsin.space
2020-05-08 00:05:12,647 - requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): botsin.space
2020-05-08 00:05:13,531 - tweettoot - INFO - toot_the_tweet() => OK. Tooted 1235112746988875777 to https://botsin.space.
2020-05-08 00:05:13,534 - tweettoot - INFO - relay() => Tweeting 1201822155803115521 to https://botsin.space
2020-05-08 00:05:13,538 - requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): botsin.space
2020-05-08 00:05:14,159 - tweettoot - INFO - toot_the_tweet() => OK. Tooted 1201822155803115521 to https://botsin.space.
2020-05-08 00:05:14,169 - tweettoot - INFO - relay() => Tweeting 1234428825464909825 to https://botsin.space
2020-05-08 00:05:14,178 - requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): botsin.space
2020-05-08 00:05:14,934 - tweettoot - INFO - toot_the_tweet() => OK. Tooted 1234428825464909825 to https://botsin.space.
2020-05-08 00:05:14,937 - tweettoot - INFO - relay() => Tweeting 1182331731644432388 to https://botsin.space
2020-05-08 00:05:14,940 - requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): botsin.space
2020-05-08 00:05:16,032 - tweettoot - INFO - toot_the_tweet() => OK. Tooted 1182331731644432388 to https://botsin.space.
2020-05-08 00:05:16,037 - tweettoot - INFO - relay() => Tweeting 1201506161335439360 to https://botsin.space
2020-05-08 00:05:16,045 - requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): botsin.space
2020-05-08 00:05:16,842 - tweettoot - INFO - toot_the_tweet() => OK. Tooted 1201506161335439360 to https://botsin.space.
2020-05-08 00:05:16,847 - tweettoot - INFO - relay() => Tweeting 1199405251280936960 to https://botsin.space
2020-05-08 00:05:16,858 - requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): botsin.space
2020-05-08 00:05:17,548 - tweettoot - INFO - toot_the_tweet() => OK. Tooted 1199405251280936960 to https://botsin.space.
2020-05-08 00:05:17,552 - tweettoot - INFO - relay() => Tweeting 1231569185458790400 to https://botsin.space
2020-05-08 00:05:17,561 - requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): botsin.space
2020-05-08 00:05:18,141 - tweettoot - INFO - toot_the_tweet() => OK. Tooted 1231569185458790400 to https://botsin.space.
2020-05-08 00:05:18,145 - tweettoot - INFO - relay() => Tweeting 1235987701297487873 to https://botsin.space
2020-05-08 00:05:18,154 - requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): botsin.space
2020-05-08 00:05:18,907 - tweettoot - INFO - toot_the_tweet() => OK. Tooted 1235987701297487873 to https://botsin.space.
2020-05-08 00:05:18,912 - tweettoot - INFO - relay() => Tweeting 1199781485097734144 to https://botsin.space
2020-05-08 00:05:18,920 - requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): botsin.space
2020-05-08 00:05:19,741 - tweettoot - INFO - toot_the_tweet() => OK. Tooted 1199781485097734144 to https://botsin.space.
2020-05-08 00:05:19,744 - tweettoot - INFO - relay() => Tweeting 1207642280535973888 to https://botsin.space
2020-05-08 00:05:19,747 - requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): botsin.space
2020-05-08 00:05:20,560 - tweettoot - INFO - toot_the_tweet() => OK. Tooted 1207642280535973888 to https://botsin.space.
2020-05-08 00:05:20,563 - tweettoot - INFO - relay() => Tweeting 1240399282995421190 to https://botsin.space
2020-05-08 00:05:20,566 - requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): botsin.space
2020-05-08 00:05:21,289 - tweettoot - INFO - toot_the_tweet() => OK. Tooted 1240399282995421190 to https://botsin.space.
2020-05-08 00:05:21,292 - tweettoot - INFO - relay() => Tweeting 1230798932449644544 to https://botsin.space
2020-05-08 00:05:21,295 - requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): botsin.space
2020-05-08 00:05:21,912 - tweettoot - INFO - toot_the_tweet() => OK. Tooted 1230798932449644544 to https://botsin.space.
2020-05-08 00:05:21,917 - tweettoot - INFO - relay() => Tweeting 1235868034062880768 to https://botsin.space
2020-05-08 00:05:21,925 - requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): botsin.space
2020-05-08 00:05:22,601 - tweettoot - INFO - toot_the_tweet() => OK. Tooted 1235868034062880768 to https://botsin.space.
2020-05-08 00:05:22,604 - tweettoot - INFO - relay() => Tweeting 1253737600466276352 to https://botsin.space
2020-05-08 00:05:22,607 - requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): botsin.space
2020-05-08 00:05:23,209 - tweettoot - INFO - toot_the_tweet() => OK. Tooted 1253737600466276352 to https://botsin.space.
2020-05-08 00:05:23,214 - tweettoot - INFO - relay() => Tweeting 1207642487860453376 to https://botsin.space
2020-05-08 00:05:23,222 - requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): botsin.space
2020-05-08 00:05:24,212 - tweettoot - INFO - toot_the_tweet() => OK. Tooted 1207642487860453376 to https://botsin.space.
2020-05-08 00:05:24,217 - tweettoot - INFO - relay() => Tweeting 1222215009771642886 to https://botsin.space
2020-05-08 00:05:24,225 - requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): botsin.space
2020-05-08 00:05:24,997 - tweettoot - INFO - toot_the_tweet() => OK. Tooted 1222215009771642886 to https://botsin.space.

[ayush-sharma]

Thanks for the terminal output. I just had to reverse the order of tweets.

I've updated my develop branch with the changes. I'll test them with a few sample accounts and verify.

[ayush-sharma]

Hi @Didou09,

Just wanted to check-in in case you need my help with something. Are the changes working as expected?