search

kensanata / mastodon-archive

Archive your statuses, favorites and media using the Mastodon API (i.e. login required)
https://alexschroeder.ch/software/Mastodon_Archive
GNU General Public License v3.0
358 stars 33 forks source link

Install "the Debian way" #78

Closed IzzySoft closed 1 year ago

IzzySoft commented 1 year ago

As discussed at Mastodon, here a short "guide" how I installed mastodon-backup without running setup.py:

First install dependencies: apt install python3-progress python3-mastodon. Then, inside the cloned repo's root, create a wrapper (e.g. mastodon.py) looking like this:

#!/usr/bin/env python3
import mastodon_archive

if __name__ == "__main__":
  mastodon_archive.main()

Make it executable, create an alias for it (alias mastodon-archive="$HOME/bin/mastodon-backup/mastodon-archive.py") or add a wrapper script to our $PATH containing just $HOME/bin/mastodon-backup/mastodon-archive.py $* (adjust the path to where you put the cloned repo to), and we are ready to go.

If now running mastodon-archive archive --with-mentions <account> fails with AttributeError: 'Mastodon' object has no attribute 'bookmarks', your distribution shipped an older version of python3-mastodon (Ubuntu 20.04 and derivates: 1.5.0). In this case, get the 1.5.1 from PyPi and replace the files in /usr/lib/python3/dist-packages/mastodon.

Done.

IzzySoft commented 1 year ago

OK, related PR now ready for review. Looking forward to your opinion :smiley:

kensanata commented 1 year ago

I don't really have an opinion since I install it using pip, but I wish more people would do it the Debian way. Or package my stuff for Debian directly without me having to get involved. :laughing:

IzzySoft commented 1 year ago

Or package my stuff for Debian directly without me having to get involved.

Hm, wait a minute. There already is a Makefile, but it's using the setup.py. I could call make -f Makefile.debian to use my own. Declare those 3 packages as dependencies. But then we end up with an install that, while being clean, would fail with bookmarks when some distris ship the old python3-mastodon v1.5.0.

With a Debian package, I wouldn't want to include what I did with the installer script, manually replacing those files. So what I could do is using my good old relman and its framework to create a *.deb and place it in my own repo. For that, if you agree, I'd

For those ending up with the old python3-mastodon we then must leave a hint somewhere that they need to take care manually. Maybe you could include a check and spit out a (warning) note like "Your python3-mastodon module is outdated, so some functionality will be unavailable. We will need at least version 1.5.1 for all features to work. See xxx for details." – and have "xxx" point to instructions how to update it manually (basically, what my debinstall.sh does; I could make that part a separate script so those users then would simply need to download and run it). Or even better: I include the distrib/ in /usr/share/MastodonArchive, so the pointer could include to simply run that (separated) script to fix things up.

How does that sound to you? All I'd need then is a ping from you whenever a new release has been made, and what tag I should base the package on. Provided I get a usable package made, that is :see_no_evil: and you'd need to tag consequently (e.g. the tag for 1.4.1 is missing at 3ca50f9b9ddae57f38e8a31912bba05ac1ce317a :wink:)

PS: That's basically what I do for some other software projects, too – e.g. for Monitorix.

PPS: If you have some RPM-based machine to test, I could also have relman spit out an *.rpm. No idea what dependency versions we'd end up there, and we'd need to find out what they are named there as well.

kensanata commented 1 year ago

Part of the problem is that I haven't been pushing my tags, that's true – not even to my primary repo at https://alexschroeder.ch/cgit/mastodon-archive. I'll have to be more conscientious about that. I pushed the two tags I have in my local repo, in any case.

I'd add Debian specific stuff to the main Makefile unless you think it really needs to live100% in a separate directory.

I guess you could simply declare a dependency to a particular version of python3-mastodon and abort if that isn't available. I don't think packaging for Debian needs to be super flexible. Alterantively, if you think this is going to be a problem, let's add the checks and messages to mastodon-archive itself. That would be the cleaner solution, anyway.

As for the solution, I think I'd be happy to leave it up to you. The README of relman sounds exciting, but I'm not a packaging person, so I don't know anything about it.

As for pings, my suggestion is to just add whatever you want me to do to RELEASE.md. That's the file I'm going to look at when making a release.

IzzySoft commented 1 year ago

I'd add Debian specific stuff to the main Makefile unless you think it really needs to live100% in a separate directory.

I think I can do that via the relman configuration (more precisely: the RPM *.spec file) so we would not need another Makefile. It's as simple as copying some files over, so the install command should do. I'll play with it a bit when time permits.

I guess you could simply declare a dependency to a particular version

Yes, that can be done. But declaring > 1.5.0 would not leave many systems where it could be installed (not sure if bullseye already added it; I'm using Mint 20.3 here which is based on Ubuntu 20.04, and that comes with 1.5.0. Most things work with that; it was only when I added --mentions that "parts broke" (bookmarks could not be retrieved). So maybe checks in mastodon-archive itself would be preferable.

And yes, I didn't want to suggest YOU get relman up and running. I have it set up here, much easier I get the SPEC file ready. I will see what I can achieve.

just add whatever you want me to do to RELEASE.md

"send a mention to Izzy" (either via Mastodon or an issue here or by mail or… take your pick :smile:) But first I need to get it ready. Shall I start with v1.4.2, or with HEAD? With HEAD I could also add mastoarch to /usr/bin (or shall I rather leave that to the users to decide?)

IzzySoft commented 1 year ago

Good news: I've successfully created a *.deb and a *.rpm. Just need testers now before I publish them.

Not so good news: the versions of python3-mastodon that ship with distributions.

So how do you want it to go? Shall I publish both as *-izzy2 built from HEAD (so contrib is inside already)? Or first update the contrib/ with its own README and split of the "update python3-mastodon to 1.5.1" script, so the "broken installs" can be easily fixed if needed? Ideally, before publishing, we'd have some volunteers confirming the packages lead to a working install.

kensanata commented 1 year ago

Please add whatever you think I need to read when it's release time to RELEASE.md as a MR, that's the best way to be sure. Otherwise I'm going to ask you for your email again, and round and round we go. I'd rather just accept your MR. :smile:

I would definitely use HEAD, including your stuff.

I think it would be best if we handled the error of old dependency, first. Let's be sure we understand what the problem is. Do you think d8a1f903 is going to solve it for us?

I'm a bit confused by the last paragraph. What is "both"? Is "update python3-mastodon to 1.5.1" a script or a warning to be displayed? (In the commit I mentioned, it's simply a warning.)

…
Get bookmarks (this may take a while)
Skipping bookmarks since your Mastodon.py library is too old!
Skipping mentions
Skipping followers
Skipping following
Saving 485 statuses, 441 favourites, 0 bookmarks, 399 mentions, 0 followers, and 0 following

Having volunteers test it sounds great to me. :smiley:

IzzySoft commented 1 year ago

Please add whatever you think I need to read when it's release time to RELEASE.md

OK, will do when we are at that point.

I would definitely use HEAD, including your stuff.

Agreed. That's also what the current (temporary) packages have.

I think it would be best if we handled the error of old dependency, first. Let's be sure we understand what the problem is. Do you think https://github.com/kensanata/mastodon-backup/commit/d8a1f9035158f44a9650a6843a438eb22cc87377 is going to solve it for us?

I'm not deep enough into the code, but I'd say yes. Just added a comment there to make the "error message" a bit clearer – including a hint on how to solve it when encountered.

What is "both"?

The *.deb and the *.rpm :wink:

Is "update python3-mastodon to 1.5.1" a script or a warning to be displayed?

Ugh, that's another "both" :rofl: First the updated warning message (so users get an idea of how to solve it when it happens), and second a script I'll add to contrib/ performing that update on systems only having older versions available in their repos. Basically the corresponding part of the current debinstall.sh. Once the packages are "live", debinstall.sh itself can be dropped then (or adjusted; let's leave that decision for later).

Having volunteers test it sounds great to me.

Definitely! So here's my proposal:

  1. I open a new issue for this: "getting mastodon-backup packaged"
  2. next to a description of the task, I attach the *.deb and *.rpm I've built last night
  3. I send out a toot (mentioning you, so you can boost) with what we're about to do and that we'd like some testers, linking to that issue. I'd not be surprised if our joined reach will attract several testers :smile:
  4. in parallel, adjustments to code, scripts and documentation are performed – taking feedback from testers into account as well. Whenever we reach a new "stage", I'll create new packages and attach them to the issue
  5. once we're satisfied and deem the results ready to go wild, I'll upload the package(s) to my repo and we make a new announcement. Saying "package(s)" meaning if one of them is ready but the other not, we start with the ready one (no idea why this could happen – but I mainly think about the RPM where I'm not entirely sure as I haven't used any RPM based system for, ugh, 15 years or so)

How does that sound to you?