search

simoniz0r / spm

spm is a simple commandline package manager written in bash
http://www.simonizor.net
GNU General Public License v2.0
40 stars 10 forks source link

Integrate with AppImageHub #1

Closed probonopd closed 7 years ago

probonopd commented 7 years ago

Please check out https://github.com/AppImage/AppImageHub, i's a central database in-the-making for AppImages.

Let's discuss which output format would be optimal for you.

If you have questions, AppImage developers are on #AppImage on irc.freenode.net.

simoniz0r commented 7 years ago

I really like how you've done VLC's appinfo. Having something like that available to grab info from to display to users would be great. It would also make my issue with checking versions below pretty much go away.

I also really like the way that you have AppImages setup on Bintray. I use this page to generate a list of AppImages available from Bintray, then use the steps that you've shown on the AppImageKit github page to find the latest version and download it.

The only small complaint that I really have is how some people on Github are sort of lazy about naming their AppImages. It would be preferable if the AppImage were named AppName-version-x86_64.AppImage, but a lot of people seem to leave out the version and/or the architecture that the AppImage is for. The version is not such a big deal; I'm planning on recording the version number from the github releases page to keep track of versions that way. Not showing which architecture the AppImage is for is slightly more annoying. It's probably safe to assume that any that don't specify are probably 64-bit, but it would be nice to have them labeled that way.

If there could be some sort of standard for how AppImages should be named, that would make it very easy to just grab the version number straight from the file name, and it would probably be less work on you for maintaining the database. I am aware that the version can be gotten by running --appimage-version, but some AppImages still don't support this, and that wouldn't really work out for checking the version before doing an upgrade. I'd have to download the AppImage, check the version, then delete it if it's the same version... not very efficient lol

probonopd commented 7 years ago

I really like how you've done VLC's appinfo.

This is an AppStream metainfo file. Currently we use these if they are present, but do not enforce their use since they can be a bit tricky to make properly. Maybe we need a tool to make this super easy to generate.

It would be preferable if the AppImage were named AppName-version-x86_64.AppImage

That is what I use for my demonstration AppImages on Bintray and what we generally recommend, but it is not a must. Depending on filenames for version and architecture is not the proper way to do this anyway. Possibly appimagetool should write this metadata into the AppImage itself. It's somewhere on my TODO list.

Please do not rely on the format of the data in database/. It could change anytime. Instead we should think about a proper output format that we could generate, and that tools like yours could use.

simoniz0r commented 7 years ago

Ideally, if there were a file for each AppImage that contained:

I could grab all of the necessary info for downloading and maintaining AppImages from those files. Having something like that available would eliminate a lot of hassle that I'm currently doing to get some of this info.

If you could maintain a file that lists all of the available AppImages, that would be great also. The method that I'm using to gather a list from Bintray works well, but it can take ~10-15 seconds sometimes. It would be quite a bit faster and more efficient to download an existing list.

probonopd commented 7 years ago

I think everything can be done but

simoniz0r commented 7 years ago

Sounds good! I can grab the versions and links for AppImages relatively easy from Bintray and Github, so that's no big deal. I'll have to look into how to use Github's API properly so I can have a more future proof method of getting the versions and links from there rather than getting it from the raw HTML. Using the method that you've shown on AppImageKit's github page to get the latest version's AppImage link from Bintray works very well, so no issue at all there.

The specific formatting of the files is not really important to me as long as they are pretty easy to read, and they stay consistent. Grabbing info from files using coreutils is pretty easy to do as long as the files' formatting does not change so I can use the same method every time.

I think that having the rest of the info that I listed available will help out a lot in the long run. Using that info, I'll be able to provide users with enough information about the AppImages to allow them to figure out what the application does, who made it, where to go with bugs, etc.

I really appreciate the work that you've put into AppImages, and it also means a lot to me that you took the time to ask what would work well for the tool I'm creating to distribute AppImages!

probonopd commented 7 years ago

So the goal should be to get rid of https://github.com/simoniz0r/spm/blob/master/AppImages-direct.lst because such a static list will, by definition, be outdated the very second it is published. Especially with continuous builds which more and more projects are using AppImage for.

Instead, let's define a format for AppImageHub that could be used by spm. But it will always be such that spm (the client) will have to check with the original location of the AppImage which version is available. This can be done, e.g., using the GitHub Releases API for projects hosted on GitHub.

simoniz0r commented 7 years ago

spm uses github's API to check the releases page to find the latest AppImages. The methods that spm uses to check for AppImages and new AppImages is working very well. It would be ideal if most (if not all) AppImages could be hosted on github, but I honestly doubt that every last one will move to Github.

Currently, I feel like it's better to list these AppImages and just warn people that the links may not be reliable than to not list them at all. As long as other sites use links that automatically redirect to download the latest AppImage, I don't really have a problem with that. For example, with AppImages that are hosted at download.opensuse.org, I would be able to run a wget -S --spider on the download url, get the Location: line, and use that to check the name of the AppImage to see if it matches the user's current AppImage.

It's when the download for an AppImage is directly linked to a specific version that is most annoying. At least with links that redirect to the latest release, the user can force an upgrade if they find they are out of date and spm doesn't pick up the new version. Download links that link to a specific version rely on the list being updated for the new version which is not ideal at all.

Also, a suggestion for AppImageHub's listing of AppImages... perhaps you could put them in a json file similar to what I have done for tar archives (with less info, of course)? I'm not sure if this would work out well for you, but I find json to be really nice to work with.

probonopd commented 7 years ago

Also, a suggestion for AppImageHub's listing of AppImages... perhaps you could put them in a json file similar to what I have done for tar archives (with less info, of course)? I'm not sure if this would work out well for you, but I find json to be really nice to work with.

Yes, that's on the TODO list.

For example, with AppImages that are hosted at download.opensuse.org, I would be able to run a wget -S --spider on the download url

Does this happen on the client or on the server?

simoniz0r commented 7 years ago

It would happen on the client so that I do not have to maintain a list of current images for all applications. It would only once per AppImage that the user has installed when the user runs the update, upgrade, and list search arguments. It only grabs the head info, so it's easier on the servers than going to their website and refreshing it a couple of times.

I already use this method to get update info for precompiled tar archives that have links that reroute to the latest version, and it works well for that.

probonopd commented 7 years ago

It would happen on the client so that I do not have to maintain a list of current images for all applications.

Agree, that's a scaleable way to do this.

simoniz0r commented 7 years ago

Giving up on this for a while. spm will still use data from AppImageHub, but will not be really "integrating" with AppImageHub until it is more fleshed out.

simoniz0r commented 6 years ago

@probonopd You might be interested in another script I've started working on called appimagedl. I figured out a reliable way to parse https://appimage.github.io/feed.json using jq and csplit to split the feed into individual files containing the info for each AppImage.

The list of AppImages can be updated automatically by using appimagedl's update argument which will re-parse feed.json and notify the user of any changes. The aim is to provide a list of AppImages to be downloaded/managed through appimagedl that have been approved by AppImageHub.

I plan on using appimageupdatetool to update AppImages when possible. When not possible, jq is used to get the latest version from Github's API. The updates through Github's API is working right now, but I haven't finished the functions for appimageupdatetool updates yet.

I also have a couple more things to finish such as a help function, a revert function, a get function to allow users to download multiple AppImages and/or old versions of an AppImage, and a man page.

After taking a closer look at how snaps and flatpaks work, I changed my mind on how AppImageHub should work, and I agree with what you've got going on there. I really like quite a few things about how the package manager for snaps works (minus things like the required daemon and automatic updates without user consent), and I figured that AppImage could use something like the snap package manager to help AppImages be distributed more easily.

If you have any questions or concerns, feel free to post an issue over at appimagedl's repo.

(Side note: I am working on learning Ruby and do plan on rewriting this in Ruby when I feel comfortable with it to avoid depending on jq and wget.)