Discussion

[antony-jr]

What will this library do ?

• Implement a simple to use interface to AppImageUpdater with Qt5 signals and slots
• Create a easier to use C++ API.
• Document the new High Level API , so everyone can use it
• Create an Async model.

The goal is to make the library Qt friendly ( I'm bad at naming! ).

Any Thoughts?

[probonopd]

👍 Maybe call this AppImageUpdateBridge to make clear it is about updating?

[probonopd]

(Moved from https://github.com/antony-jr/GHReleaseBridge/issues/1)

Hey @antony-jr please check out libappimageupdate. It implements the update information mechanisms described in the AppImageSpec.

If you integrate libappimageupdate we might be interested. I am sure that @TheAssassin is happy to answer any questions you might have.

[antony-jr]

There is a saying I like:

Perfection is Achieved Not When There Is Nothing More to Add, But When There Is Nothing Left to
Take Away
-- Antoine de Saint-Exupery

We don't need the Magic Codes

As the updater is inbuilt inside the AppImage I think having it check the Magic codes is just a waste of time ( please let me know if I am wrong? ) ( for integrity we can use SHA256 sums ).

Reasons:

• The configuration can be done on a more reliable source like a json file inside the AppImage , this avoids the nasty code and its much faster and userfriendly.
• The configuration can also be directly inputed in the code block. So this also proves integrity.
• We don't have to relay on system specifics , by avoiding the magic code search, I think we can make the code elegant

Any thoughts? @TheAssassin

And thanks for the zsync2 @TheAssassin , its really a life saver!

[probonopd]

There is a saying I like:

Perfection is Achieved Not When There Is Nothing More to Add, But When There Is Nothing Left to Take Away -- Antoine de Saint-Exupery

That could go on the AppImage homepage... 👍

True, if AppImageUpdaterBridge is invoked from within the AppImage itself, then of course it doesn't need to check the AppImage magic bytes.

[TheAssassin]

As the updater is inbuilt inside the AppImage I think having it check the Magic codes is just a waste of time ( please let me know if I am wrong? ) ( for integrity we can use SHA256 sums )

libappimageupdate does check for the magic bytes to determine the type which defines where it has to look for the update information. I don't think that this really tiny check is a "waste of time" or anything else, and it is actually necessary because it is the only way to gather this information (not only reliably, but at all, anything else would mean to perform checks for all types for every file, but also won't scale well for future AppImage types).

Using JSON files (or anything similar to that) from within the AppImage doesn't make sense to me as we have a specification defining an update mechanism, that is supported by this library. Extracting files from an AppImage is a certainly more expensive operation than reading 2-3 bytes from a file's beginning. And as libappimageupdate is primarily built to update external AppImages (e.g., as a self contained tool (appimageupdatetool, AppImageUpdate), or some app store like applications), it makes sense to have this as default.

for integrity we can use SHA256 sums

Magic numbers/bytes != integrity check. They are used to identify the AppImage type, but that's not related to any specific patching or updating code, only for determining meta information required to start the update processes. I don't know what you're trying to express with this sentence. Feel free to elaborate to clarify this point.

The configuration can be done on a more reliable source like a json file inside the AppImage , this avoids the nasty code and its much faster and userfriendly.

If you want to change this update mechanism to be defined by a file inside the AppImage, that'd only work for self-updating AppImages, but your code might even be useful to update external AppImages. I don't think adding such a limitation is necessary.

Also, re user friendliness, when creating AppImages with appimagetool, you just have to pass the -u flag and give it the update information string, that process is well documented, and there's even a GitHub releases type which interprets what the GitHub releases API returns. That being built in makes the library pretty portable and self contained.

libappimageupdate is built to serve the purpose of making use of the update processes specified in the AppImage specification. There won't be support for anything other than this in the library itself. If you really insist on your other ideas (which in my opinion aren't any superior (easier to use, more efficient, ...) to what AppImage provides), you'll have to write these bits yourself. Feel free to take a look at libappimageupdate's code and how it makes use of libzsync2.

nasty code

Please elaborate on what you consider nasty code in relation to libappimageupdate, AppImageUpdate, the AppImage update mechanisms or anything related. I can't really reenact this part.

Oh, and by the way, thanks for the kind words about zsync2. I know it lacks most docs at the moment, but I read from your words that you figured out how to use it. Feel free to add some docs or write some requests for such docs in issues on GitHub.

[antony-jr]

@TheAssassin not extracting the appimage , if the program is executed ( i.e the Updater or the main program [ Both are same ]) , it is loaded to the memory , now the AppImage and our instance of the App i.e the Updater is detached and thus we require no operation calls to read the configuration and so this is much faster than reading and parsing (and the parsing is really not a good practice ) the first 4 bytes from a file in the disk.

[antony-jr]

we can also use zsync's sha1 checks on the final result , if wanted!

[antony-jr]

I think I should rewrite zsync for Qt with a clean API interface. So please wait!

[probonopd]

I thought that was just what @TheAssassin had done with zsync2?

[TheAssassin]

@probonopd zsync2 has a C++11 interface, so, standard types instead of the Qt versions (but actually it's all compatible...). I don't see the issue leading to the decision of @antony-jr to rewrite the original zsync (which has had a lot of issues, which took many weeks of intensive work to fix all those issues in zsync2. You shouldn't underestimate the amount of engineering that was necessary to create both libzsync2 and libappimageupdate.

[antony-jr]

Yeah , I'm fine with it , once I finish with zsync I can easily build the Updater with elegance. I'm aiming for a api like this.

AIUpdaterBridge Bridge({
       {"username", "antony-jr"},
       {"repo" , "QArchive"},
       {"version", "0.0.1"},
       {"assetFormat", "QArchive-{version}.tar.gz"},
       {"installationPath", "./"},
       {"debug", false}
});

just simple as GHReleaseBridge.

[probonopd]

@antony-jr of course it is your decision but I would prefer to just have one supported version that we use for AppImage, so that we need to fix issues in just one rather than multiple locations. libappimageupdate (which uses libzsync2 internally) was specifically developed with being used in GUI frontends including Qt ones in mind.

[antony-jr]

Then why mix it with FlTk , Qt is just perfect by its own.

[probonopd]

I'm aiming for a api like this.

What should that API do?

Please be aware that the AppImageSpec specifies how update information is handled inside AppImages. All of the work has already been done by @TheAssassin. Please don't re-invent the wheel, just focus on making a Qt GUI for libappimageupdate which does all the real work.

[antony-jr]

yes thats just a template

[antony-jr]

I do use some code from @TheAssassin 's work.

[antony-jr]

I'm Just wrapping the client with Qt , I will be using zsync-curl as its much cleaner! and curl is not that bad.

[probonopd]

Please note that we want all the AppImages be updated in the same way. We don't want some AppImages that do not use update information as specified in AppImageSpec but use some other way of updating, because this breaks all other existing AppImage-handling and updating tools out there.

There are already applications that can update all the AppImages on a system. If you invent your own thing, that would break.

[probonopd]

{"assetFormat", "QArchive-{version}.tar.gz"}

Please explain what that means. AppImages are never supposed to be inside a .tar.gz.

[antony-jr]

its just a template , I'm following the Specs. Just concentrating on understanding zsync for now.

[probonopd]

       {"username", "antony-jr"},
       {"repo" , "QArchive"},

Please explain what that means. AppImages are supposed to be updated from the update information stored inside the AppImage. How this works is specified in the AppImageSpec, and libappimageupdate has already implemented it.

[probonopd]

its just a template

Yes but my point is - there is no need for a "template" since the information you are looking for is already in the AppImage and in the tools we have made for dealing with AppImages, such as appimagetool. Before you invest a lot of time to create new things, please invest some time to learn about what is already there.

I'm following the Specs

:+1: This is the relevant part: https://github.com/AppImage/AppImageSpec/blob/master/draft.md#update-information

[antony-jr]

okay

[TheAssassin]

@antony-jr you're constantly mixing up projects. Are you referring to zsync, zsync-curl or zsync2? One never knows what you're talking about.

Short history lesson:

• zsync: original project, written in very old style C, designed as CLI tool, so a lot of static globals etc., not at all usable as a library by other projects
• zsync-curl: zsync, but using cURL as the backend to have support for HTTPS connections. Has had the same issues as zsync, as it was pretty much the same code base.
• zsync2: Mostly a rewrite of the relevant code, especially main code and a bunch of other problematic parts, provides a library with a clean C++11 API, everything's contained in instances of ZSyncClient, etc. Meant to be used by external projects. Has been developed to be used by AppImageUpdate, libappimageupdate and the tools which want to make use of those.

[antony-jr]

I first saw zsync2 but the libs are not good with Qt , so I got into zsync-curl

[probonopd]

Can you please describe what exactly you mean by "the zsync2 libs are not good with Qt" and why you think that zsync-curl is more suitable for Qt?

Also, why do you think you need to interact with zsync2 directly at all? It should all be wrapped by libappimageupdate which would be what you would be using for Qt.

[antony-jr]

I think I will go with zsync2 ?

[antony-jr]

I just thought , why use open source libs when Qt got them all , cpr , args , and gtest.

[antony-jr]

if we can replace those libs with Qt , we can easily create a amazing async model for zsync2

[antony-jr]

but now its not the time right ?

[antony-jr]

okay getting back to work!

[antony-jr]

@TheAssassin I will be using zsync2 for now , if possible I will help you later when I finish this project.

[probonopd]

I just thought , why use open source libs when Qt got them all , cpr , args , and gtest.

The reason is that we need it for non-Qt projects, too. So we are aiming at a an architectural separation between a toolkit-independent libraries and Qt, Gtk+,... GUIs. This way, we don't have to maintain the core code in more than one place.

[TheAssassin]

why use open source libs when Qt got them all , cpr , args , and gtest.

That's a very weird way to look at free software...

zsync2 will most likely never ever be built around Qt. Qt is a way too big dependency for this tool. args is an amazing command line parser, which works perfectly fine, is very compact, and can be linked statically without any hassle. Same goes for gtest, but it's way less relevant in such considerations, as it's only used for unit testing (which we don't have too many of anyway at the moment).

For AppImageUpdate, there were thoughts about building a Qt interface. But noone's thinking of writing core itself (libappimageupdate) against Qt. And you shouldn't expect this to change. You seem to be pretty confident that a rewrite in Qt would be desired, which is not the case...

[antony-jr]

Qt Framework has a async model. Its about self-updating right. If Qt is used for the libappimageupdate then we can build elegant Qt GUI which will work on all linux distros and thus using Qt is wise because when we use Qt GUI libs we get all the advantages of Networking , Argument Parsing and other things together so why waste them ?

[antony-jr]

if we can use only one framework then maintaining the code is much easier! Qt docs are good too..

[antony-jr]

Qt and AppImages are the best combination.

[probonopd]

if we can use only one framework

We cannot.

AppImage and AppImageUpdate are not just for Qt apps, but we want the updater experience to have a native look and feel in Qt apps. Similar for other application frameworks. So we need to use Qt for Qt applications and Gtk+ for Gtk+ applications and Electron for Electron applications.

If Qt is used for the libappimageupdate then we can build elegant Qt GUI

And what would Gtk+ applications and Electron applications use? They would certainly not add a Qt dependency to their code just for the updating. Or would you accept a Gtk+ dependency in your Qt applications?

[antony-jr]

ok got it!

[antony-jr]

I am trying to do something like this , https://github.com/AppImage/AppImageUpdate/issues/30 . with Qt Only

[antony-jr]

I already done it with an electron app but just a prototype with python and Qt. exercism-gui-installer

[antony-jr]

@TheAssassin Do I need the submodules for CPR , because cmake is throwing an error. I'm forced to clone the entire curl lib and mongoose project , but the instruction is not given in the readme of zsync2 , so I'm confused! Do I need to clone the submodules of CPR ?

[probonopd]

@antony-jr what are you trying to do?

Please follow the instructions given in https://github.com/AppImage/AppImageUpdate/wiki/Guide-to-libappimageupdate - at which step are you running into the difficulties you describe?

[antony-jr]

got it.

[TheAssassin]

You need the submodules, yes. They aren't used by cpr actually, but the default CMake configuration expects them. A workaround is planned but not yet ready to use.

[antony-jr]

From the discussion I've finally come to a conclusion.

What does this bridge provide ?

• As I said , using libappimageupdate , this header-only ( or a lib? ) will provide a light-wieght bridge to Qt , Thus it makes it easier to make native GUI.
• Even if the App is not a Qt based program we can still use something like a AppWrapper , where the AppWrapper reads the configuration from a json file or inside its code block. This AppWrapper will open a QProcess when there is no Update And starts a GUI Updater when there is an Update thus Even Non-Qt Projects can implement the autoupdate without hassle, AppRun will point to AppWrapper and inturn the AppWrapper will point to the program executable (this configuration can also be inside the json file , self contained inside AppWrapper using Qt Resource System ) , Then finally the user can simply throw the AppWrapper , Configuration files(json) and their executables inside the AppImage , When there is a new update , AppWrapper is used to update the AppImage and thus the new configuration is provided from the update itself.
• Since the new update replaces AppWrappers or The AppImages configuration or magic codes , we don't need to have a central db

This is just a bridge and will not provide a GUI , it just makes it easier to integrate with Qt Apps , The AppWrapper was my mainstream.

Note : GHReleaseBridge was able to Update AppImages Automatically without libappimageupdate but as @probonopd said I am replacing or reinventing this bridge for zsync and libappimageupdate.

[probonopd]

Even if the App is not a Qt based program

Then the application will not want to bunde AppImageUpdaterBridge, but use something else with a native UI (e.g., Gtk+, Electron) instead. So I would take this out of scope.

we can still use something like a AppWrapper , where the AppWrapper reads the configuration from a json file or inside its code block

The "configuration" is already part of the AppImage. It is called the "update information" and libappimageupdate already handles it for you. No need to reinvent.

(Skipped the sentences talking about AppWrapper and json files, since I think these are not needed)

This is just a bridge and will not provide a GUI

Now I am entirely confused. I thought the whole point of this exercise was to take libappimageupdate and build a Qt GUI around it that applications can embed.

User Story: "I as an application author want to make my AppImage self-updateable from the menu, "Check for newer version and update..."

Something like this:

Or do you have something different in mind?

[antony-jr]

As the title said its a Bridge , think of it as a core to the main GUI. I will creating this bridge because.

• libappimageupdater does not go well with signals and slots. and thus this part has to be wrapped with signals and slots to make a good Qt GUI.
• Using this bridge I will also create a GUI , Some user may want to use their imaginations so they can also use this bridge ( I will document this too ).
• Having a clean logic first is the key to a great GUI App and thus I think I will have to concentrate on the Core and then move on to the GUI part.

[TheAssassin]

Basically, I think it is fairly simple to write a Qt-ish wrapper for libappimageupdate, which I think is what you're trying to do. libappimageupdate works asynchronously by creating threads when necessary, that works quite automagically. I think this is what you're trying to do, but with every new content, you kind of change your announcements...

[antony-jr]

@TheAssassin True. Thats why this is just going to be header-only , easier on the users and developers. Just copy the header , refer the docs , design your gui and bind the controls.

The Default design would what we will be creating with this bridge.

Note : libappimageupdate does not go well with signals and slots eventhough its asynchronously. what about progress , its not in async , you have to loop through which will block the main thread and will cause some unexpected results. Not only progress but some things like error handling can be done cleanly with signals and slots.

i even gone through the source but the progress is not async , its like a series of sync without blocking!