appimaged (or a new AppImageShareDaemon -- insertion by -kp-) could advertise itself in LAN with Zeroconf so that other appimaged instances could show applications residing on other machines in the same LAN, too.

One could even have a dedicated appimaged instance on an "AppImage server" (e.g., a Raspberry Pi). This would be a nice showcase for AppImages running from network shares.

I. Background

ZeroConf is formally specified in a couple of IETF RFCs:

RFC 6762: "multicast DNS" (mDNS)
www.ietf.org/rfc/rfc6762.txt
RFC 6763: "DNS Service Discovery" (DNS-SD)
www.ietf.org/rfc/rfc6763.txt

mDNS works on the local network completely without and/or independent from a "classical" central DNS server being present or configured. It can register hostnames/IP-adresses to the local. domain (note the trailing dot here!), and more. As the name says, it works through "multicast" packages.

DNS-SD is an additional protocol which can register any kind of service on the network. It uses TXT records riding in the mDNS packages to make its info public.

For AppImages to share on the network it would be just the task of defining a few well-thought out set of TXT record names and the kind of values they should contain. A peer client wanting to use one of the services would be able to pick up all info it needs in order to connect and use it. (See further down with a few very quickly made-up TXT record names which of course are not yet well thought out.)

For Linux it is the Avahi project which implemented the ZeroConf/mDNS/DNS-SD functionalities.

CUPS versions from already a few years back use this to publish available shared printers (they no longer use the old "CUPS browsing" protocol, they now use DNS-SD/mDNS).
Apple uses it for a lot of other services as well ("Find my Mac" / "Find my iPhone" / TimeMachine Backup / iTunes music sharing...).
Many open source projects have embraced it as well in order to make any service easy to discover which they provide on networks (Gobby, iChat, Bazaar, KDE System Guard, PulseAudio, PostgreSQL... to dump a rather complete list of well-known service names in human readable form just run avahi-browse -d | more. Only NextCloud-ers do not yet understand how much more userfriendly they could become if they embraced a feature request asking for it.)

Apple decided to use Bonjour as its marketing name for ZeroConf (basically, they invented the whole thing as well -- they hired Stuart Cheshire to implement it, after he had first started to discuss these ideas on some newsgroups before he was an Apple employee).

II. DEMO

Just a very quick demo how that could look...

It is very easy to make a (currently still fake) service announcement with the help of avahi-announce.

I came up with a few fake TXT records to put into such an announcement and made it in my local WLAN/LAN. I invoked the following two commands on two Raspberry Pis, one running Raspbian, the other openSUSE Tumbleweed:

$  avahi-publish        \
    --subtype=_zsync2._sub._appimaged._tcp  \
    --service "AppImage Daemon (brand-new AppImages on store)" \
      _appimaged._tcp   \
      666               \
      zsync2="yes"      \
      ipfs="yes"        \
      dat="yes"         \
      daturl="dat://84607344b36b54a3352eb41fe772702615da086c9bc6a436d288bdf9cc68481b" \
      zsync2url="https://ipp-everywhere-pi3.local.:1337/zsync2/" \
      ipfsurl="/ipfs/QmYwAPJzv5CZsnA625s3Xf2nemtYgPpHdWEz79ojWnPbdG/readme"

$  avahi-publish       \
    --subtype=_zsync2._sub._appimaged._tcp \
    --service 'AppImage Daemon (only stable AppImages here, sorry!)' \
      _appimaged._tcp  \
      987              \
      zsync2="yes"     \
      ipfs="no"        \
      dat="yes"        \
      daturl="dat://8e35d1942265e6dc55537bfcbedefeb6c277686810a3781f90df1b63c72d5ec4" \
      zsync2url="https://leapraspi.local.:1337/zsync2/"

Now see, how a Mac running Bonjour Browser can pick up this (for him completely useless) info alongside other info using ZeroConf (a.k.a. Bonjour on the Mac):

macOS: Bonjour Browser displaying AppImage service announcement info

Next, this is how an iPhone or an iPad sees it, if the app Discovery (which also browses DNS-SD announcements) is installed:

How to navigate iPhone App 'Discovery'

iPhone App 'Discovery' displaying info from AppImage service announcement

The avahi-discover GUI browser on Linux shows this:

avahi-discover on Linux displaying info from AppImage services available in the LAN

An avahi-browse command line also picks up this info easily from any Linux node on the network:

$  avahi-browse -a -r -t | less
[....]
=  wlan1 IPv6 AppImage Daemon (only stable AppImages here, sorry!) _appimaged._tcp      local
   hostname = [leapraspi.local]
   address = [fe80::ba27:ebff:fe74:37f4]
   port = [888]
   txt = ["zsync2url=https://leapraspi.local.:1337/zsync2/" \
          "daturl=dat://8e35d1942265e6dc55537bfcbedefeb6c277686810a3781f90df1b63c72d5ec4" \
          "dat=yes"  \
          "ipfs=no"  \
          "zsync2=yes"]
=  wlan1 IPv4 AppImage Daemon (brand-new AppImages on store) _appimaged._tcp      local
   hostname = [ipp-everywhere-pi3.local]
   address = [192.168.76.27]
   port = [666]
   txt = ["ipfsurl=/ipfs/QmYwAPJzv5CZsnA625s3Xf2nemtYgPpHdWEz79ojWnPbdG/readme" \
          "zsync2url=https://ipp-everywhere-pi3.local.:12080/zsync2/" \
         "daturl=dat://84607344b36b54a3352eb41fe772702615da086c9bc6a436d288bdf9cc68481b" \
          "dat=yes"  \
          "ipfs=yes" \
          "zsync2=yes"]
[....]

Of course, an end-user would never need to see these complicated looking messages. Above stuff I only wrote for reference in advance of actually implementing it, before development starts.

Users would have just a very simple and super-easy to use interface, were they could

...enable or disable "AppImages Exchange and Update Browsing"
...enable or disable "AppImages Sharing for My Own"
...if enabled, see all other AppImage daemons announcing their AppImages
...see a list of AppImages available on the network
...be able to download or update those AppImages which they want to.

So far, so clear - but how would we announce the individual AppImages that are available, the information from their desktop files and icons?

You don't announce all that information via Zeroconf at all. What you can do with Zeroconf systems is "service discovery", i.e., announce that there's a service, e.g., appimaged. Other appimaged instances can use the IP-port pair and then use a normal TCP connection (to reach an HTTP API for instance) for the actual data exchange.

But before we start to just share "all" the information with "everyone" please remember that a lot of systems are used in non-private networks as well, where sharing information about all AppImages is most likely not desired (or even not permitted, which could cause serious trouble for an employee). When designing this feature, we'll have to take into account those considerations. The information which applications are run on which systems is very sensitive. This kind of meta data must not be disclosed irresponsibly. Non trustworthy systems like the PlayStations, Android devices, and other proprietary systems (including Windows) scan the networks they use regularly, too, and since they're proprietary, you can't know what information is disclosed to whom. We should at least make it as difficult as possible to intelligence agencies, proprietary software vendors etc., to abuse this feature for collecting meta data about people.

I'd suggest to implement encryption of those transports and add an authorization layer to the system, i.e., by generating self-signed TLS certificates, and whenever a user tries to connect to the own appimaged, appimaged can ask the user whenever that user is allowed access (showing the information from the certificate). Also, instead of sharing a list of AppImages available on the system, we should make it a "I need this file, do you have it?" like system. Sharing the entire list doesn't really add anything to the update process, as the question to be answered for a single update process is where to get the blocks for a single file from.

Thinking about the example of non-private networks, one might also receive a lot of connection requests in such systems, when they're not properly isolating their clients from each other. There should be a way to either recognize non-private networks (which is pretty much impossible, remember the dialog in Windows asking about whether this was a private, work, or public network?), or ask for authorization for single networks on connection.

All those things do disturb UX a bit, but the cost-benefit factor clearly shows that the additional security brought by these systems are worth it.

By the way, when adding networking to appimaged, we'll really have to perform a code analysis, to make sure the system is secure and doesn't have any security issues allowing remote code execution etc. In the current state, I could imagine there's quite a bunch of bugs that might be used for such purposes. We recently started a memory leak hunt in AppImageKit's code, which is a good start, but we should perform more of those analyses to search for buffer overflows etc.

User Stories

Simon prepares for a public talk at FOSDEM 2019 about "AppImage -- How One File Is Multiple Applications". He wants to share a few dozen new AppImages with his audience and sets up a new Raspberry Pi system using openSUSE Tumbleweed.

He installs everything and configures it in general, making sure all his AppImages are there as well. He enables appimaged to run, as he always does. Also, he downloads the newly released AppImageSharingDaemon-x86_64.AppImage and starts it.

The phone rings, he gets distracted and after the call he's in a hurry to catch his train to Bruxelles.
Sylvie sits in the audience FOSDEM 2019, waiting for Simon to appear for his talk.

She already knows a thing or two about the new feature(s) Simon is about to demo for the first time. She also has started AppImageSharingDaemon from an AppImage on her Debian notebook, and made sure she had added the --appimage-sharing-takeonly CLI parameter.

After Simon entered the room and booted up his Raspi, Sylvie expected to see at least one instance of "AppImage Sharing Host(s)" appear in a list. When after two more minutes this didn't happen, she sends a DM to Simon: "Did you remember to use --appimage-sharing-give too?"
Simon does not see the DM from Sylvie, but in at last minute he remembers himself.

He connects to the Raspbi @ "https://ai-raspbi.local.:60080", authenticates, goes to the config and checkmarks the Enable sharing all selected AppImages.

Then he starts his talk.
Finally, Sylvie can see an instance of "AppImage Sharing Host(s)" appear in the list. Its name is listed as "Simon's AppImages (Public)".

She does not see a hostname or a domainname. She clicks the link provided. A local browser-like pane in her AppImageSharingDaemon-GUI appears and in it a list of about a dozen AppImage names.

Whenever she clicks the "details" checkmark beside a name, she gets more info about the AppImage. Amongst these details is the exact version info about the app itself as well as which version of AppImageKit was used to package it, name of the packager, its size, updateinfo, facts about the signature and some more.

Sylvie starts the download of an AppImage with an interesting sounding name from Simon's shared ones and turns her attention back to the talk
Kurt, sitting beside Sylvie, is shoulder-surfing and can partially follow what she is doing.

He recognised that (and heard Simon mention it just now) that he needs the new and separate AppImageSharingDaemon thingie to run in order to take advantage of this new feature. After downloading it from github.com/AppImage/AppImageKit/releases he also starts it.

His intention is to share one specific AppImage he had packaged the day before.

But it doesn't seem to work. He whispers to Sylvie, asking: "How does this thingie work?" Her short answer is: "Look at the --help output, dude!"
Kurt runs AppImageSharingDaemon --help and sees, amongst other hints, two lines saying:

--appimage-share-files='/path1/appimage1,appimage2,...' : Share only named appimages
--appimage-share-files='all' : Share all appimages found by appimaged

He does as advised and shares his one special AppImage.

At this very moment, Simon points this out in his talk: "You can specify which AppImages to share either with a CLI parameter or through a GUI."

Kurt now concentrates on how Simon explains and demos the GUI.
Sylvie checks a thing in her setup.

She looks at her "dot" files, looking for a directory Simon had just mentioned. It is supposed to be named "${HOME}/.appimagesharingdaemon/shared/".

It's not there. Nor is the other mentioned directory "${HOME}/.appimagesharingdaemon/nevershare/" She had seen Kurt's instance appear in the list of "AppImage Sharing Host(s)", so she knows the command line works. While she is normally using CLIs, this time she wants to see how the (Web)GUI of AppImageSharingDaemon looks like.

She connects to https://localhost:60080/, authenticates and proceeds to mark two of her many AppImages to be shared publicly.

For this, she has to browse through her file system, because one of the two is not (yet) in her standard AppImage dir, but still in "${HOME}/Downloads", and she does not bother to move it now.

After she's done, she looks again for her dotfiles.

Now she sees a newly created directory: "${HOME}/.appimagesharingdaemon/public/" is there now. Inside she identifies two symlinks pointing to the AppImages she previously selected to be shared. And one of the symlinks indeed goes to "${HOME}/Downloads/my.AppImage".
After Simon's talk is over, both Kurt and Sylvie go look and poke a bit more at the (Web)GUI of their respective AppImageSharingDaemon.

They are pleasantly surprised that the HTML help pages seem to be rather complete and explain very clearly how it works.

They also can see that the list of their own shared AppImages is nicely organized and easy to modify. They can un-share all currently shared AppImages at once, or un-share them selectively. There is even a list of "Previously shared" AppImages, whose sharing status can be restored very fast. Plus, there are even separate lists which let them specify networks that get special treatment. They say, for example, "Always share on 192.168.71.0/24" or "Never share on 134.169.0.0/16".
Sylvie also noted, that in her list of AppImages presented by AppImageSharingDaemon, there was one marked as:
```
"Partially downloaded today:
   Name: <interesting-sounding-name>
   88% from 'Simon's AppImages (Public)';
   SHA512: a7f7a85a7e4c05c3333dc6ccc1d44a101c477ae8d7dc97e6ebf3fafeabf88ca7"
```
Besides this entry, there's an option to "Stop -- Don't try to complete this download. She does not bother to enable this and closes here notebook.
In the evening Kurt, TheAssassin and Sylvie want to meet Simon for some Asian food. The place is crowded. Simon is late.

Kurt shares his mobile phone hotspot with the other two, because they do not want to connect to the Asian restaurant's open WiFi. They all play a bit, discuss about and share impressions about Simon's talk about their new AppImageSharingDaemon toy, when suddenly Sylvie notices a modified entry in her list:
```
"Downloaded today (completed):
   Name: <interesting-sounding-name>
   88% from 'Simon's AppImages (Public)' (1st chunk);
   SHA512: a7f7a85a7e4c05c3333dc6ccc1d44a101c477ae8d7dc97e6ebf3fafeabf88ca7"
   12% from 'TheAssassin's AppImages for Nerds' (final chunk);
   SHA512: a7f7a85a7e4c05c3333dc6ccc1d44a101c477ae8d7dc97e6ebf3fafeabf88ca7"
```

@TheAssassin:

"You don't announce all that information via Zeroconf at all. What you can do with Zeroconf systems is "service discovery", i.e., announce that there's a service, e.g., appimaged."

Correct.

@probonopd, if you ever used AirPrint on an iOS, then you've seen that you do not see all the details for all the printers available. You get a list of print service names first. Then you select one. Only for the selected on you get more details (not so much, in the case of AirPrint).

Before selecting a printer, everything happens via mDNS. After selecting it, the print client makes an IPP-connection (using info such as IP address and port learned from the mDNS browsing) to the printer and does a "get-printer-attributes"-query next. After an IPP response, the GUI options to select duplex (if available), stapling (if available), colorprinting (if available) and punching (if available) get displayed. Unavailable print options will not be displayed at all.

"But before we start to just share "all" the information with "everyone" please remember that a lot of systems are used in non-private networks as well, where sharing information about all AppImages is most likely not desired (or even not permitted, which could cause serious trouble for an employee). When designing this feature, we'll have to take into account those considerations. The information which applications are run on which systems is very sensitive. This kind of meta data must not be disclosed irresponsibly."

Of course.

We are only starting to discuss such a feature. And surely, this needs a lot of debate and hashing out first. Not a single line of code should be written (unless in a proof-of-concept branch), unless all major features and self-imposed restrictions for such a feature-set are clear.

"We should at least make it as difficult as possible to intelligence agencies, proprietary software vendors etc., to abuse this feature for collecting meta data about people."

Whenever you take part in computer networks, meta data are abound...

Of course, this must be an opt-in feature, and not be enabled by default. There are many ways possible to think up, how exactly that could be implemented. One of the simplest concepts to do this is in a separate sibling daemon to appimaged, which has to be "installed" separately, and which has to be "enabled" separately, and where the user can

Enable "listening" only (getting info about available AppImages).
Enable "sharing" in addition (announcing info about which AppImages he himself shares)
Disable/Enable all sharing info per individual AppImage...

But as @probonopd mentioned, this could be especially interesting for separate nodes in a network, not necessarily running any AppImage by themselves, because they don't have an interactive user sitting in front of their screens, but living the miserable life of a Raspberry Pi in some dark, dusty, cobwebby cupboard.

"Also, instead of sharing a list of AppImages available on the system, we should make it a 'I need this file, do you have it?' like system."

That's pretty much against the whole idea. How would a user know in advance which AppImage he/she is interested in? Often they'll not know.

Very often it will happen instead that they see a surprising one and think "Oh, this is available as an AppImage? I'm interested, I want to try it now..."

"By the way, when adding networking to appimaged, we'll really have to perform a code analysis, to make sure the system is secure and doesn't have any security issues allowing remote code execution etc."

That's right. And appimageupdate already does "networking", so it's not completely new.

"We recently started a memory leak hunt in AppImageKit's code, which is a good start, but we should perform more of those analyses to search for buffer overflows etc."

And that's a good thing!

"By the way, when adding networking to appimaged, we'll really have to perform a code analysis, to make sure the system is secure and doesn't have any security issues allowing remote code execution etc."

That's right. And appimageupdate already does "networking", so it's not completely new.

AppImageUpdate doesn't share any code with AppImageKit. Also, AppImageUpdate doesn't run as a daemon. As we know the AppImageKit code base has quite some issues in that direction, it should be added to the TODO list to perform a security related code review before making it a network service.

Generally, I'm just adding my security considerations about what I read so far. I only consider such a system for use with AppImageUpdate, not the initial AppImage distribution, for which an implementation is relatively simple, and secure. Initial distribution has a lot more issues and considerations to take into account than AppImageUpdate's use case has.

But hey, it's open for discussion.

We are only starting to discuss such a feature.

I did understand that. And I give my opinions. Although it's quite detailed already, it doesn't mean that it has to be done that way.

"Downloading in advance" seems like a totally bad idea. Some troll could easily sit down on a conference and "share" a ton of bloated AppImages to spam the target system. This is never a good idea. There once was a website spamming computers with cat pics... That would only make sense when people would look into a store like app what apps they can install which are available locally or so. But then, a UI was required, which would go against the principle of being automagic that most AppImage tools share.

Users having to enable federation like that (or even AppImage wise), especially on the CLI, seems a bit too complicated UX wise. We'll most likely need a UI for that, maybe with checkboxes, or a little context menu for the DE's launcher.

Whenever you take part in computer networks, meta data are abound...

That is a bit of a weird attitude about the principle of data economy ("Prinzip der Datensparsamkeit" in the "Bundesdatenschutzgesetz" ("BDSG")) which is generally a good idea to apply to any information systems. One shouldn't just "share" stuff for the sake of it. Like with the old SMB shares which Windows just created, which any network service could read out (at least read only) without any authentication. I don't like the idea of just sharing all applications I have installed, and I don't think we should make anyone do so. Make that functionality yield the Tor browser for instance., that flags you as suspicious to third parties.

Generally, I think we should not focus on the initial AppImage distribution, that's more of an issue for App Store like applications like Nitrux's Software Center which @azubieta develops.

@TheAssassin wrote:

"AppImageUpdate doesn't share any code with AppImageKit. Also, AppImageUpdate doesn't run as a daemon."

Whatever... Don't expect me to (already) know such fine details. And does it matter in the context of discussing $broad_topic of issue #556?

"As we know the AppImageKit code base has quite some issues in that direction, it should be added to the TODO list to perform a security related code review before making [...]"

Ok, go ahead.

"And I give my opinions. Although it's quite detailed already, it doesn't mean that it has to be done that way."

Feel free to propose other "user stories".

"'Downloading in advance' seems like a totally bad idea."

Was that ever proposed? Where? Here?

Or are you beating a dead horse?

"One shouldn't just 'share' stuff for the sake of it."

Dead horse...

"I don't like the idea of just sharing all applications I have installed, and I don't think we should make anyone do so."

Dead horse...

"Generally, I think we should not focus on the initial AppImage distribution, that's more of an issue for App Store [...]"

Häh? I don't understand what you mean by "initial AppImage distribution". Or is this again a dead horse?

Please comment on topic as $headline of $issue here formulated.

For other things, open a new $issue.

I think we have two separate use cases that we should distinguish:

Imagine a large organization, e.g., a university. Now imagine a new release of LibreOffice comes out. It would be good if not everyone would have to download everything from the public Internet, but rather from the local network. In this case, the user already knows what he wants to have. This use case could be addressed by a system like IPFS "automatically" (without additional implementation of Zeroconf on our side)
Imagine a household (or the same user using multiple machines). Wouldn't it be cool if the user had an option (opt-in; like sharing printers or files) to share (where the license allows it) already-downloaded applications, in the easiest possible way? In this case, the user doesn't already know what he wants to have. This is where a Zeroconf implementation on our side could potentially simplify things

This is what NeXT did in NeXTSTEP 1.0 in 1989, quoted from their README:

AppImage / AppImageKit

AppImageShareDaemon: Advertise in LAN with Zeroconf #556

I. Background

II. DEMO

User Stories