search

malxau / yori

Yori is a CMD replacement shell that supports backquotes, job control, and improves tab completion, file matching, aliases, command history, and more.
http://www.malsmith.net/yori/
MIT License
1.24k stars 31 forks source link

github binary release #21

Open t214c opened 4 years ago

t214c commented 4 years ago

Hi, Malcolm

I remember an issue opened a few weeks ago about this but I can't find it. You said the binaries are available at your site but there area couple of problems:

It would be very helpful if you added the binary releases here too. Not the online installer (because it would have the same problem) but maybe:

Thanks a lot for understanding!

malxau commented 4 years ago

The issues that seem related to this are #6 and #10 .

This seems like it's referring to two different things:

The reason that I haven't been looking at the offline point is because software distributed these days needs to have an updating story, and the updating system implies an online packaging system, and once that exists an installer consuming it is very natural. Put another way, if the same bits happened to be installed via some other mechanism, there'd still be an online updating component that has all of the flaws it currently does.

Since Yori has no registry entries or similar, copying it to a new machine is just about copying the files. There needs to be one Internet connected machine to download content either way.

In terms of establishing trust, I'd fully admit that what exists today is deficient. Establishing trust with an online updater seems like it can't be done with hashes, because the hashes of a new version are unknown, and if they can be communicated in a tamper proof fashion they wouldn't be needed because the update could be communicated directly via that mechanism. What I'd been assuming is the real solution for integrity checking in an online world is a form of digital signature that can enforce same-origin, so that any new version can be attested to come from the same source as the previous version. Once that is done, TLS is redundant, and the only remaining issue is establishing trust of the stub installer. That said, I've been looking into other areas and am happy to accept contributions.

The reason for not using github to host the online repository is because it is accessed by client software, it forms part of an API, so the repository needs to be in a specific form that can be guaranteed to exist in a compatible form for a long time. It seems easier to do this with a full VM host and long-lived DNS entry, because I can promise to make it work for a decade, whereas I cannot promise that a third party service will exist in unmodified form on that kind of timescale.

I'd suggest though that online installation and online updating are becoming expected features for software, and I'd encourage you to consider how to solve issues within the realm of the online installer rather than trying to push the updating problem to the user. My life would be a lot easier if updating was a user problem, but that doesn't seem to be where things are going.

t214c commented 4 years ago

Apologies for the late reply!

My short response: I think it is still a useful thing to add a binary release here on GitHub.

In more details:

I agree that keeping software updated is almost always a good thing, but you there are a few problems of requiring an online installer:

  1. You may not have an internet connection at installation time.
  2. Not all machines have (or should have) an internet connection. You should be able to take a full installer with you to another machine using a USB stick or something similar.
  3. The problem with the lack of TLS on your site is not just MITM, there is also passive snooping on the contents.
  4. Even with TLS, a connection to you site is in itself a point of identification, as opposed to something as popular as GitHub.
  5. This is a secondary distribution method for those who need it. You can choose not to mention it on you site or in the ReadMe here and even put a warning to discourage it in favor of the main update mechanism. If the user has a reason to use this other method, I think you should allow it.

Establishing trust with an online updater seems like it can't be done with hashes, because the hashes of a new version are unknown, and if they can be communicated in a tamper proof fashion they wouldn't be needed because the update could be communicated directly via that mechanism.

You can put the hashes on GitHub. It would be less likely for someone to compromise both your site and your GitHub account.

The reason for not using github to host the online repository is because it is accessed by client software, it forms part of an API, so the repository needs to be in a specific form that can be guaranteed to exist in a compatible form for a long time. It seems easier to do this with a full VM host and long-lived DNS entry, because I can promise to make it work for a decade, whereas I cannot promise that a third party service will exist in unmodified form on that kind of timescale.

You don't have to use GitHub to host the installer-accessed repository. Just a copy of the binaries for those who need it.

malxau commented 4 years ago
  1. You may not have an internet connection at installation time.
  2. Not all machines have (or should have) an internet connection. You should be able to take a full installer with you to another machine using a USB stick or something similar.

This should work today. See things like "ypm -download-stable". Obviously you need some internet connection somewhere at some time, but you can install many machines from it. Note also that ysetup.exe looks for packages in a subdirectory by default which was done to allow USB/ISO installation. See SetupLocalPathsToCheck .

  1. The problem with the lack of TLS on your site is not just MITM, there is also passive snooping on the contents.

This is saying that you trust me to know that you downloaded it, but don't trust your ISP.

  1. Even with TLS, a connection to you site is in itself a point of identification, as opposed to something as popular as GitHub.

This is saying that you don't trust me but do trust GitHub/Microsoft.

Obviously people are free to decide who they do and do not trust. Note that in terms of source code, you can download it from either my server or GitHub, and after that me/GitHub/Microsoft/your ISP have no idea what you choose to do next.

You can put the hashes on GitHub. You don't have to use GitHub to host the installer-accessed repository.

These two things directly contradict each other. If the installer needs to talk to GitHub, it becomes part of the API. I'm treating GitHub as a useful place to host the code where other people can see it and contribute to it, but if it disappears tomorrow, the code still moves forward.

The real point is, this is MIT licensed code. I'm going to keep trying to ensure the online installer meets as many needs as it can, and if others would prefer to build their own binaries and/or distribute my binaries in some other way, they are certainly free to do so.

t214c commented 4 years ago
  1. Even with TLS, a connection to you site is in itself a point of identification, as opposed to something as popular as GitHub.

This is saying that you don't trust me but do trust GitHub/Microsoft.

No! I trust you enough to run your code on my machine! I trust that GitHub won't change your code or releases. I mean behavioral tracking. Accessing niche websites helps identify a user across IPs or even countries as opposed to connecting to GitHub and SourceForge. It is not the end of the world, just a point I was making.

Obviously people are free to decide who they do and do not trust. Note that in terms of source code, you can download it from either my server or GitHub, and after that me/GitHub/Microsoft/your ISP have no idea what you choose to do next.

I wish I was skilled enough to do that!

You can put the hashes on GitHub. You don't have to use GitHub to host the installer-accessed repository.

These two things directly contradict each other. If the installer needs to talk to GitHub, it becomes part of the API. I'm treating GitHub as a useful place to host the code where other people can see it and contribute to it, but if it disappears tomorrow, the code still moves forward.

I meant putting the hashes as a helpful addition, not make it part of a new delivery method to replace the current one. Many many developers release their programs on GitHub with checksums, even if they have their own website which also has the downloads.

The real point is, this is MIT licensed code. I'm going to keep trying to ensure the online installer meets as many needs as it can, and if others would prefer to build their own binaries and/or distribute my binaries in some other way, they are certainly free to do so.

I didn't mean to antagonize you. It was just a request. Sorry if I came across as demanding.

pgodwin commented 7 months ago

Agree with your points, however adding another use case: older machines not connected to the internet. It'd be great to be able to drop the binaries straight over to them via say CD-R or Zip Disk.

Alternatively, if a "download only" option was added to ysetup so that the packages can be downloaded rather than installed. Looking at it now, when I run ysetup it pulls the amd64 versions, which won't run on say NT 3.51 or 4.0.

malxau commented 7 months ago

There’s a ypm –download-stable option for that. It's not part of ysetup just because I can't see how to make the UX clean given it seems like a very different operation.

Once packages are downloaded, ysetup will look in pkg, yori, ypm, and ysetup subdirectories for packages and will use those rather than talk to the Internet. The intention is to allow these to be on a CD or similar. I just disconnected from the Internet now to try it and it still seems to work.