search

conda-forge / miniforge

A conda-forge distribution.
https://conda-forge.org/miniforge
Other
6k stars 317 forks source link

MD5sum claimed inside Mambaforge-pypy3-MacOSX-x86_64.sh doesn't match md5sum of the file #418

Open rsharris opened 1 year ago

rsharris commented 1 year ago

Solution to issue cannot be found in the documentation.

Issue

Downloaded Mambaforge-pypy3-MacOSX-x86_64.sh by clicking the link in this repo's README.

Inside the file, it lists an MD5. This does not match the md5 that the file actually has.

Is this the expected behavior?

I realize there's a catch-22 there — how could you insert the md5 into the file without that changing the actual md5. So I guess my real question is just what is that the md5 of?

Installed packages

(not relevant)

Environment info

(not relevant)
dbast commented 1 year ago

The md5 is not the md5 of the entire installer (due to the reason you figured out yourself). The .sh installers are a shell script with binary payload at the end. The md5 is only the checksum of that payload. See e.g. the header template https://github.com/conda/constructor/blob/main/constructor/header.sh (search for MD5).

rsharris commented 1 year ago

I see, thanks.

Upon further inspection, I see that the install .sh file does validate that. I will observe, though, that this doesn't seem to provide much (if any) security. Anyone who could change the payload could also just change the expected MD5, right?

In any case, my original concern has been addressed, thanks.

dbast commented 1 year ago

The md5 is only there to protect against download errors in the payload part... It cannot provide any security as it does not cover the entire file. You can download the according Mambaforge-pypy3-MacOSX-x86_64.sh.sha256 file and check via shasum -a 256 Mambaforge-pypy3-MacOSX-x86_64.sh that the content matches and thus the installer hasn't changed.

rsharris commented 1 year ago

Ah, excellent.

Since I originally downloaded Mambaforge-pypy3-MacOSX-x86_64.sh by clicking the link in this repo's README, it wasn't immediately obvious where that .sha256 file lives. With a little digging I did find it, though, thanks.

For the sake of anyone reading this thread in the future, that .sha256 file can be found among the releases, at github.com/conda-forge/miniforge/releases .

rsharris commented 1 year ago

Actually, the user does have to dig further to get the right .sha256 file.

You need to know the release number, and if you downloaded from the link in the readme, you don't know what that is. You can find it in the header of the downloaded .sh file. In my case it is 22.9.0-3.

Then the file you are looking for is Mambaforge-pypy3-{version}-MacOSX-x86_64.sh.sha256. I did find that among the files at github.com/conda-forge/miniforge/releases , and it matches the the sha256 that I computed locally on the file I downloaded, Mambaforge-pypy3-MacOSX-x86_64.sh .

@dbast Sorry to make such a mountain out of this molehill, and thanks for your help.

hmaarrfk commented 1 year ago

Can you adapt the instructions in https://github.com/conda-forge/miniforge#unix-like-platforms

hmaarrfk commented 1 year ago

if you are interested in getting a specific release, you can do so using github's releases. They have a predictable pattern, but maybe our naming hasn't been very consistent.

rsharris commented 1 year ago

@hmaarrfk I wasn't interested in a specific release, per se. I got the "latest installer with Mamba and PyPy in the base environment" for my architecture, using the link under https://github.com/conda-forge/miniforge#mambaforge-pypy3 . The version number is not advertised there, nor is it in the filename. That's fine — there are good reasons for that.

But then I wanted to verify the .sh file I had downloaded. At that point, as near as I can tell, one needs to know the version number, the predictable pattern, and that the .sha256 file even exists. I wasn't able to locate the .sha256 file by a filename that didn't include the version number. Maybe it exists, but I didn't find it.

I don't see how the https://github.com/conda-forge/miniforge#unix-like-platforms instructions would help for that. Unless you're saying that .../releases/latest/download/Mambaforge-$(uname)-$(uname -m).sh.sha256 will exist there.

I should make it clear that I'm not really complaining here.

hmaarrfk commented 1 year ago

It is somewhat questionable if getting the sha from the same source as the download is safe.

If you do it in an automated way, then the same person that edits the payload can edit the sha256.

This is why, as I understand it, we did not include a sha for the bash file without the version in it. See the artifacts in our releases page https://github.com/conda-forge/miniforge/releases

Adapting the command referenced, you can you can however do it for specific versions:

VERSION=22.11.1-2
wget "https://github.com/conda-forge/miniforge/releases/download/${VERSION}/Mambaforge-${VERSION}-$(uname)-$(uname -m).sh"
wget "https://github.com/conda-forge/miniforge/releases/download/${VERSION}/Mambaforge-${VERSION}-$(uname)-$(uname -m).sh.sha256"

It seems we were better than I thought with our tag naming convention https://github.com/conda-forge/miniforge/tags

void-rooster commented 1 year ago

+1 on adding verification instructions to the download page.