Do not rely on Github archive checksums

[leto]

Instead, we should have our own Github fork of all packages and upload a Github Release with static tarball + checksum.

$ grep 'github.*archive' depends/packages/* 
depends/packages/crate_blake2_rfc.mk:$(package)_download_path=https://github.com/gtank/$($(package)_crate_name)/archive/
depends/packages/crate_sapling_crypto.mk:$(package)_download_path=https://github.com/zcash-hackworks/$($(package)_crate_name)/archive/
depends/packages/crate_zip32.mk:$(package)_download_path=https://github.com/zcash-hackworks/$($(package)_crate_name)/archive/
depends/packages/googlemock.mk:# url=https://github.com/google/googlemock/archive/release-1.7.0.tar.gz
depends/packages/googlemock.mk:$(package)_download_path=https://github.com/google/$(package)/archive
depends/packages/googletest.mk:$(package)_download_path=https://github.com/google/$(package)/archive
depends/packages/libevent.mk:$(package)_download_path=https://github.com/libevent/libevent/archive
depends/packages/libgmp.mk:$(package)_download_path=https://github.com/joshuayabut/$(package)/archive
depends/packages/libgmp.mk:$(package)_download_path=https://github.com/ca333/$(package)/archive
depends/packages/librustzcash.mk:$(package)_download_path=https://github.com/zcash/$(package)/archive/
depends/packages/utfcpp.mk:$(package)_download_path=https://github.com/MyHush/$(package)/archive/
depends/packages/zeromq.mk:$(package)_download_path=https://github.com/ca333/libzmq/archive

Background info: https://github.com/zcash/zcash/issues/2610

https://github.com/libgit2/libgit2/issues/4343#issuecomment-328631745

[oDinZu]

I don't know if there is an easier way of doing this, I am manually going to main distributor or source. Along the way of the search, I am having a tough time finding zcash crates; would you have any options on finding crates:

• sapling_crypto,
• blake2_rfc,
• zip32?

Also, the googlemock and googletest.mk are in one package now called googletest. All other packages have been found.

https://github.com/zcash/zcash/issues/3786 Is there a command or something to check packages with local version to most recent version?

Edit:

It seems the creates zip32 and sapling_crypto are not used any more according to @str4d #3870 : https://github.com/zcash/zcash/issues/3786#issuecomment-579477972

Plus, a python script to check versions!?

[leto]

@csharpee if you can't find a crate publicly it's likely that is because it only lives inside the zcash.git or librustzcash.git repos (most Rust crates just moved from librustzcash to zcash git repo).

This ticket is just to track the dependencies I listed above, not zcash crates. That stuff changes too fast and by the time you are done things will be different. Don't touch that stuff for now.

[oDinZu]

Okay.

What you are looking for is these dependencies need to be hosted from MyHush repository?

So download, sha256sum, gpg --verify, upload to GitHub, then fork to MyHush repo, next update package download path?

As an example (after packages been verified from source and uploaded to GitHub),

$(package)_version=4.3.1
$(package)_download_path=https://github.com/MyHush/packagesExample/libzmq
$(package)_download_file=v$($(package)_version).tar.gz
$(package)_file_name=libzmq-$($(package)_version).tar.gz
$(package)_sha256_hash=cb8ebe5b60dadeb526745610d6237f05a98aba287114d8991dad1fa14f4be354

From here, we will manage the packages and dependencies locally via our repository.

Directory would look like:

• MyHush
• • PackagesExample
• • • Package 1
• • • Package 2 ...etc

The PackagesExample could then be a tar.gz with its own checksum and each package would also have its own checksum..

This is what I am comprehending, please let me know if this is what you are looking for.

[leto]

@csharpee there is no gpg involved, just checksums.

You can basically emulate what I did in depends/packages/libsodium.mk and utfcpp.mk as well. Do a git log -p on those files to see how they have changed recently.

Your $(package)_download_path above is incorrect, make sure it has the structure like in libsodium/utfcpp. To verify it works, the simplest thing is to do a fresh clone with no dependencies downloaded yet and then do a fresh build.

I suggest trying it for one single package, test it and then make a PR, which I can review.

The manual part of this is that I may need to help you create a repository as well as upload a release for it.

Thanks! :+1:

[oDinZu]

Ahh, okay beautiful; I see, each package is forked individually to MyHush, rather than a "PackagesExample" directory.

Thanks for the guidance Duke :) I am working on this now. I will create a repository on csharpee github, then upload the file and update deps.mk and do a clean clone and build to verify.