search

harmony-one / bounties

Bounty program is to help the community take part in the development of the Harmony blockchain. It covers from core feature to validator tooling, from dApp development to DeFi integration.
MIT License
59 stars 23 forks source link

CentOS/RHEL/AL2: Support installation for Harmony #71

Closed jacksteroo closed 2 years ago

jacksteroo commented 2 years ago

Description

Package and distribute the Harmony binaries for Linux's CentOS/RHEL/AL2 distribution. For example, CentOS's yum install harmony-one should initiate installation of Harmony binaries alongside all the dependencies. Also allow the installation to be pinned to a certain version.

Context

This encourages ease of development and testing, and further helps with decentralizing the ecosystem. Review the guidelines for RHEL and CentOS for further context. This should also work for the AWS Amazon Linux 2 (AL2) distribution.

Acceptance Criteria

Reward

USD $500 (in ONEs)

gitcoinbot commented 2 years ago

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

This issue now has a funding of 3426.2931 ONE (502.66 USD @ $0.15/ONE) attached to it.

gitcoinbot commented 2 years ago

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

Work has been started.

These users each claimed they can complete the work by 265 years, 1 month from now. Please review their action plans below:

1) 0xcarnyx has started work.

Learn more on the Gitcoin Issue Details page.

gitcoinbot commented 2 years ago

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

Work for 3426.2931 ONE (1138.53 USD @ $0.33/ONE) has been submitted by:

ranmses commented 2 years ago

Hello @LeoHChen @jacksteroo I've submitted work through the gitcoin bounty page. Is there an ETA to review it? Thanks.

omhmichaels commented 2 years ago

Hello @LeoHChen @jacksteroo I've submitted work through the gitcoin bounty page. Is there an ETA to review it? Thanks.

Hey @Ranmses, Im reviewing your submission right now and will get back to you with more info shortly after testing shortly with an update. Thanks for the fantastic submission and waiting harmoniously.

💙

omhmichaels commented 2 years ago

Also can you telegram me @omhmichaels . Posted this on gitcoin yesterday but didn't add on here.

ranmses commented 2 years ago

Thanks for following up, @omhmichaels. Please let me know if you need anything.

omhmichaels commented 2 years ago

Thanks for following up, @omhmichaels. Please let me know if you need anything.

Currently testing the repo in amazonlinux and centos containers. Ill have more for you within an hour or two.

The following is some quick thoughts from my personal notes in more of a general brainstorm a format if your interested. Ill ask add more structured feedback in a pull request to your repository.

What version of Centos, Amazon Linux, and RHEL was this tested on?

Getting the repository

Need of deployment infrastructure? Options to make available to public ? ETA / Steps to get package approved in yum official repository? Hosting of repository?

Outline of procedures being ran in prebuild.sh or basics of building yum package resource.

omhmichaels commented 2 years ago

@ranmses question, in your README.md you say assumptions where made? What were those?

ranmses commented 2 years ago

Hey @omhmichaels

What version of Centos, Amazon Linux, and RHEL was this tested on?

Tested with clients running RHEL 8.4, CentOS 8.4 and Amazon Linux 2. I have built the RPMs on a server running RHEL 8.4 and go 1.17.2.

Need of deployment infrastructure? Options to make available to public ? ETA / Steps to get package approved in yum official repository? Hosting of repository?

Most open source projects with active development hosts yum repositories themselves, I would advise to do so. Something like what you already have for testing.

This approach gives you full control over news releases, and enables you to simplify things, e.g., instead of having to publish new packages to every official RHEL-like distro/major release you want to support (RHEL7/8, CentOS7/8, AL2, etc.), you can just have options based on major releases like 7/8/9.

In this development I simplified even more, doing everything in only one build/repo (which may not be possible in the future as explained here). A middle ground option would be to get the packages into EPEL, which already applies this major-release approach. I'm not familiar with their approval process, but here's what I found about it.

But then again, having to comply with additional rules may slowdown new releases, which may not be ideal.

Outline of procedures being ran in prebuild.sh or basics of building yum package resource.

Shellscript pre rpmbuild line by line:

  1. Setup shebang and ask for Harmony version as first script input (lines 1,2)
  2. Install RPM packages needed for RPM build (3,4)
  3. Create RPM structure directory tree (5)
  4. Create/clean-up directory structure for go packages inside the tree (6-10)
  5. Clone source repos (11-15)
  6. Get go version requirement and version release from code (16-19)
  7. Create the spec file inside the tree by making a copy of the spec template (20-22)
  8. Inject Harmony, release and go versions onto the spec file (23-25)
  9. Create the source code tar file inside the tree (26,27)
  10. Clean up temporary files and exit (28,29)

Summing it all up, install needed packages and create: RPM build tree, spec file, source code tar file. We need those three resources to be properly set up in order to build the RPM package.

With everything in place now we can just rpmbuild --bb <spec file> and the RPM will be created at rpmbuild/RPMS/x86_64 directory. The building itself is very straightforward, but generating those three resources can be as hard as possible since it needs to harmoniously link many requirements such as of code language, rpmbuild version, OS distro, etc.

in your README.md you say assumptions where made? What were those?

Well, I had to make decisions about some questions. A couple of them:

Should go be installed during build? Or can I suppose it was already on the build system? About the go version, should it be one from distro repo or can it be from source? I supposed it was already installed, and from source. Then I make some checks during build.

Should I clone source during build? Or can I suppose it was already on the build system? Can I always use the latest versions of mcl and bls? I decided to clone them here, getting the Harmony specific branch/version and the latest mcl/bls.

Which source code commit should I use for setting up the RPM release? For a given Harmony version, how do I know if a commit is ready for release? I decided to use the command git describe --long here, after cloning the branch for that specific Harmony version.

omhmichaels commented 2 years ago

Hey @omhmichaels

What version of Centos, Amazon Linux, and RHEL was this tested on?

Tested with clients running RHEL 8.4, CentOS 8.4 and Amazon Linux 2. I have built the RPMs on a server running RHEL 8.4 and go 1.17.2.

Need of deployment infrastructure? Options to make available to public ? ETA / Steps to get package approved in yum official repository? Hosting of repository?

Most open source projects with active development hosts yum repositories themselves, I would advise to do so. Something like what you already have for testing.

This approach gives you full control over news releases, and enables you to simplify things, e.g., instead of having to publish new packages to every official RHEL-like distro/major release you want to support (RHEL7/8, CentOS7/8, AL2, etc.), you can just have options based on major releases like 7/8/9.

In this development I simplified even more, doing everything in only one build/repo (which may not be possible in the future as explained here). A middle ground option would be to get the packages into EPEL, which already applies this major-release approach. I'm not familiar with their approval process, but here's what I found about it.

But then again, having to comply with additional rules may slowdown new releases, which may not be ideal.

Outline of procedures being ran in prebuild.sh or basics of building yum package resource.

Shellscript pre rpmbuild line by line:

  1. Setup shebang and ask for Harmony version as first script input (lines 1,2)
  2. Install RPM packages needed for RPM build (3,4)
  3. Create RPM structure directory tree (5)
  4. Create/clean-up directory structure for go packages inside the tree (6-10)
  5. Clone source repos (11-15)
  6. Get go version requirement and version release from code (16-19)
  7. Create the spec file inside the tree by making a copy of the spec template (20-22)
  8. Inject Harmony, release and go versions onto the spec file (23-25)
  9. Create the source code tar file inside the tree (26,27)
  10. Clean up temporary files and exit (28,29)

Summing it all up, install needed packages and create: RPM build tree, spec file, source code tar file. We need those three resources to be properly set up in order to build the RPM package.

With everything in place now we can just rpmbuild --bb <spec file> and the RPM will be created at rpmbuild/RPMS/x86_64 directory. The building itself is very straightforward, but generating those three resources can be as hard as possible since it needs to harmoniously link many requirements such as of code language, rpmbuild version, OS distro, etc.

in your README.md you say assumptions where made? What were those?

Well, I had to make decisions about some questions. A couple of them:

Should go be installed during build? Or can I suppose it was already on the build system? About the go version, should it be one from distro repo or can it be from source? I supposed it was already installed, and from source. Then I make some checks during build.

Should I clone source during build? Or can I suppose it was already on the build system? Can I always use the latest versions of mcl and bls? I decided to clone them here, getting the Harmony specific branch/version and the latest mcl/bls.

Which source code commit should I use for setting up the RPM release? For a given Harmony version, how do I know if a commit is ready for release? I decided to use the command git describe --long here, after cloning the branch for that specific Harmony version.

Thank you so much for this. Helped a lot in gaining a quick understanding without having to read all the docs from RHEL. I added my tests output here https://github.com/omhmichaels/gc-b71/blob/master/debug.md along with the Dockerfiles and steps to build for the containers I used during tests.

omhmichaels commented 2 years ago

Also your questions at the bottom are something I had thought of as well when working on the Homebrew bounty. Particullarly the part about the versioning/ which commit to use. What i would like to see is corresponding versioned releases of MCL and BLS along with each official release harmony release.

ranmses commented 2 years ago

Hello,

I have updated the work with fixes and added a new section containing details about VM images and instructions for building/testing on containers.

The bounty's description did not mention container testing, but I have decided to accommodate the need.

@omhmichaels you now have permission to merge.

LeoHChen commented 2 years ago

hi, @ranmses Thanks for all the update. Very decent work. Sorry for the delay due to the holiday break. Let me close this bounty. I think it is good enough for now.

LeoHChen commented 2 years ago

payout will be send in this week

gitcoinbot commented 2 years ago

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

The funding of 3426.2931 ONE (1139.34 USD @ $0.31/ONE) attached to this issue has been approved & issued to @ranmses.