Package distribution method for RISC-V Sail model

jjscheel commented 9 months ago

Technical Group

Golden Model SIG

ratification-pkg

Technical Debt

Technical Liaison

Bill McSpadden

Task Category

SAIL model

Ratification Target

3Q2023

Statement of Work (SOW)

Component names: Sail

Requirements: Currently, the RISC-V Sail model must be built (compiled) by the user. It has been requested that the model be compiled and distributed using industry-standard package management tools.

Deliverables:

A build a distribution method for the Sail model. Must include yum, apt and a package management for Windows and MacOS.
Documentation for the method.

Acceptance Criteria:

Accepted PRs into the Sail repo for all workflow activities
Packages that build, install on the target OSs, and pass the ACT test.

Projected timeframe: (best guess date) 6 person months

SOW Signoffs: (delete those not needed)

Development partner sign-off
Golden Model SIG sign-off

Waiver

[ ] Freeze
[ ] Ratification

Pull Request Details

No response

lazyparser commented 9 months ago

I am interested in accepting this task, and I think PLCT Lab's technical background can help. Since this week and next week are the Chinese New Year, I estimate that I can return some research results and plans around February 28 at the earliest.

jjscheel commented 9 months ago

@lazyparser and PLCT have expressed interest. I've also asked Greg Sterling to engage here too. I'll get him added to the issue as soon as I can.

U2FsdGVkX1 commented 7 months ago

Fedora has already built all the dependencies and tested please add this repo for testing https://mirror.iscas.ac.cn/plct/distros/fedora/

jjscheel commented 7 months ago

Thanks for the information. @gsterlin, please take a look at this.

yuzibo commented 7 months ago

There are Debian packages about sail/sail-riscv and their dependent packages. Source code is here.

Meanwhile, ocaml-linenoise, omd and lem has been pushed Debian salsa repo under Debian OCaml team and was waiting to sponsor them. Other packages are doing similar things.

jrtc27 commented 7 months ago

Given how rapidly Sail is being modified at the moment to support new features for the model I’m not convinced having packages in Debian stable is at all useful. It seems highly likely that by the time they are released they won’t be usable.

yuzibo commented 7 months ago

Make sense.

If we want to use a fresh deb/rpm package, we would better to keep one final package, in other words, to keep sail-riscv package with an upstream distribution. But before this, these dependent packages should be packaged into distribution to reduce the maintenance burden.

jjscheel commented 7 months ago

@jrtc27, valid point.

@yuzibo, I agree that building packages now helps ensure that dependencies are available when Sail stabilizes.

gsterlin commented 7 months ago

My belief is that even though we may be iterating over packages faster than we'd like, it would also give us some ability to talk across differing distributions to discuss what versions of software, and specifically which packages were used, that gave specific results. This should help us consider branching strategies for the models, and being able to provide specifications about what went into the model. My hope is that this would give us a more common framework from which we could discuss changes, and test them out in more consistent ways.

It is also possible that I need to spend more time understanding how things work and come together for sail, and I am open to learning more about how things work today, especially where we see issues or possible room for improvement.

yuzibo commented 7 months ago

linksem has been packaged and waited for lem uploaded first.

yuzibo commented 7 months ago

bisect-ppx has been packaged to the Debian OCaml namespace and uploaded by a Debian Developer.

U2FsdGVkX1 commented 7 months ago

sail-riscv on Fedora docker image has been built. by the way, we will upstream dependencies and sail as soon as possible, sorry for the delay

jjscheel commented 7 months ago

@gsterlin, would you kindly collect all the progress and summarize this? I suspect that there's benefit in knowing what pkgs are in which ecosystem. Then, we can tie this into the whole broader plan. Thx.

U2FsdGVkX1 commented 6 months ago

https://github.com/orgs/fedora-sail/repositories

The spec files for lem linksem ocaml-omd ocaml-linenoise ocaml-pprint have been pushed They are dependencies of sail-riscv, but there is another part that is still in process Now wait for the upstream

yuzibo commented 6 months ago

On Debian side update: ocaml-linenoise has been uploaded to Debian official archive.

So, until now. bisect-ppx has been uploaded to Debian NEW queue and ocaml-linenoise are already in Debian official. We still have omd, lem and linksem need to be uploaded to Debian for sail/riscv-sail.

gsterlin commented 6 months ago

Here is a breakdown of the current packages for the distributions discussed in this thread:

yuzibo commented 6 months ago

Debian： Still waiting for omd to be reviewed/uploaded. No progress on the other packages now.

U2FsdGVkX1 commented 6 months ago

Fedora: all dependencies have been pushed, and is waiting for review.

gsterlin commented 5 months ago

I have been working on a tool with Bill M. to help compare environments. The intent is to report on/verify if an environment has what is believed to be the necessary settings and software in place to function properly for a given repository. I have catalogued roughly 10 environment variables that indicate if an environment has been setup properly, and roughly 10 versions of software or modules that appear to be necessary to build sail-riscv.

I am working to wrap up the scripting over the next week or so, and removing hard coded values and transforming them into a json file so that we can make adjustments for different repositories and branches as necessary.

Over the next week or two I believe I will have the code in a state worth reviewing, and will then begin working with Bill M. to determine where best to begin working with teams for this kind of environment validation.

yuzibo commented 5 months ago

@gsterlin thanks for your work, please tell me if I can help here.

Update on Debian side: omd was sponsored by my sponsor.:)

U2FsdGVkX1 commented 5 months ago

Sorry for the late reply Fedora: No updates yet, still under review

yuzibo commented 5 months ago

Update on Debian side: omd was uploaded to Debian archives and lem was waiting to be reviewed.

U2FsdGVkX1 commented 5 months ago

Fedora: Review Request has been submitted for lem linksem ott ocaml-omd ocaml-linenoise ocaml-pprint https://bugzilla.redhat.com/show_bug.cgi?id=2291284 https://bugzilla.redhat.com/show_bug.cgi?id=2291285 https://bugzilla.redhat.com/show_bug.cgi?id=2291286 https://bugzilla.redhat.com/show_bug.cgi?id=2291288 https://bugzilla.redhat.com/show_bug.cgi?id=2291289 https://bugzilla.redhat.com/show_bug.cgi?id=2291290

jjscheel commented 5 months ago

@gsterlin, perhaps when you run the June 25 meeting for me you could talk more about your plans here. Thoughts?

yuzibo commented 4 months ago

Debian: lem was reviewing for the past two weeks.

U2FsdGVkX1 commented 4 months ago

Fedora package still waiting for review……

yuzibo commented 4 months ago

Debian: linksem was uploaded to salsa under the Debian OCaml team to wait to be reviewed. lem was still reviewing.

U2FsdGVkX1 commented 4 months ago

ocaml-omd ocaml-linenoise ocaml-pprint It has been reviewed. I have pushed the new version and waiting for another review.

yuzibo commented 3 months ago

Debian: lem has been uploaded to FTP NEW queue.

U2FsdGVkX1 commented 3 months ago

ocaml-pprint has been reviewed and approved ocaml-linenoise has some issues and cannot meet Fedora packaging rules. I have already pull request to the upstream ocaml-omd is awaiting further review

jjscheel commented 3 months ago

@gsterlin, would you mind sharing your progress for the next meeting on August 6?

gsterlin commented 3 months ago

Absolutely! A quick note is that I have been working with Carl to leverage some tools (asdf/mise, and direnv) to do the environment variable/configuration and tool version management. This replaces the code and configuration files I had written, greatly simplifying the problem. We have made a lot of progress over the last two weeks and will have some content to share shortly that will help new comers work with our environments, and provide the building blocks for enabling discussions about which versions of packages or environment variables are necessary for proper environments per repository.

yuzibo commented 3 months ago

Debian packaging: FTP master has commented on lem and waiting for review again.

U2FsdGVkX1 commented 3 months ago

ocaml-omd was also approved! Other packages waiting for upstream response

yuzibo commented 2 months ago

Debian: lem was rejected due to d/copyright issue and uploaded to NEW waiting for review again.

yuzibo commented 2 months ago

Debian packaging: no update in the past two weeks, still waiting for lem to be approved to upload.

U2FsdGVkX1 commented 2 months ago

fedora: No updates in the past two weeks No response to pull requests for ocaml-linenoise, maybe we should consider other methods…

jjscheel commented 2 months ago

@gsterlin, I wonder if Wei Fu can provide some guidance on the ocaml-linenoise PR.

U2FsdGVkX1 commented 2 months ago

Wei Fu is my mentor, but I think we should contact the maintainer of ocaml-linenoise (or Fedora packager)

U2FsdGVkX1 commented 2 months ago

Sorry, my English may cause some misunderstand, let me clarify "consider other methods" means that ocaml-linenoise has made some changes, and these changes have not been upstreamed to linenoise, so they behave inconsistently, and the fedora's reviewer hopes to be able to upstream these changes, so "other methods" should be Contact the maintainer of linenoise to accept this PR or Explain to the fedora's reviewer why there are differences between ocaml-linenoise and linenoise, and then accept this package

yuzibo commented 1 month ago

Debian packaging: lem was uploaded to Debian official archives already. linksem and sail is packaging again.

U2FsdGVkX1 commented 1 month ago

Since there has been no response from the upstream of linenoise, I sent the patch to the downstream of Fedora, but I think Fedora may not accept this patch (because Fedora requires the upstream to accept the patch first). However, without this patch, linenoise and ocaml-linenoise will behave differently, which will cause ocaml-linenoise to not meet the packaging requirements :(

U2FsdGVkX1 commented 1 month ago

Good news! Fedora, as the downstream, has already merged this change!

yuzibo commented 1 month ago

Debian packaging: linksem is packaging and it is waiting for lem's one commit.

Summary: Now we only have linksem and sail left as dependencies of riscv-sail that need to be uploaded to Debian.

U2FsdGVkX1 commented 1 month ago

Since the linenoise patch was merged by Fedora, ocaml-linenoise is being reviewed again

yuzibo commented 4 weeks ago

Debian packaging: linksem is being reviewed by my sponsor.

U2FsdGVkX1 commented 4 weeks ago

The reviewer has not responded yet maybe I need to ask Wei Fu for help

U2FsdGVkX1 commented 2 weeks ago

There is a small change in the linenoise package on fedora. Due to my mistake in the past, the symbols were not exported. Fortunately, the linenoise package on fedora has been merged https://src.fedoraproject.org/rpms/linenoise/pull-request/3

ocaml-linenoise is waiting for review again

yuzibo commented 2 weeks ago

Debian packaging update: linksem is waiting to be uploaded still.

jjscheel commented 2 weeks ago

Thanks, folks!

riscv-admin / dev-partners