Discuss options and details around Adoptium Marketplace

[smlambert]

Based on some conversation both in working group, transition meetings and the Adoptium community call, let's put down some additional details about an Adoptium marketplace and start to flush out implementation details and challenges.

What is a marketplace?

• a place where member distributors can market their binaries, essentially a centralized location on a website
• TBD - determine if marketplace extends into serving up Docker images via a 'marketplace' Dockerhub repo

Criteria?

• member of Adoptium WG
• binaries pass compliance (TCKed by vendors)
• binaries pass AQA suite

How can these be served up?

• based on what we do today, the API can serve up binaries that sit in a Github releases repo, least friction path would be to
  • ask members to push their binaries to a Github releases repo
  • our API can pick up tarballs from the releases repo and serve it up directly and (optionally) feed it into an installers pipeline to produce installers to be served up via the marketplace
  • if Docker images are also considered as part of the marketplace, our tooling can pull tarballs from releases repos to generate various images (and push to Dockerhub)

It may be useful to look at the diagram from https://github.com/AdoptOpenJDK/TSC/issues/158#issuecomment-641708282 where the JDK production swim lane is owned by the member who in the publish stage pushes to a releases repo.

Then the installers and docker images swim lane could (potentially) be a mechanism of the marketplace rather than expect members to each deliver unique solutions for these artifacts.

[aahlenst]

a place where member distributors can market their binaries, essentially a centralized location on a website

We should document what vendors can expect because that also influences the design of the website. I assume that the more we treat the vendor's binaries like Temurin, the more attractive the marketplace will be.

Criteria?

:+1:

(And: no bickering like "our binary is better than yours" :smirk: )

How can these be served up?

That's a great proposal. I like to expand it with some details (my proposal, some parts might be controversial, therefore up for discussion):

• For every version produced by every marketplace member, a repository (naming pattern: <productname>-<version>-binaries, for example temurin-11-binaries or dragonwell-8-binaries) is created in the Adoptium GitHub org where the vendor can upload their binaries to [vendor neutrality is ensured, no problems because we have to obtain binaries from somewhere].
  • Vendors upload tarballs or zip files (depending on the platform), each one accompanied by a sha256sum file that's signed with the vendor's key. Ideally, as part of the process, we verify the signature and add our own [to make it easier for consumers to trust the files].
• All downstream artifacts (installers, packages, container images) are created from the vendor's binaries on Adoptium properties using Adoptium tooling and are (re-)signed using Adoptium's certificates. Vendors can push their artwork, licenses into the respective Adoptium repos. [This ensures consistency, allows us to fix problems without having to chase vendors, reduces problems because of Defender having to trust certificates, varying expiration dates, ... but yeah, it's an additional burden on the project.]
• Linux packages are uploaded into the main package archive used by Adoptium. [Consumers don't have to configure a second repository, no additional keys need to be configured, rotated or trusted.]
• Container images are uploaded to Docker Hub into the Adoptium org in a "folder" that's named after the vendor. Results in image names like adoptium/dragonwell:11 or `adoptium/temurin:11.

The Linux packages are already being prepared to work like that. MSI (Windows), PKG (macOS) would need separate jobs that aren't embedded into the build pipeline. Notarization on macOS might need some attention because resigning might break the jmods.

[tellison]

I will add this topic to the Adoptium WG agenda for discussion

[karianna]

CC @johnoliver as he starts to think about API design

[smlambert]

some notes from working group call on Thursday March 25th:

• must be member of Adoptium working group to list in the marketplace
• products listed in the marketplace need to be compliant (pass TCKs) as tested and self-declared via each members OCTLA
• products listed in marketplace pass the AQA quality bar (as defined in the AQAvit project) / while this is an open/transparent quality assurance program, that anyone can run and make factual statements about (our distro runs/passes/shares open test results), only members of Adoptium working group will get to associate the quality mark against their products. There is an analog of this, Jakarta EE TCKs (public, and anyone can make factual statements based on their results, but only WG members may be listed on the Jakarta compatible products page)
• discussion about preference for linking out to member downloads, rather than the proposal to standardize installers/docker images
• asked members for input to this issue, to ensure we understand everyone's perspective to arrive at the right balance and solution

[smlambert]

Notes from Thursday, April 1st WG call:

• Divide this marketplace into several categories, 3 main ones:
  • sitemap and how website should look, form for help from EF creative team is now filled out, sitemap is high priority, to get something to both student team and designers
  • AQA criteria - define what is required to have the AQA mark against an entry in the marketplace (see AQA definition document for a starting point)
  • TCK is self-certification by vendors

[smlambert]

Additional note raised in Community call regarding APIv4 planning:

• question to how the APIv4 will support marketplace, can/should we advise members to store binaries in github releases, so the API can serve them up easily?

[karianna]

Further to this @johnoliver believes the new API at Adoptium could map an Adoptium API request to the vendor API or implement a binary discovery mechanism in the case where there is no API available. There would not necessarily be a need to ask vendors to store their binaries in GitHub releases.

[karianna]

Another option is to back onto foojay.io's API (pros and cons to that).

[tellison]

We have to let vendors serve up their own binaries. Adoptium is just one of the vendors. So conceptually there is the current role and a new role for our API when moved to Adoptium:

• the current API role, much as it exists today (assets|binaries|info|etc), remains the way to get details about and download Temurin builds. The jvm_impl becomes a lot less interesting, as it will only return useful data for hotspot. For backwards compatibility I'd be inclined to keep the parameter, and have it return the empty set for openj9;

  • the market API role, that returns info about Adoptium Working Group Members’ binaries, including the location where the binary can be downloaded (hosted by the vendor/outside Adoptium). Adoptium won't modify or distribute those binaries directly. This part of the API returning metadata is the equivalent to loading a webpage containing a link to the vendor's download. It should report if it is a direct download or not.

The "market API" is likely a new path through the same api.adoptium.net - that’s an implementation detail :-) But it should be clear when you are asking for information about the available binaries listed in the vendor marketplace, and when you are asking about info for Temurin binaries. As such, I would suggest not trying to update all the existing API paths to accommodate other vendors' information. Just populate a new marketplace catalog and include Temurin in there.

[karianna]

I think we're going to need some concrete examples in the form of HTTP Get requests to ensure we are walking the correct line in each case.

[gdams]

closing as this has now moved to https://github.com/adoptium/adoptium/issues/7