search

AdoptOpenJDK / openjdk-api

DEPRECATED and replaced by of https://github.com/AdoptOpenJDK/openjdk-api-v3 - See README
https://api.adoptopenjdk.net/README
Apache License 2.0
15 stars 15 forks source link

Versions are inconsistent/incomplete/misplaced #132

Closed bdw429s closed 4 years ago

bdw429s commented 5 years ago

There is currently no good way to get a standard version number out of the API which makes it very difficult to interact with the API.

There is a binaries[ i ].version property in the JSON response but it has two issues:

This leaves us with the only remaining way to get the full version for a given release, which is the release_name. However, the release names are all over the place for different versions of OpenJDK which makes it difficult to do any sort of automated interaction with your API that needs to seamlessly work with any version of OpenJDK. Here are some examples:

I get that Adopt is just copying a lot of these things from upstream but this needs to be improved. There's way too much information crammed into this property and it's far to inconsistent to parse.

The Adopt API needs to pick a single, consistent format to represent the version of a release which holds true for all artifacts, regardless of what random format OpenJDK used. An example would be semantic versioning.

My suggestion is as follows:

As an example, the release named jdk-9.0.4+11 could potentially have a release_version of just 9.0.4+11 and a release named jdk8u162-b12_openj9-0.8.0 could potentially have a release_version of just 8.0.162+12.

bdw429s commented 5 years ago

Reminder to myself: Once this ticket is complete, enter a ticket for the API to accept versions (and possibly partial versions) for filtering. i.e.

/info/releases/openjdk9?version=9.0.4%2B11 or maybe ranges like /info/releases/openjdk9?version=9.0.x

johnoliver commented 5 years ago

@bdw429s how do you feel about the following:

[
  {
    "os": "linux",
    "architecture": "x64",
    "binary_type": "jdk",
    "openjdk_impl": "hotspot",
    "binary_name": "OpenJDK11U-jdk_x64_linux_hotspot_11.0.1_13.tar.gz",
    "binary_link": "https://github.com/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11.0.1%2B13/OpenJDK11U-jdk_x64_linux_hotspot_11.0.1_13.tar.gz",
    "binary_size": 195648421,
    "checksum_link": "https://github.com/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11.0.1%2B13/OpenJDK11U-jdk_x64_linux_hotspot_11.0.1_13.tar.gz.sha256.txt",
    "version": "11",
    "version_data": {
      "openjdk_version": "11.0.1+13",
      "semver": "11.0.1.13"
    },
    "heap_size": "normal",
    "download_count": 26729,
    "updated_at": "2018-11-15T19:25:13Z",
    "timestamp": "2018-11-15T19:25:13Z",
    "release_name": "jdk-11.0.1+13",
    "release_link": "https://github.com/AdoptOpenJDK/openjdk11-binaries/releases/tag/jdk-11.0.1%2B13"
  },
  {
    "os": "linux",
    "architecture": "x64",
    "binary_type": "jre",
    "openjdk_impl": "hotspot",
    "binary_name": "OpenJDK11-jre_x64_linux_hotspot_11_28.tar.gz",
    "binary_link": "https://github.com/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11%2B28/OpenJDK11-jre_x64_linux_hotspot_11_28.tar.gz",
    "binary_size": 41597311,
    "checksum_link": "https://github.com/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11%2B28/OpenJDK11-jre_x64_linux_hotspot_11_28.tar.gz.sha256.txt",
    "version": "11",
    "version_data": {
      "openjdk_version": "11+28",
      "semver": "11.0.0.28"
    },
    "heap_size": "normal",
    "download_count": 789,
    "updated_at": "2018-10-03T05:13:01Z",
    "timestamp": "2018-10-03T05:13:01Z",
    "release_name": "jdk-11+28",
    "release_link": "https://github.com/AdoptOpenJDK/openjdk11-binaries/releases/tag/jdk-11%2B28"
  }
]

note version_data added. Some examples of the form version_data can take:

version_data: {
  "openjdk_version": "11.2.1+10",
  "semver": "11.2.1.10"
}
version_data: {
  "openjdk_version": "9-ea+19",
  "semver": "9.0.0.19-ea",
  "preview": "ea"
}
version_data: {
  "openjdk_version": "9+181",
  "semver": "9.0.0.181"
}
version_data: {
  "openjdk_version": "11+28",
  "semver": "11.0.0.28"
}
version_data: {
  "openjdk_version": "10.0.1+10",
  "semver": "10.0.1.10"
}
version_data: {
  "openjdk_version": "9.0.4+12",
  "semver": "9.0.4.12"
}
version_data: {
  "openjdk_version": "9.0.4+12-openj9-0.9.0",
  "semver": "9.0.4.12",
  "optional": "openj9-0.9.0"
}

version_data has the properties: semver - Symantic version conversion of openjdks version openjdk_version - Original openjdk version optional - $OPT from jep223 preview - $PRE from jep223

semver is of the form: <major>.<minor>.<security>, then with optional .<build> and -ea, i.e: 9-ea+19 = 9.0.0.19-ea.

I have left off the optional stuff like openj9-0.9.0 from the semver but it could be included after a + but it would make it a bit messier.

It is possible some may prefer "+ea" for the early access tag, but as per https://semver.org/ hyphen seemed more appropriate.

bdw429s commented 5 years ago

hi @johnoliver , I like where you're going with this, but I'd like to point out that 1.2.3.4 is not valid semver as it only allows for major, minor, and patch with period delimiters. The other two optional pieces of data you can provide in semver are prerelease identifier and build number. Prerelease ID is preceded by a hyphen (-alpha, -beta, -rc, -snapshot, -beta.1, -beta.2, etc) and build number is preceded by a plug sign. i.e. 1.2.3+456.

For in the case of, say, 9.0.0.19-ea the proper semver equivalent would be 9.0.0-ea+19 assuming that 19 was the build metadata (note build comes after prerelease ID). Ironically, the later releases from OpenjDK are actually pretty close to semver, though the spec isn't 100% on whether minor and patch are optional through a lot of implementations like npm's flavor allows that.

To provide a few more examples from release names I mentioned in the ticket:

bdw429s commented 5 years ago

@johnoliver Just checking to see if you saw my reply.

johnoliver commented 5 years ago

I have an example working, you can see some of the example mappings at:

https://github.com/johnoliver/openjdk-api/blob/e7f3456759b7e2d3efe2f1dc41a285f9e8ba3a43/test/versions.test.js

Do those look correct to you?

Some examples of data produced:

...
      {
        "os": "windows",
        "architecture": "x32",
        "binary_type": "jdk",
        "openjdk_impl": "openj9",
        "binary_name": "OpenJDK8U-jdk_x86-32_windows_openj9_8u192b12_openj9-0.11.0.zip",
        "binary_link": "https://github.com/AdoptOpenJDK/openjdk8-binaries/releases/download/jdk8u192-b12_openj9-0.11.0/OpenJDK8U-jdk_x86-32_windows_openj9_8u192b12_openj9-0.11.0.zip",
        "binary_size": 87221947,
        "checksum_link": "https://github.com/AdoptOpenJDK/openjdk8-binaries/releases/download/jdk8u192-b12_openj9-0.11.0/OpenJDK8U-jdk_x86-32_windows_openj9_8u192b12_openj9-0.11.0.zip.sha256.txt",
        "version": "8",
        "version_data": {
          "openjdk_version": "8u192-b12_openj9-0.11.0",
          "semver": "8.0.192+12",
          "optional": "openj9-0.11.0"
        },
        "heap_size": "normal",
        "download_count": 287,
        "updated_at": "2018-11-21T11:54:02Z"
      },
...
...
  {
    "os": "linux",
    "architecture": "x64",
    "binary_type": "jre",
    "openjdk_impl": "hotspot",
    "binary_name": "OpenJDK11-jre_x64_linux_hotspot_11_28.tar.gz",
    "binary_link": "https://github.com/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11%2B28/OpenJDK11-jre_x64_linux_hotspot_11_28.tar.gz",
    "binary_size": 41597311,
    "checksum_link": "https://github.com/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11%2B28/OpenJDK11-jre_x64_linux_hotspot_11_28.tar.gz.sha256.txt",
    "version": "11",
    "version_data": {
      "openjdk_version": "11+28",
      "semver": "11.0.0+28"
    },
    "heap_size": "normal",
    "download_count": 1318,
    "updated_at": "2018-10-03T05:13:01Z",
    "timestamp": "2018-10-03T05:13:01Z",
    "release_name": "jdk-11+28",
    "release_link": "https://github.com/AdoptOpenJDK/openjdk11-binaries/releases/tag/jdk-11%2B28"
  },
...
bdw429s commented 5 years ago

@johnoliver Yes, those look great.

karianna commented 5 years ago

@bdw429s latest PR should hopefully have resolved this for you, can you please check and comment!

bdw429s commented 5 years ago

@karianna The version data itself looks good, but I still think it's in the wrong place. See in the ticket were I said:

The version needs moved up to the release level.

If there are 15 binaries for a single release, they should all have the same version data.

johnoliver commented 5 years ago

We did discuss this, problem with this is that if you move it up to the release level, then for calls where the binaries are not grouped by release, such as the latestAssets the binary asset will have to have the version_data added to it, making what properties a binary asset has inconsistent between different calls to the api.

bdw429s commented 5 years ago

@johnoliver Yes, @ParkerM and I were discussing this in Slack yesterday. To be clear, not every call to a given API should have a consistent response format, but every call to the same resource should. It's ok to share a data structure across resources, but to the point it makes sense. I've never used the latestAssets part of the API, but it sounds like it's representing a different resource and therefore would naturally follow another structure. Or, potentially, should be refactored so it is the same. I feel a bit like we've tossed the baby with the bath water when we avoid the right thing because we can only do it in 95% of the cases.

jcompagner commented 5 years ago

i find this release api really not wel documented or how to use it for example i do the info query to get what is now the latest:

https://api.adoptopenjdk.net/v2/info/releases/openjdk12?release=latest&openjdk_impl=openj9&os=mac&arch=x64&type=jre&heap_size=normal

that gives me a nice json But now i want to fix on that release.. So i want to have something like:

https://api.adoptopenjdk.net/v2/binary/releases/openjdk12?openjdk_impl=openj9&os=mac&arch=x64&type=jre&heap_size=normal&release=

but what do i fill in into that release property that it works? it would be nice if i just could copy past the release_name in it:

https://api.adoptopenjdk.net/v2/binary/releases/openjdk12?openjdk_impl=openj9&os=mac&arch=x64&type=jre&heap_size=normal&release=jdk-12.0.2+10_openj9-0.15.1

but that doesn't work: Unknown release format "jdk-12.0.2 10_openj9-0.15.1"

So how should i know then what to give into that?

Problem with doing release=latest is that for example the maven-download-plugin will cache that an will not test if there is suddenly really another updated resource under that url... And i dont want to always download it when i am making my build for the 3 os's that we support

i guess what i can do now is just take the binary url from the info request:

https://github.com/AdoptOpenJDK/openjdk12-binaries/releases/download/jdk-12.0.2%2B10_openj9-0.15.1/OpenJDK12U-jre_x64_mac_openj9_12.0.2_10_openj9-0.15.1.tar.gz

and use that in my build to fix it

By the way i need to fix it because the annoying part is that the downloads of those tar.gz file still have a very specific directory inside it where the version is in "jdk-12.0.2+10-jre"

so build tools like maven or ant when i extract it really need to know that suddenly there is a very specific sub dir in it... i would love if there would be tar that just removed that "jdk-12.0.2+10-jre" sub dir and that the contents was all one up the chain..

karianna commented 5 years ago

@johnoliver - see if @jcompagner comments give you any new use cases for v3 but I think you've covered all of this.

For those watching this issue -> https://app.swaggerhub.com/apis/johnoliver/API/3.0.0-beta#/

jcompagner commented 5 years ago

i would say that a query of "what is here" that gives you result of multiply releases would give you also the parameter values if you would query a bit more strict.. So the get param value should be able to copy paste from a response of another..

And it would be nice if the param names of the response like:

https://api.adoptopenjdk.net/v2/info/releases/openjdk12?release=latest&openjdk_impl=openj9&os=mac&arch=x64&type=jre&heap_size=normal that gives:

"release_name": "jdk-12.0.2+10_openj9-0.15.1",

can then be used as an input param to another So i would rename "release=lastest" in the url to "release_name=latest" to reflect that it is that property and then be able to do &release_name=jdk-12.0.2+10_openj9-0.15.1 in a get to exactly that that..

Because it seems a bit that "release_name" in the output reflects to "release" in the param right? i would make that the same

karianna commented 4 years ago

v3 API resolved this (from the API perspective), some issues remain open in the build repo.