search

AdoptOpenJDK / openjdk-api

DEPRECATED and replaced by of https://github.com/AdoptOpenJDK/openjdk-api-v3 - See README
https://api.adoptopenjdk.net/README
Apache License 2.0
15 stars 15 forks source link

V2 API server still seems unstable #190

Closed sxa closed 3 years ago

sxa commented 4 years ago

Related: #35

We're still seeing quite a bit of instability (404 errors) when downloading via the API. We need this to be stable to avoid impacting customers.

I had a couple of cases where the website wasn't rendering the download parts, and our ansible playbook testing has been hitting 404s as well (e.g. https://ci.adoptopenjdk.net/view/work%20in%20progress/job/VagrantPlaybookCheck/OS=Win2012,label=vagrant/247/console)

TASK [Java_install : Download Java13] ******************************************
fatal: [172.28.128.120]: FAILED! => {"changed": false, "dest": "C:\\temp\\jdk-13.zip", "msg": "Error downloading 'https://api.adoptopenjdk.net/v2/binary/releases/openjdk13?openjdk_impl=hotspot&os=windows&arch=x64&release=latest&type=jdk&heap_size=normal' to 'C:\\temp\\jdk-13.zip': The remote server returned an error: (404) Not Found.", "status_code": 404, "url": "https://api.adoptopenjdk.net/v2/binary/releases/openjdk13?openjdk_impl=hotspot&os=windows&arch=x64&release=latest&type=jdk&heap_size=normal", "win_get_url": {"dest": "C:\\temp\\jdk-13.zip", "url": "https://api.adoptopenjdk.net/v2/binary/releases/openjdk13?openjdk_impl=hotspot&os=windows&arch=x64&release=latest&type=jdk&heap_size=normal"}}

PLAY RECAP *********************************************************************
sxa commented 4 years ago

I believe @johnoliver increased the resources available to the API recently but that does not appear to have resolved the issues.

karianna commented 4 years ago

@sxa555 - probably worth moving to v3 of the API now.

sxa commented 4 years ago

Is it production ready now? The front of the api website documents V2 and says V3 is being worked on, but the README in the APi repo directs you to use V3

sxa commented 4 years ago

Will test switching it over - just had another 404 from V2 at https://ci.adoptopenjdk.net/job/VagrantPlaybookCheck/OS=CentOS6,label=vagrant/263/console

TASK [adoptopenjdk_install : Install latest release if one not already installed (Linux/x64)] ***
FAILED - RETRYING: Install latest release if one not already installed (Linux/x64) (3 retries left).
FAILED - RETRYING: Install latest release if one not already installed (Linux/x64) (2 retries left).
FAILED - RETRYING: Install latest release if one not already installed (Linux/x64) (1 retries left).
fatal: [172.28.128.103]: FAILED! => {"attempts": 3, "changed": false, "msg": "Failure downloading https://api.adoptopenjdk.net/v2/binary/releases/openjdk13?openjdk_impl=hotspot&os=linux&arch=x64&heap_size=normal&release=latest&type=jdk, HTTP Error 404: Not Found"}

And https://ci.adoptopenjdk.net/job/VagrantPlaybookCheck/OS=CentOS7,label=vagrant/263/console


TASK [adoptopenjdk_install : Install latest release if one not already installed (Linux/x64)] ***
FAILED - RETRYING: Install latest release if one not already installed (Linux/x64) (3 retries left).
FAILED - RETRYING: Install latest release if one not already installed (Linux/x64) (2 retries left).
FAILED - RETRYING: Install latest release if one not already installed (Linux/x64) (1 retries left).
fatal: [172.28.128.103]: FAILED! => {"attempts": 3, "changed": false, "msg": "Failure downloading https://api.adoptopenjdk.net/v2/binary/releases/openjdk13?openjdk_impl=hotspot&os=linux&arch=x64&heap_size=normal&release=latest&type=jdk, HTTP Error 404: Not Found"}
sxa commented 4 years ago

FYI @willsparker since I suspect you've seen this a few times too :-)

karianna commented 4 years ago

Is it production ready now? The front of the api website documents V2 and says V3 is being worked on, but the README in the APi repo directs you to use V3

I'd say v3 is ready to use yeah, especially for internal usage.

Willsparker commented 4 years ago

When I get a spare minute I'll test a branch that changes the adoptopenjdk_install URL to v3, on the vagrantPlaybookCheck job and link the results :-)

Willsparker commented 4 years ago

Actually, scratch that, just saw @sxa555 has a PR in

johnoliver commented 4 years ago

Yeah v3 is probably good enough for internal, given now we have a staging server setup so we can properly test the api before deployment. We also fixed an issue where due to a number of programs using the same github api tokens, they were draining the quota and causing some of these issues, this should mean that v2 will be more stable in the future.

karianna commented 4 years ago

As discussed on the call, this is likely a rate-limiting issue and we can use the V3 API trick of backing off and using last modified. Let's see if we can implement that patch here in V2 to stabilize it.

Eskibear commented 4 years ago

Has v2 been removed? 

$> curl -L 'https://api.adoptopenjdk.net/v2/info/releases/openjdk8'
Not found

Using nodejs client to send the request, from other location (to check if it's networking issue), I also got a 404 error.

UPDATE: It gets back now after 10 min. I have one question, when will v2 be out of service? Is there any exact deadline, if we don't have plan to migrate to v3 now.

sxa commented 4 years ago

Great question @Eskibear!

We do not have a specific date at the moment - it's unlikely to be removed in the next few months though as there is still a significant amount of use of the V2 API and we're only just in the processes of announcing V3 properly (although many have already moved to it as it's been stable, and we've been using it ourselves for the past few months internally on the project in order to ensure it's quality)

Everyone is encouraged to move to V3 ASAP though. The 404 errors are caused by instability of the V2 API at the moment (hence this issue!) and absolutely not as a result of starting to decomission the V2 API.

Hope that helps.

ibauersachs commented 4 years ago

At least today, v3 is also unstable. Getting a lot of timeouts and now even 503s.

johnoliver commented 4 years ago

@ibauersachs No idea why there would be instability for v3, there were a few restarts on friday due to updates in relation to the release but should not have been for significant amounts of time, do you know roughly what time you were getting these timeouts?

ibauersachs commented 4 years ago

That was around the time I left the above comment, so roughly between 20:00 and 22:00 UTC on April 17.

By release I assume you mean the jdk8u252/jdk11.0.7 etc. updates? I would definitely not expect any downtime on the API for this.

johnoliver commented 4 years ago

We had some issues with incorrect classification of the new ea releases of openj9 arch that caused us to have to do a redeploy of the api, that will have caused some downtime

karianna commented 3 years ago

@johnoliver - I think the v2 API has been stable for quite some time, can we close this now?

Eskibear commented 3 years ago

@karianna I just tried, v2 API is even more unstable than before, especially for openjdk11. Can anyone take a look?

# 11  -> (404)
$> curl -L "https://api.adoptopenjdk.net/v2/info/releases/openjdk11"
Not found

# 11  -> (404)
$> curl -L "https://api.adoptopenjdk.net/v2/info/releases/openjdk11?openjdk_impl=hotspot&arch=x64&os=mac&type=jdk&release=latest"
Not found

# 8 -> (404)   NOTE: unstable, later when I tried the same command, it's ok.
$> curl -L 'https://api.adoptopenjdk.net/v2/info/releases/openjdk8'
Not found

# 8  -> (ok)
$> curl -L "https://api.adoptopenjdk.net/v2/info/releases/openjdk8?openjdk_impl=hotspot&arch=x64&os=mac&type=jdk&release=latest"
{
  "release_name": "jdk8u275-b01",
  "release_link": "https://github.com/AdoptOpenJDK/openjdk8-binaries/releases/tag/jdk8u275-b01",
  "timestamp": "2020-11-11T15:56:17Z",
  "release": true,
  "binaries": [
    {
      "os": "mac",
      "architecture": "x64",
      "binary_type": "jdk",
      "openjdk_impl": "hotspot",
      "binary_name": "OpenJDK8U-jdk_x64_mac_hotspot_8u275b01.tar.gz",
      "binary_link": "https://github.com/AdoptOpenJDK/openjdk8-binaries/releases/download/jdk8u275-b01/OpenJDK8U-jdk_x64_mac_hotspot_8u275b01.tar.gz",
      "binary_size": 101776105,
      "checksum_link": "https://github.com/AdoptOpenJDK/openjdk8-binaries/releases/download/jdk8u275-b01/OpenJDK8U-jdk_x64_mac_hotspot_8u275b01.tar.gz.sha256.txt",
      "installer_name": "OpenJDK8U-jdk_x64_mac_hotspot_8u275b01.pkg",
      "installer_link": "https://github.com/AdoptOpenJDK/openjdk8-binaries/releases/download/jdk8u275-b01/OpenJDK8U-jdk_x64_mac_hotspot_8u275b01.pkg",
      "installer_size": 105453786,
      "installer_checksum_link": "https://github.com/AdoptOpenJDK/openjdk8-binaries/releases/download/jdk8u275-b01/OpenJDK8U-jdk_x64_mac_hotspot_8u275b01.pkg.sha256.txt",
      "installer_download_count": 2380,
      "version": "8",
      "version_data": {
        "openjdk_version": "8u275-b01",
        "semver": "8.0.275+1"
      },
      "heap_size": "normal",
      "download_count": 373,
      "updated_at": "2020-11-11T15:57:01Z"
    }
  ],
  "download_count": 15873
}

# 15 -> (ok)
$> curl -L "https://api.adoptopenjdk.net/v2/info/releases/openjdk15?openjdk_impl=hotspot&arch=x64&os=mac&type=jdk&release=latest"
{
  "release_name": "jdk-15.0.1+9",
  "release_link": "https://github.com/AdoptOpenJDK/openjdk15-binaries/releases/tag/jdk-15.0.1%2B9",
  "timestamp": "2020-10-23T20:46:22Z",
  "release": true,
  "binaries": [
    {
      "os": "mac",
      "architecture": "x64",
      "binary_type": "jdk",
      "openjdk_impl": "hotspot",
      "binary_name": "OpenJDK15U-jdk_x64_mac_hotspot_15.0.1_9.tar.gz",
      "binary_link": "https://github.com/AdoptOpenJDK/openjdk15-binaries/releases/download/jdk-15.0.1%2B9/OpenJDK15U-jdk_x64_mac_hotspot_15.0.1_9.tar.gz",
      "binary_size": 195773522,
      "checksum_link": "https://github.com/AdoptOpenJDK/openjdk15-binaries/releases/download/jdk-15.0.1%2B9/OpenJDK15U-jdk_x64_mac_hotspot_15.0.1_9.tar.gz.sha256.txt",
      "installer_name": "OpenJDK15U-jdk_x64_mac_hotspot_15.0.1_9.pkg",
      "installer_link": "https://github.com/AdoptOpenJDK/openjdk15-binaries/releases/download/jdk-15.0.1%2B9/OpenJDK15U-jdk_x64_mac_hotspot_15.0.1_9.pkg",
      "installer_size": 201928998,
      "installer_checksum_link": "https://github.com/AdoptOpenJDK/openjdk15-binaries/releases/download/jdk-15.0.1%2B9/OpenJDK15U-jdk_x64_mac_hotspot_15.0.1_9.pkg.sha256.txt",
      "installer_download_count": 4518,
      "version": "15",
      "version_data": {
        "openjdk_version": "15.0.1+9",
        "semver": "15.0.1+9"
      },
      "heap_size": "normal",
      "download_count": 2664,
      "updated_at": "2020-10-23T20:48:09Z"
    }
  ],
  "download_count": 54866
}
karianna commented 3 years ago

Have landed a PR to only refresh every 12 hours - but this will need a proper fix on Monday.

karianna commented 3 years ago

OK so last night's PR did the trick to stabilize, it's roughly an 8-sec outage every 15 mins for the refresh cycle now as opposed to 90 sec. But we now only pull back the last 50 days of results. More work will be done this week to try and get it to point to the V3 backend cache.

Again please do upgrade to V3 as soon as you can.

gdams commented 3 years ago

This should this resolved now that we have no dependency on the Octokit API data anymore