Foundationdb.org Service 503

davidvella commented 2 years ago

I am trying to download the foundation db client libraries but the foundationdb.org website is having issue and I receive 503 exceptions:

##[error]#18 [builder  9/12] RUN wget "https://foundationdb.org/downloads/6.3.22/ubuntu/installers/foundationdb-clients_6.3.22-1_amd64.deb"
##[error]#18 sha256:9ade7b951a0354d7e37b293d383c99f656c7f4d03b75ce75d5ed227b0eb2dfe3
##[error]#18 0.352 --2021-12-15 09:59:33--  https://foundationdb.org/downloads/6.3.22/ubuntu/installers/foundationdb-clients_6.3.22-1_amd64.deb
##[error]#18 0.361 Resolving foundationdb.org (foundationdb.org)... 17.56.48.13, 17.125.250.130
##[error]connected.
##[error]503 Service Temporarily Unavailable
##[error]#18 1.039 2021-12-15 09:59:34 ERROR 503: Service Temporarily Unavailable.

<html><head>
<title>503 Service Unavailable</title>

<style>
body {
    background-color: #f8f8f8;
    color: #505050;
    font-family: "SF Hello", "Myriad Set Pro", "Myriad Set", "Lucida Grande", Arial, sans-serif;
}

hr {
    margin: 10px 0px 10px 0px;
    border: 1px solid #666;
}

#container {
    margin-left: 50px;
    position: absolute;
    top: 50%;
    transform: translateY(-50%);
    width: 55%;
}

#logo {
    margin-right: 10px;
}

#code {
    font-size: 64px;
    font-weight: bold;
}

#msg {
    font-size: 32px;
    font-weight: lighter;
    margin-top: 10px;
    color: #505050;
}
#desc {
    margin-bottom: 33px;
}

#details {
    font-size: 13px;
    color: #909090;
    display: table;
    background-color: #fefefe;
    width: 100%;
    padding: 10px 0px 10px 0px;
}

.detail {
    display: table-row;
}

.detail .key {
    font-weight: bold;
    display: table-cell;
    text-align: right;
    padding: 1px 10px 1px 12px;
    white-space: nowrap
}

.detail .value {
    display: table-cell;
    font-family: "SF Mono", Courier, Monaco, monospace;
    padding: 1px 0px 1px 0px;
}

</style>
</head>
<body>
<div id="container">

<div id="code"><span id="logo"></span>503</div>
<div id="msg">Service Unavailable</div>
<hr>
<div id="desc">It appears no upstream tasks are announcing via service discovery.
          Please check the health of the upstream tasks.</div>

<div id="details">
<div class="detail"><span class="key">Access Time:</span><span class="value">Wed, 15 Dec 2021 11:12:09 GMT</span></div>

<div class="detail"><span class="key">Reference ID:</span><span class="value">d6a9b89b-8645-4afd-a5e0-a57b2991b643</span></div>

<div class="detail"><span class="key">Version:</span><span class="value">AppleHttpServer/6dc1540be35565f1555759da807d61fe</span></div>

</div>
</div>

</body></html>

jzhou77 commented 2 years ago

Yes, the site is down for maintenance. @ammolitor can update when the site is back.

falsandtru commented 2 years ago

https://apple.github.io/foundationdb/downloads.html uses https://www.foundationdb.org. Which is the correct domain?

ducc commented 2 years ago

Download links on https://apple.github.io/foundationdb/downloads.html are all broken and therefore causing docker builds to fail - any updates on when this will be fixed?

falsandtru commented 2 years ago

Indeed, https://www.foundationdb.org is also down.

imcarrow commented 2 years ago

COran commented 2 years ago

I'm still having issue to retrieve the image of foundationdb client : now the version I'm downloading is not a valid deb file... Do you have an ETA when the website will be up again ?

aragastmatb commented 2 years ago

Same, got 404 while trying to get https://www.foundationdb.org/downloads/6.2.22/rhel7/installers/foundationdb-clients-6.2.22-1.el7.x86_64.rpm

ammolitor commented 2 years ago

The site is still down. I do not have an ETA for it coming back.

PierreZ commented 2 years ago

The site is still down. I do not have an ETA for it coming back.

Can we help you somehow on this?

DenisBY commented 2 years ago

it would be nice if you upload packages to, i.e., github.

ammolitor commented 2 years ago

site is partially back (still no downloads, intentionally) should have assets here on GitHub under releases very soon.

kenkeiter commented 2 years ago

Good call moving to GitHub's releases feature! And thanks for getting the site up and running 🚀

If would be cool if you could set up a redirect on the site from the old package URLs to their corresponding GitHub Releases asset URLs so automation doesn't get broken. I think I saw a couple people that were relying upon that...

ammolitor commented 2 years ago

Release assets are populating now: https://github.com/apple/foundationdb/releases/tag/6.3.22

Moving forward, this will be the place to download from. The webiste download pages will be removed.

@kenkeiter, I like your suggestion, but if you've ever tried to host a public website from inside of a large company, I'm sure you can appreciate the lack of flexibility. In fairness, I am having to update parts of my build tooling to point to new links as well.

DenisBY commented 2 years ago

@ammolitor will you upload packages for old releases as well? We are using 6.3.13

davidvella commented 2 years ago

@ammolitor will you upload packages for old releases as well? We are using 6.3.13

I can see them here: https://github.com/apple/foundationdb/releases/tag/6.3.13

davidvella commented 2 years ago

@ammolitor I can now download from github. Will the documentation need to be updated as part of this ticket or can I close?

Here is my dockerfile changes I made below.

ARG FDB_WEBSITE=https://github.com
# Install FoundationDB Client Libraries (for compilation)

RUN wget "${FDB_WEBSITE}/apple/foundationdb/releases/download/${FDB_VERSION}/foundationdb-clients_${FDB_VERSION}-1_amd64.deb"

RUN dpkg -i foundationdb-clients_${FDB_VERSION}-1_amd64.deb

falsandtru commented 2 years ago

That is nice. @ammolitor Can you unify the download paths of GitHub and Foundationdb.org to make it possible to switch the host?

DenisBY commented 2 years ago

@davidvella right, sorry. I just need to expand Assets :)

ammolitor commented 2 years ago

@davidvella I am updating all of the dangling references to downloading from the website today, I think it is safe to close this issue.

sebastienh commented 2 years ago

We used to be able to access https://www.foundationdb.org/downloads/$FDB_VERSION/linux/fdb_$FDB_VERSION.tar.gz will this file be available on GitHub too? I don't see it in the files listed.

davidvella commented 2 years ago

We used to be able to access https://www.foundationdb.org/downloads/$FDB_VERSION/linux/fdb_$FDB_VERSION.tar.gz will this file be available on GitHub too? I don't see it in the files listed.

A quick google search. If you look at the changes in the last Dockerfile below (it should have what you need):

https://github.com/FoundationDB/fdb-kubernetes-operator/commit/8a1913099efdb5dbf543f0954f718c3a11aa5ca4

sebastienh commented 2 years ago

@davidvella Thanks for the answer but I am not using the debian package for installation. In fact, I am following https://github.com/apple/foundationdb/blob/master/packaging/docker/Dockerfile, and it is broken too. They are using linux binaries. I could change my image building process but if the file would ever become available, it would be simpler. If it is not produced anymore, I will need to move to your suggestion.

davidvella commented 2 years ago

@davidvella Thanks for the answer but I am not using the debian package for installation. In fact, I am following https://github.com/apple/foundationdb/blob/master/packaging/docker/Dockerfile, and it is broken too. They are using linux binaries. I could change my image building process but if the file would ever become available, it would be simpler. If it is not produced anymore, I will need to move to your suggestion.

@sebastienh What I tried before which could be another suggestion for you is to copy the files what you need from the Docker image itself.

ammolitor commented 2 years ago

@sebastienh that file is not produced by the CMake/CPack build. it is a custom file that was created for reasons that are unknown to me. Moving forward that file will not produced or published.

If you take a look at https://github.com/apple/foundationdb/blob/master/packaging/docker/build-images.sh you'll see that a fake website (that is referenced with a file:/// URL) is used to create the images. This process allows for building the docker images at the same time as FDB is compiled and packaged for release.

The history here is that the concept of packaging FDB in docker images came after the original build process. Between that, and the fact that team that created the original docker images did not work on the FDB build process, we are left with something that looks and feels very much like it was put together with duct tape and string.

I will be fixing the Dockerfile(s) but I can't say exactly when 😞

For now my suggestion would be to modify or copy parts of the build-images.sh script to fit your needs, or copy files from the release images as @davidvella suggested.

sebastienh commented 2 years ago

@ammolitor thanks for the clear answer.

I mainly use this docker image for unit testing locally, so I don't need something production ready. I adapted the existing Dockerfile that produced the image based on what I had already and took inspiration from the Kubernetes operator for the installation of the packages using the Debian files and I came up with something that works for me. I leave it here and maybe that will be useful for you or somebody else. I install the foundationdb-clients first which the server packages needs, and then proceed with the foundationdb-server per say. I am using it to create my local server and all my unit tests pass as before.


ARG FDB_VERSION
ARG FDB_LIBRARY_VERSIONS="${FDB_VERSION}"
ARG FDB_WEBSITE=https://github.com/apple/foundationdb/releases/download

RUN set -eux && \
    curl --fail -L ${FDB_WEBSITE}/${FDB_VERSION}/foundationdb-clients_${FDB_VERSION}-1_amd64.deb -o fdb.deb && \
    dpkg -i fdb.deb && rm fdb.deb && \
    mkdir -p /usr/bin/fdb/${FDB_VERSION%.*} && \
    for binary in fdbbackup fdbcli; do \
        download_path=/usr/bin/fdb/${FDB_VERSION%.*}/$binary && \
        curl --fail -L ${FDB_WEBSITE}/${FDB_VERSION}/$binary.x86_64 -o $download_path && \
        chmod u+x $download_path; \
    done && \
    mkdir -p /usr/lib/fdb

RUN set -eux && \
    curl --fail -L ${FDB_WEBSITE}/${FDB_VERSION}/foundationdb-server_${FDB_VERSION}-1_amd64.deb -o fdb.deb && \
    dpkg -i fdb.deb && rm fdb.deb && \
    mkdir -p /usr/bin/fdb/${FDB_VERSION%.*} && \
    for binary in fdbmonitor fdbserver ; do \
        download_path=/usr/bin/fdb/${FDB_VERSION%.*}/$binary && \
        curl --fail -L ${FDB_WEBSITE}/${FDB_VERSION}/$binary.x86_64 -o $download_path && \
        chmod u+x $download_path; \
    done && \
    mkdir -p /usr/lib/fdb

Or course this could be refined, but I just needed something that works to be able to run unit tests.

apple / foundationdb

Foundationdb.org Service 503 #6154