Create Mason API & Server

[flippmoke]

It would be interesting we were able to create a lightweight client such as has been proposed in #187, but the information on how and where builds are completely seperate from the client.

The Mason server would live on an address such as https://mason.mapbox.com/. Several simple endpoints would exist for this server such as:

https://mason.mapbox.com/download which would be a request to download a specific package, you would provide a simple JSON object that you would POST to this endpoint.

{
    "package": "geometry",
    "version": "v0.9.0",
    "arch": "osx",
    "type": "debug",
     // Other options here
}

The server would then respond either with a path to download or would simply provide a 307 HTTP code redirect to the s3 location for the proper file to be downloaded or some other structure. We could also provide back an expect hash for the file if we wanted to simply return a path and hash as a JSON response

{
     "package": "geometry",
     "build": 4313,
     "path": "https://mason.mapbox.com/geometry/4313/build.tar.gz",
     "hash": "12345123412341234g34123",
}

This would be nice because we could then use a database to manager the list of packages and provide a friendly user interface perhaps for managing of these packages. Additionally we could have rebuilds a single version of a package and point to the "correct" one for each version so we are never accidentally blasting over a build.

Additionally it would be easy to provide back a list of all the packages for a specific version:

https://mason.mapbox.com/list

{
    "package": "geometry",
    "arch": "osx",
    "type": "release",
    // other options
}

the server then could respond back with JSON with something such as:

{
     "package": "geometry",
     "versions": [ "0.4.0", "0.5.0", "0.9.0" ]
}

[kkaefer]

This would be nice because we could then use a database to manager the list of packages and provide a friendly user interface perhaps for managing of these packages.

What would be the use case for "managing" the packages? Who would use it? What would you do with it?

Additionally it would be easy to provide back a list of all the packages for a specific version:

What would it be used for?

We could also provide back an expect hash for the file if we wanted to simply return a path and hash as a JSON response

What's the benefit of providing this in-band?

[flippmoke]

What would be the use case for "managing" the packages? Who would use it? What would you do with it?

Mistakes are made and sometimes we might need to remove or update a package in the past. We might add more options going into the future, and have a mapping that is independent of the client means that we could change paths, names, and many other things with out changing the on the client.

What would it be used for?

If I have no idea of what packages are out there during development, this is a great simple way to know what all is out there. This is especially true as we divorce the client from the build system. It would be nice to know what all existing builds are on S3 in a simple command line method.

What's the benefit of providing this in-band?

If you had a failure in the download of the file or a hash was not matching for some reason you would be quickly able to test. This could also be used to not download a new version if you were unsure of your current version of your file. If the hashes do not match, you would need to redownload etc. It also would help in the situation where you republished a version of a file.

[kkaefer]

Mistakes are made and sometimes we might need to remove or update a package in the past.

In the past, we've used the S3 console or CLI client to resolve those errors; the number of times when we've need them are extremely low, so I think building out a UI for it doesn't make sense.

we could change paths, names, and many other things with out changing the on the client.

We were thinking about versioning the directory layout on the server, so whenever we're introducing breaking changes to the layout, we'd update the version prefix. To avoid storing duplicate packages, S3 supports redirects that we could create with a script.

If you had a failure in the download of the file or a hash was not matching for some reason you would be quickly able to test.

We've been using Mason for quite some time, and I haven't had any issues with download integrity so far. A better approach might be to sign the binaries after building them on CI infrastructure, then verify the signature, which gives us both authenticity and integrity and doesn't require a per-file checksum that we need to transfer to the client too.