I know unfortunately we haven't made an official release of API Umbrella in quite some time (embarrassingly long, really). For several years, we've been working in various other branches of the code base which have diverged from main, which has made it difficult to maintain main. I've had intentions of making some final releases of what was previously the main branch (and that could perhaps still happen if there's demand), but at this point, I think we're going to move forward with merging the code we're actively working on into main so we can then maybe figure out better release strategies moving forward. But apologies we haven't had the bandwidth to be better open-source stewards for this project.
What was previously in main is now in a 0.16.x branch that can live on for reference. This branch includes various upgrades over what were made over the last 0.15.x releases, but these changes never made their way into a formal release.
After this pull request is merged, main will then reflect the work of 0.17.x moving forward.
So what's changed?
Lua-based web-app: The web-app component in 0.16.x and prior is a Ruby on Rails v4.2 web application. Rails v4 hasn't been supported for quite some time, so this has long been a potential security issue. In this PR, the web-app component is now a Lua web application (using Lapis). In light of the rest of the proxy stack being written in Lua, this better consolidates things and allows for better reuse, since all the server-side code is now Lua.
We've made various additions to the web app over the years, but from an API perspective, the v1 APIs provided by the web app for public or admin consumption should be compatible with what the Rails app provided.
Switch to PostgreSQL: In 0.16.x and prior, the "general" database for API key storage, API backends, etc is Mongo. This has now been migrated to use PostgreSQL. This was initially driven by various compliance requirements we faced, but PostgreSQL is probably a better fit at this point, and easier to find support or managed services for.
Before we release 0.17.x, we want to have a formalized upgrade and migration process. There are various scripts we used to migrate from Mongo to Postgres with zero downtime included in this this PR, so a migration is certainly possible, but this process needs to be cleaned up and better documented.
Remove bundled databases: To simplify maintenance, we no longer bundle a copy of the databases (Elasticsearch, and previously Mongo) in our packages. The embedded versions were mostly just meant for demo purposes, so it was always a better idea to run those separately, but by removing those, it's less for us to worry about upgrading.
Container-first builds/releases: Previously, our release process involved generating many RPM and deb packages for many different distros. This is a big reason why releasing new formal versions was always such a hurdle since we didn't really have the bandwidth to support all of these distros or packaging processes. While a lot of the code to build these packages across distros remains, unless there's significant demand, we may move away from distro package releases altogether. Instead, we're mainly focused on building container images (for Docker, etc) which lets us focus on compatibility for only 1 distro (currently Debian Bullseye), and better matches our own needs and release process.
Other: Too many other changes over the past few years that I'm probably forgetting. A few random ones I can think of:
Config files are now validated according to a schema, so invalid configuration settings in the YAML files will throw errors.
Lots of efficiency, speed, and memory improvements. For example, a new rate limiting algorithm based on estimates, that's much more memory efficient.
Upgrading the admin UI app to the latest versions of Ember.
Envoy is now used as the final proxy layer to proxy to API backends to provide easier dynamic configuration, proper load balancing, health checks, and better failover behavior.
The example "static-site" as been upgraded and nested within this repo under the example-site component.
Translations now use a gettext-based approach, making it easier for us to keep translations in sync.
I know unfortunately we haven't made an official release of API Umbrella in quite some time (embarrassingly long, really). For several years, we've been working in various other branches of the code base which have diverged from
main, which has made it difficult to maintainmain. I've had intentions of making some final releases of what was previously themainbranch (and that could perhaps still happen if there's demand), but at this point, I think we're going to move forward with merging the code we're actively working on intomainso we can then maybe figure out better release strategies moving forward. But apologies we haven't had the bandwidth to be better open-source stewards for this project.What was previously in
mainis now in a0.16.xbranch that can live on for reference. This branch includes various upgrades over what were made over the last 0.15.x releases, but these changes never made their way into a formal release.After this pull request is merged,
mainwill then reflect the work of0.17.xmoving forward.So what's changed?
web-app: Theweb-appcomponent in0.16.xand prior is a Ruby on Rails v4.2 web application. Rails v4 hasn't been supported for quite some time, so this has long been a potential security issue. In this PR, theweb-appcomponent is now a Lua web application (using Lapis). In light of the rest of the proxy stack being written in Lua, this better consolidates things and allows for better reuse, since all the server-side code is now Lua.0.16.xand prior, the "general" database for API key storage, API backends, etc is Mongo. This has now been migrated to use PostgreSQL. This was initially driven by various compliance requirements we faced, but PostgreSQL is probably a better fit at this point, and easier to find support or managed services for.0.17.x, we want to have a formalized upgrade and migration process. There are various scripts we used to migrate from Mongo to Postgres with zero downtime included in this this PR, so a migration is certainly possible, but this process needs to be cleaned up and better documented.example-sitecomponent.