Tasks and/or Goals for Updating/Improving Amazonka

[jkachmar]

amazonka and gogol are really powerful "force multipliers" for Haskell when it comes to using the language for a lot of common tasks in cloud environments, but both projects have felt more like they're on life-support as opposed to being actively maintained since #484.

Is there currently any project roadmap for actively maintaining and improving amazonka?

If not, is there any interest in trying to set one up?

I feel like there are probably a few regular/professional users of the library who would have ideas of what they'd like to see that could be used to drive current/future development.

[brendanhay]

This probably deserves a longer form reply, but to answer your question directly - there is no currently public roadmap for maintaining and improving amazonka and gogol. I have fairly elaborate in motion plans and development work for improving both of these libraries, which currently lives only in a private repository and my head.

Some of the observations I've made during development of these libraries made me reevaluate the approach(es) and led to the current 'life support' status you mention:

• The original intent was to provide a very direct mapping of the types from the API service definitions to Haskell. They were to be the substrate that the community would then build richer, more natural abstractions atop of. This philosophy ended up being compromised, hence the inclusion of a concrete transformer etc. which I think was a mistake, and would prefer to offer up more a direct :: Env -> request -> IO response, typeclass-less API to users.
• There has been a (relatively) limited amount of community involvement due to the poorly documented and structured codebase. Too much time fixing and not enough time docucmenting to give others a starting point.
• The predominant consumers of libraries of this nature are businesses/corporations and to be honest, I only need two fingers to count those that have offered to contribute resources (money or otherwise) to help with the development of either library. Given this is not a 'library-for-fun', it's somewhat demotivating to multiple subsequent entire weekends fixing commercial software.
• Maintenance burden for 300+ (and growing) APIs is prohibitive given the current state of the tooling. And this really is the biggest elephant in the room, alot of the other issues are moot if the library was trivial to regenerate, better support out of date types vs update upstream definitions (InstanceType missing "z3.massive" errors, etc.), and so on.

In light of these, in late 2017 I started exploring using shake to unify the build infrastructure, replace the shell scripts, and generally make the whole process more automated and non-reliant on my personal workflow. Some additional work has been done to develop a core intermediate-representation that we can elaborate any service definition (Amazon, Google/JSONSchema, OpenAPI, etc.) into something shares common traversals such as renaming, prefixing and other core AST transformations that are currently duplicated between the two library generators. Initially this looks like it will massively reduce the maintenance overhead and open up the library to many more potential contributors.

The main constraint is of course, as always, time. I wanted to get all of this to a tidier, less exploratory state, before opening it up to the wider community as a path forward for version 2.0 of both libraries.

[MichaelXavier]

I think I'm one of those commercial users. My current job essentially doesn't give me the bandwidth I'd need to triage issues on this repo or do heavy work on it. My involvement is sporadic and as-needed within my company. This shake work sounds promising. I wonder if we could get it specified enough to mount a community effort to get that done and then really start opening up the project to more direct contributors and maintainers with a stake in the code. Perhaps even to the point in putting a moratorium on feature work that doesn't just come as a PR until the project is in a good state for active development. I feel super bad when I open my notifications for this project and see dozens of people having problems, but I'm not going to spend precious free time working on that sort of thing. I just don't have the energy.

[jkachmar]

Given the state of the outside world at the moment, especially in my home city, I'm going to be spending a lot more time indoors for the foreseeable future. In light of this I've been looking over old issues I've opened up to see if there's anything I can do to help close them myself.

Most importantly, I want to say thanks to @brendanhay for the in-depth response to my original issue. amazonka and gogol are two of the most impressive Haskell libraries I've had the good fortune of using, and your effort here has improved my Haskell development experience both directly and transitively many times over.

I'd also like to somewhat echo what @MichaelXavier said above: if there's any chance that I could assist with the aforementioned shake work, I'd absolutely be willing to set aside free time to help here. I can't imagine that I'll be doing much else with my free time for the next few months, anyway 😅 .

[endgame]

The broad strokes of this discussion still look relevant, though the details appear to have changed (building is nixified instead of shake-ified). Keeping open for now, since a broader roadmap discussion is probably worth having once 2.0 gets done.

[brendanhay]

The shake work mentioned is separate from the Nix/Bazel work. It's related to having a generator that is fully incremental (ie. only regenerate what has changed - which makes it nicely compatible with Bazel) and possibly embeddable, ie. only generate what you need/use directly into your project - as a build optimisation, so you don't need to say, build/depend on the entirety of amazonka-ec2 when you only use a handful of functions/types.