Integrate with CodeArtifact for module publishing

[ericzbeard]

First class support for client-side modules is something many customers are missing with CloudFormation. Rain now offers a few ways to enable this, with Rain::Module and Pkl support, but we still don't have a standard way to publish and share modules. CodeArtifact could be a mechanism to give us the equivalent of npm install.

The CodeArtifact service can host generic files in addition to standard package managers like NPM. It might make adoption of modules (!Rain::Module) easier if we gave users an easy way to publish and consume libraries.

rain module publish ./my-modules --domain my-domain --repository my-repo

By default, rain would create or use a domain called cloudformation-rain, and a repo called cloudformation-rain-modules.

Users could install modules like this:

rain module install my-modules ./modules

This would use the default domain and repo, download the latest version, unzip it if necessary, verify the hash, and write them to disk.

We could also consider a lock file of some kind the record the package version.

CodeArtifact usage should safely fall into the free tier for casual usage, but we shouldn't bootstrap it by default. Add a param to the bootstrap command to set it up, and use a confirmation prompt with first time usage of rain module.

[ericzbeard]

@hariprakash-j @null93 @khmoryz @kddejong I would like to hear your feedback on this idea

[null93]

I think this is a really cool idea. I do not use rain modules in production yet because of some limitations. It is a really good abstraction but not a perfect one for our use case. That being said, it would be really cool to see this.

An alternative way you can host these modules could be via a simple url as well. That way, it doesn't rely on any particular service or require a login. You can host them on a public Github repo and the url you use can be versioned to a certain tag, branch, or commit.

This could simplify the implementation because all you would have to do is change the Rain::Module intrinsic function to support downloading modules from a url.

[ericzbeard]

This could simplify the implementation because all you would have to do is change the Rain::Module intrinsic function to support downloading modules from a url.

It already does! That would be the simplest way but I was thinking an npm/pip-like experience might feel more familiar for developers.

Can you tell me about the limitations that prevent you from using it? It's still experimental so we have room to change how it works.

[hariprakash-j]

This is a great idea, I've been thinking about having rain work like how helm does for Kubernetes, fetch templates and pass in params to create a deployment, this could be really work like that.

Modules already supports params so its already a little like helm already.

We could have a lock file like you said to make installing modules process repeatable a sort of a npm_modules folder to store all the downloaded modules, preferably a central one instead of having it on the project level ? this way many projects could share the modules and not have to download it every time, this would also not waste space on the user's system.

An option to download modules directly from a git repo would be nice too, similar to how go modules works and how rain itself can be installed using go. I really find it convenient to just put stuff in github and get it from there and not package it and push it from github to some artifact hosting service, but this is probably just me though.

just a few thoughts to start but it is a really cool idea !