awslabs / amazon-emr-cli

A command-line interface for packaging, deploying, and running your EMR Serverless Spark jobs
Apache License 2.0
36 stars 13 forks source link

Finalize CLI API #26

Open dacort opened 1 year ago

dacort commented 1 year ago

The EMR CLI has been available for a while now and through my own usage and others, we have a good idea of the final set of commands and subcommands that should be supported by the CLI.

Today, certain things are confusing:

Typically, I only use run ... --build, but in CI/CD pipelines both package and deploy can be useful. Package if you want to move the assets yourself and deploy if you want to have the CLI do the copy for you in 1 step.

It would be useful to be able to chain these commands as opposed to providing parameters. For example:

emr build deploy run --entrypoint file.py ... would perform all of build, deploy, and run in that order. That said there are some things that don't make sense, so should protect against scenarios like this.

monometa commented 5 months ago

Hi,

Regarding this point:

Do we need both package and deploy? Or can we simply build and run?

I believe it makes sense, as sometimes it's useful to have an opportunity to check the resulting ZIP structure and modules underneath. Otherwise, there will be a need to download it from S3, etc.