search

apache / airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
https://airflow.apache.org/
Apache License 2.0
37.4k stars 14.36k forks source link

Add SSMRunCommandOperator #42619

Open muraria opened 1 month ago

muraria commented 1 month ago

Description

Amazon Systems Manager (SSM) is used by system admins to manage AWS resources like EC2. It also provided ability for companies to run scripts on the EC2 machines securely using it's RunCommand feature. Adding SSM RunCommand Operator to the Amazon provider package will unlock migration to Airflow from legacy job scheduling frameworks like AutoSys

Use case/motivation

I work with a few customers that are in the process of migrating jobs from AutoSys and other legacy frameworks to Airflow. For these customers, a lot of their existing jobs are in shell scripts running on EC2. When they migrate to Airflow, they are looking for an easy way to lift those scripts and run on Airflow. SSHOperator could be an option for this but in many corporate environments, SSH is disabled for security reasons. One of the secure ways AWS recommends executing a remote script on EC2 is by using SSM RunCommand.

Related issues

No response

Are you willing to submit a PR?

Code of Conduct

boring-cyborg[bot] commented 1 month ago

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

mr1holmes commented 1 month ago

@vincbeck I would like to work on this, can you please assign it to me

vincbeck commented 1 month ago

Sure!

shubham22 commented 2 weeks ago

@mr1holmes - curious if you're still working on it?

mr1holmes commented 2 weeks ago

Sorry I didn't get time to work on it. You can assign it to someone else. Apologies

Shlomit-B commented 3 days ago

@eladkal @vincbeck - Hey, I would like to work on this one if that's ok

vincbeck commented 2 days ago

Sure :)