Added `config-pull` command to the CLI

chelma commented 8 months ago

Description

Added the config-pull command to the CLI. The default behavior is to retrieve the currently deployed configuration, which is then written to disk in the repo directory. The user can also request the --previous config, or a specific --config-version. The command aborts if the location on disk it tries to write to is already occupied.

Tasks

https://github.com/arkime/aws-aio/issues/142

Testing

Added unit tests

Tested against my own AWS account:


(.venv) chelma@3c22fba4e266 aws-aio % ./manage_arkime.py config-pull --cluster-name MyCluster --capture
2023-12-15 13:23:49 - Debug-level logs save to file: /Users/chelma/workspace/Arkime/aws-aio/manage_arkime/manage_arkime.log
2023-12-15 13:23:49 - Using AWS Credential Profile: None
2023-12-15 13:23:49 - Using AWS Region: default from AWS Config settings
2023-12-15 13:23:49 - Retrieving current config bundle...
2023-12-15 13:23:50 - Placed config bundle on disk:
/Users/chelma/workspace/Arkime/aws-aio/config-MyCluster-XXXXXXXXXXXX-us-east-2-capture-v1.zip

(.venv) chelma@3c22fba4e266 aws-aio % ./manage_arkime.py config-pull --cluster-name MyCluster --capture --previous 2023-12-15 13:23:57 - Debug-level logs save to file: /Users/chelma/workspace/Arkime/aws-aio/manage_arkime/manage_arkime.log 2023-12-15 13:23:57 - Using AWS Credential Profile: None 2023-12-15 13:23:57 - Using AWS Region: default from AWS Config settings 2023-12-15 13:23:57 - Retrieving previous config bundle... 2023-12-15 13:23:57 - There was no previously deployed config; aborting...

(.venv) chelma@3c22fba4e266 aws-aio % ./manage_arkime.py config-pull --cluster-name MyCluster --viewer --previous 2023-12-15 13:24:22 - Debug-level logs save to file: /Users/chelma/workspace/Arkime/aws-aio/manage_arkime/manage_arkime.log 2023-12-15 13:24:22 - Using AWS Credential Profile: None 2023-12-15 13:24:22 - Using AWS Region: default from AWS Config settings 2023-12-15 13:24:22 - Retrieving previous config bundle... 2023-12-15 13:24:23 - Placed config bundle on disk: /Users/chelma/workspace/Arkime/aws-aio/config-MyCluster-XXXXXXXXXXXX-us-east-2-viewer-v5.zip

(.venv) chelma@3c22fba4e266 aws-aio % ./manage_arkime.py config-pull --cluster-name MyCluster --viewer --config-version 4 2023-12-15 13:24:55 - Debug-level logs save to file: /Users/chelma/workspace/Arkime/aws-aio/manage_arkime/manage_arkime.log 2023-12-15 13:24:55 - Using AWS Credential Profile: None 2023-12-15 13:24:55 - Using AWS Region: default from AWS Config settings 2023-12-15 13:24:55 - Retrieving version 4 config bundle... 2023-12-15 13:24:57 - Placed config bundle on disk: /Users/chelma/workspace/Arkime/aws-aio/config-MyCluster-XXXXXXXXXXXX-us-east-2-viewer-v4.zip

(.venv) chelma@3c22fba4e266 aws-aio % ./manage_arkime.py config-pull --cluster-name MyCluster --viewer --config-version 4 2023-12-15 13:24:59 - Debug-level logs save to file: /Users/chelma/workspace/Arkime/aws-aio/manage_arkime/manage_arkime.log 2023-12-15 13:24:59 - Using AWS Credential Profile: None 2023-12-15 13:24:59 - Using AWS Region: default from AWS Config settings 2023-12-15 13:24:59 - Retrieving version 4 config bundle... 2023-12-15 13:24:59 - Could not write to the location /Users/chelma/workspace/Arkime/aws-aio/config-MyCluster-XXXXXXXXXXXX-us-east-2-viewer-v4.zip because a file already exists there



## License
I confirm that this contribution is made under an Apache 2.0 license and that I have the authority necessary to make this contribution on behalf of its copyright owner.

awick commented 8 months ago

This works great, although I wish if I don't use --config-version that I could request both capture and viewer.

I also at some point hope we can add an option to build out the config dir (for those who aren't using a repo to store their config), but I know that is out of scope for the issue.

chelma commented 8 months ago

This works great, although I wish if I don't use --config-version that I could request both capture and viewer.

I also at some point hope we can add an option to build out the config dir (for those who aren't using a repo to store their config), but I know that is out of scope for the issue.

Good suggestions. I'd love to discuss what a unified experience across these config-* commands should look like, as a second set of eyes could help simplify things. I could be wrong, but I feel like there's a balancing act here of what make sense for a single command vs. what makes sense from a parallelism perspective (e.g. intuitive similarity across all the commands).

chelma commented 8 months ago

Captured the suggestions in https://github.com/arkime/aws-aio/issues/140

arkime / aws-aio