Added a basic config-list command to the CLI

[chelma]

Description

• Added a basic implementation of config-list. The default behavior is to retrieve the metadata of all config bundles the user has deployed. The user can get the currently (and most recent previously) deployed config using the --deployed flag.
• One thing to consider is what the "default" should be. I can see good argument to flipping what I've done around and making the default be the deployed config and have an --all flag to grab everything. I guess it comes down to how you think about a list-type call. IMO, the default should be "give me everything" and flags are used to filter, even if I suspect most users will use a flag most of the time.

Tasks

• https://github.com/arkime/aws-aio/issues/139

Testing

• Added unit tests
• Ran manually against my AWS account, with all options. Here's a few runs.

(.venv) chelma@3c22fba4e266 aws-aio % ./manage_arkime.py config-list --cluster-name MyCluster --viewer --deployed
2023-12-12 14:06:48 - Debug-level logs save to file: /Users/chelma/workspace/Arkime/aws-aio/manage_arkime/manage_arkime.log
2023-12-12 14:06:48 - Using AWS Credential Profile: None
2023-12-12 14:06:48 - Using AWS Region: default from AWS Config settings
2023-12-12 14:06:48 - Retrieving config details...
2023-12-12 14:06:48 - Config Details:
{
    "current": {
        "s3": {
            "bucket": "arkimeconfig-968674222892-us-east-2-mycluster",
            "key": "viewer/6/archive.zip"
        },
        "version": {
            "aws_aio_version": "1",
            "config_version": "6",
            "md5_version": "1a3e8232c093b419d590c0709d523759",
            "source_version": "v0.1.1-62-gd833c35",
            "time_utc": "2023-09-22 18:58:36"
        }
    },
    "previous": {
        "s3": {
            "bucket": "arkimeconfig-968674222892-us-east-2-mycluster",
            "key": "viewer/5/archive.zip"
        },
        "version": {
            "aws_aio_version": "1",
            "config_version": "5",
            "md5_version": "60b50c3ebdd8b02641d5d412f46519a3",
            "source_version": "v0.1.1-62-gd833c35",
            "time_utc": "2023-09-22 18:44:02"
        }
    }
}

(.venv) chelma@3c22fba4e266 aws-aio % ./manage_arkime.py config-list --cluster-name MyCluster --viewer
2023-12-12 13:52:04 - Debug-level logs save to file: /Users/chelma/workspace/Arkime/aws-aio/manage_arkime/manage_arkime.log
2023-12-12 13:52:04 - Using AWS Credential Profile: None
2023-12-12 13:52:04 - Using AWS Region: default from AWS Config settings
2023-12-12 13:52:04 - Retrieving config details (may take a bit)...
2023-12-12 13:52:07 - Config Details:
[
    {
        "s3": {
            "bucket": "arkimeconfig-XXXXXXXXXXXX-us-east-2-mycluster",
            "key": "viewer/6/archive.zip"
        },
        "version": {
            "aws_aio_version": "1",
            "config_version": "6",
            "md5_version": "1a3e8232c093b419d590c0709d523759",
            "source_version": "v0.1.1-62-gd833c35",
            "time_utc": "2023-09-22 18:58:36"
        }
    },
    {
        "s3": {
            "bucket": "arkimeconfig-XXXXXXXXXXXX-us-east-2-mycluster",
            "key": "viewer/5/archive.zip"
        },
        "version": {
            "aws_aio_version": "1",
            "config_version": "5",
            "md5_version": "60b50c3ebdd8b02641d5d412f46519a3",
            "source_version": "v0.1.1-62-gd833c35",
            "time_utc": "2023-09-22 18:44:02"
        }
    },
    {
        "s3": {
            "bucket": "arkimeconfig-XXXXXXXXXXXX-us-east-2-mycluster",
            "key": "viewer/4/archive.zip"
        },
        "version": {
            "aws_aio_version": "1",
            "config_version": "4",
            "md5_version": "d89f1396c8ccb54ec8c1db15ed8e51a8",
            "source_version": "v0.1.1-62-g8ac9e44",
            "time_utc": "2023-09-22 18:00:02"
        }
    },
    {
        "s3": {
            "bucket": "arkimeconfig-XXXXXXXXXXXX-us-east-2-mycluster",
            "key": "viewer/3/archive.zip"
        },
        "version": {
            "aws_aio_version": "1",
            "config_version": "3",
            "md5_version": "aa6cb798d96b86edf5e5d4560c0751b4",
            "source_version": "v0.1.1-62-g8ac9e44",
            "time_utc": "2023-09-22 17:56:00"
        }
    },
    {
        "s3": {
            "bucket": "arkimeconfig-XXXXXXXXXXXX-us-east-2-mycluster",
            "key": "viewer/2/archive.zip"
        },
        "version": {
            "aws_aio_version": "1",
            "config_version": "2",
            "md5_version": "640321fdab99767578cfbdb75fedcb1d",
            "source_version": "v0.1.1-62-g8ac9e44",
            "time_utc": "2023-09-22 17:25:37"
        }
    },
    {
        "s3": {
            "bucket": "arkimeconfig-XXXXXXXXXXXX-us-east-2-mycluster",
            "key": "viewer/1/archive.zip"
        },
        "version": {
            "aws_aio_version": "1",
            "config_version": "1",
            "md5_version": "1c42f3c8f3b70abe58afac52df89ba15",
            "source_version": "v0.1.1-62-g8ac9e44",
            "time_utc": "2023-09-22 16:05:28"
        }
    }
]

License

I confirm that this contribution is made under an Apache 2.0 license and that I have the authority necessary to make this contribution on behalf of its copyright owner.

[chelma]

Ooof, looks like I forgot to run the linter before pushing 😛

[awick]

Oh might be nice to call this from manage_arkime.yml, don't have to check the results, just so it doesn't error out

[chelma]

At some point I might add a --human option instead of json

Added this to the list of future features in https://github.com/arkime/aws-aio/issues/140. Good suggestion!

Oh might be nice to call this from manage_arkime.yml, don't have to check the results, just so it doesn't error out

I didn't follow this one; mind re-phrasing or expanding a bit?

[awick]

a test to .github/workflows/manage_arkime.yml

[chelma]

a test to .github/workflows/manage_arkime.yml

Ah! Yeah, that makes sense. Still getting integrated w/ the CI stuff you worked into the repo. Will do.