awslabs / aws-c-s3

C99 library implementation for communicating with the S3 service, designed for maximizing throughput on high bandwidth EC2 instances.
Apache License 2.0
94 stars 38 forks source link

Environment Detection #361

Closed waahm7 closed 10 months ago

waahm7 commented 10 months ago

Description of changes: Adds new APIs to retrieve information about the current environment and some preconfigured configurations for some EC2 instances. Currently, the CPU and CPU group are not utilized, but will be in the future. NIC information and syncing between preconfigured and runtime information are left as a to-do. Output on trn1n.32xlarge:

{
    'instance_type': 'trn1n.32xlarge',
    'max_throughput_gbps': 800,
    'has_recommended_configuration': true,
    'cpu_groups': [
    {
        'cpu_group_index': 0,
        'cpus_in_group': 64,
        'usable_network_devices': [
        ]
    },
    {
        'cpu_group_index': 1,
        'cpus_in_group': 64,
        'usable_network_devices': [
        ]
    }
    ]
}

Output on c5n.18xlarge:

{
    'instance_type': 'c5n.18xlarge',
    'max_throughput_gbps': 100,
    'has_recommended_configuration': false,
    'cpu_groups': [
    {
        'cpu_group_index': 0,
        'cpus_in_group': 36,
        'usable_network_devices': [
        ]
    },
    {
        'cpu_group_index': 1,
        'cpus_in_group': 36,
        'usable_network_devices': [
        ]
    }
    ]
}

Output on t2.large:

{
    'instance_type': '',
    'max_throughput_gbps': 0,
    'has_recommended_configuration': false,
    'cpu_groups': [
    {
        'cpu_group_index': 0,
        'cpus_in_group': 2,
        'usable_network_devices': [
        ]
    }
    ]
}

Changes I made from Jonathan's changes:

codecov-commenter commented 10 months ago

Codecov Report

Merging #361 (210fdc8) into main (24e1823) will decrease coverage by 1.34%. The diff coverage is 46.05%.

:exclamation: Current head 210fdc8 differs from pull request most recent head aba49d9. Consider uploading reports for the commit aba49d9 to get more accurate results

Additional details and impacted files [![Impacted file tree graph](https://app.codecov.io/gh/awslabs/aws-c-s3/pull/361/graphs/tree.svg?width=650&height=150&src=pr&token=J4KP54FVLF&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=awslabs)](https://app.codecov.io/gh/awslabs/aws-c-s3/pull/361?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=awslabs) ```diff @@ Coverage Diff @@ ## main #361 +/- ## ========================================== - Coverage 89.60% 88.27% -1.34% ========================================== Files 18 19 +1 Lines 4896 5039 +143 ========================================== + Hits 4387 4448 +61 - Misses 509 591 +82 ``` | [Files](https://app.codecov.io/gh/awslabs/aws-c-s3/pull/361?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=awslabs) | Coverage Δ | | |---|---|---| | [source/s3.c](https://app.codecov.io/gh/awslabs/aws-c-s3/pull/361?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=awslabs#diff-c291cmNlL3MzLmM=) | `95.23% <100.00%> (-0.92%)` | :arrow_down: | | [source/s3\_platform\_info.c](https://app.codecov.io/gh/awslabs/aws-c-s3/pull/361?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=awslabs#diff-c291cmNlL3MzX3BsYXRmb3JtX2luZm8uYw==) | `44.59% <44.59%> (ø)` | |