Add missing aws instances

[JacobValdemar]

I have created a Go program to add missing instances to aws.csv. I have documented it. I have run it.

As far as I can tell, this actually makes the file contain 100% of instances available on AWS.

Please cross/double check the added instances in aws.csv and see if there are any systematic errors or some sort of misunderstanding that makes the output wrong. If you find anything wrong I will try to fix the issues (but not manual fixes, I try to automate as much as possible when we are working with data in this scale).

Fixes #232

[da-ekchajzer]

Thank you for your efficiency. I'll try to review it as soon as possible. I'll probably move the script around if that's ok with you.

[JacobValdemar]

@da-ekchajzer No problem you move it around. I just placed it in the hack folder because that is what I usually see OS projects do :)

[JacobValdemar]

@da-ekchajzer I attempt to calculate CPU.units and CPU.core_units now (d0ae9e2). One thing I find mysterious is that the API seems to complete data that it actually has 😳

Let's use c7i.12xlarge as an example.

It has this row:

c7i.12xlarge,AWS,rack,,48,192,6,,Xeon Platinum 6455B,Intel,xeon platinum,Sapphire Rapids,,,96,32,12,0,0,0,0,,,,,2;2;2,2.99;1;5,4,50;0;100,1,35040,0.33;0.2;0.6,0,RAM.capacity not verified

However, when the API is probed, we get the following response:

JSON Response

```json { "impacts": { "gwp": { "embedded": { "value": 0.0088, "min": 0.005305, "max": 0.01426, "warnings": [ "End of life is not included in the calculation" ] }, "use": { "value": 0.202, "min": 0.1819, "max": 0.2426 }, "unit": "kgCO2eq", "description": "Total climate change" } }, "verbose": { "duration": { "value": 1.0, "unit": "hours" }, "ASSEMBLY-1": { "impacts": { "gwp": { "embedded": { "value": 0.0001906, "min": 0.0001906, "max": 0.0001906, "warnings": [ "End of life is not included in the calculation" ] }, "use": "not implemented", "unit": "kgCO2eq", "description": "Total climate change" } }, "units": { "value": 1, "status": "ARCHETYPE", "min": 1, "max": 1 }, "duration": { "value": 1.0, "unit": "hours" } }, "CPU-1": { "impacts": { "gwp": { "embedded": { "value": 0.003367, "min": 0.003367, "max": 0.003367, "warnings": [ "End of life is not included in the calculation" ] }, "use": { "value": 0.5645, "min": 0.5645, "max": 0.5645 }, "unit": "kgCO2eq", "description": "Total climate change" } }, "units": { "value": 6.0, "status": "ARCHETYPE", "min": 6.0, "max": 6.0 }, "core_units": { "value": 18, "status": "COMPLETED", "source": "Completed from name name based on https://docs.google.com/spreadsheets/d/1DqYgQnEDLQVQm5acMAhLgHLD8xXCG9BIrk-_Nv6jF3k/edit#gid=224728652.", "min": 18, "max": 18 }, "die_size": { "value": 485.0, "status": "COMPLETED", "unit": "mm2", "source": "Average value of Skylake with 18 cores", "min": 485.0, "max": 485.0 }, "model_range": { "value": "Xeon Platinum", "status": "COMPLETED", "source": "Completed from name name based on https://docs.google.com/spreadsheets/d/1DqYgQnEDLQVQm5acMAhLgHLD8xXCG9BIrk-_Nv6jF3k/edit#gid=224728652.", "min": "Xeon Platinum", "max": "Xeon Platinum" }, "manufacturer": { "value": "Intel", "status": "COMPLETED", "source": "Completed from name name based on https://docs.google.com/spreadsheets/d/1DqYgQnEDLQVQm5acMAhLgHLD8xXCG9BIrk-_Nv6jF3k/edit#gid=224728652.", "min": "Intel", "max": "Intel" }, "family": { "value": "Skylake", "status": "COMPLETED", "source": "Completed from name name based on https://docs.google.com/spreadsheets/d/1DqYgQnEDLQVQm5acMAhLgHLD8xXCG9BIrk-_Nv6jF3k/edit#gid=224728652.", "min": "Skylake", "max": "Skylake" }, "name": { "value": "Intel Xeon Platinum 8124M", "status": "COMPLETED", "source": "fuzzy match", "min": "Intel Xeon Platinum 8124M", "max": "Intel Xeon Platinum 8124M" }, "tdp": { "value": 240, "status": "COMPLETED", "unit": "W", "source": "Completed from name name based on https://docs.google.com/spreadsheets/d/1DqYgQnEDLQVQm5acMAhLgHLD8xXCG9BIrk-_Nv6jF3k/edit#gid=224728652.", "min": 240, "max": 240 }, "duration": { "value": 1.0, "unit": "hours" }, "avg_power": { "value": 245.01421745712904, "status": "COMPLETED", "unit": "W", "min": 245.01421745712904, "max": 245.01421745712904, "warnings": [ "value for one cpu unit" ] }, "time_workload": { "value": [ { "time_percentage": 100.0, "load_percentage": 100.0 } ], "status": "INPUT", "unit": "%" }, "usage_location": { "value": "IRL", "status": "INPUT", "unit": "CodSP3 - NCS Country Codes - NATO" }, "use_time_ratio": { "value": 1.0, "status": "ARCHETYPE", "unit": "/1", "min": 1.0, "max": 1.0 }, "hours_life_time": { "value": 35040.0, "status": "COMPLETED", "unit": "hours", "source": "from device", "min": 35040.0, "max": 35040.0 }, "workloads": { "value": [ { "load_percentage": 0, "power_watt": 28.799999999999997 }, { "load_percentage": 10, "power_watt": 76.8 }, { "load_percentage": 50, "power_watt": 180.0 }, { "load_percentage": 100, "power_watt": 244.8 } ], "status": "COMPLETED", "unit": "workload_rate:W" }, "params": { "value": { "a": 122.0350413339143, "b": 0.06408625230562343, "c": 20.451103048159847, "d": -4.39105124571944 }, "status": "COMPLETED", "source": "From TDP" }, "gwp_factor": { "value": 0.384, "status": "COMPLETED", "unit": "kg CO2eq/kWh", "source": "https://www.sciencedirect.com/science/article/pii/S0306261921012149", "min": 0.384, "max": 0.384 } }, "RAM-1": { "impacts": { "gwp": { "embedded": { "value": 0.021, "min": 0.01194, "max": 0.04036, "warnings": [ "End of life is not included in the calculation" ] }, "use": { "value": 0.04188, "min": 0.04188, "max": 0.04188 }, "unit": "kgCO2eq", "description": "Total climate change" } }, "units": { "value": 12.0, "status": "ARCHETYPE", "min": 12.0, "max": 12.0 }, "capacity": { "value": 32.0, "status": "ARCHETYPE", "unit": "GB", "min": 32.0, "max": 32.0 }, "density": { "value": 1.2443636363636363, "status": "COMPLETED", "unit": "GB/cm2", "source": "Average of 11 rows", "min": 0.625, "max": 2.375 }, "duration": { "value": 1.0, "unit": "hours" }, "avg_power": { "value": 9.088, "status": "COMPLETED", "unit": "W", "min": 9.088, "max": 9.088, "warnings": [ "value for one ram strip" ] }, "time_workload": { "value": [ { "time_percentage": 100.0, "load_percentage": 100.0 } ], "status": "INPUT", "unit": "%" }, "usage_location": { "value": "IRL", "status": "INPUT", "unit": "CodSP3 - NCS Country Codes - NATO" }, "use_time_ratio": { "value": 1.0, "status": "ARCHETYPE", "unit": "/1", "min": 1.0, "max": 1.0 }, "hours_life_time": { "value": 35040.0, "status": "COMPLETED", "unit": "hours", "source": "from device", "min": 35040.0, "max": 35040.0 }, "params": { "value": { "a": 9.088 }, "status": "COMPLETED", "source": "(ram_electrical_factor_per_go : 0.284) * (ram_capacity: 32.0) " }, "gwp_factor": { "value": 0.384, "status": "COMPLETED", "unit": "kg CO2eq/kWh", "source": "https://www.sciencedirect.com/science/article/pii/S0306261921012149", "min": 0.384, "max": 0.384 } }, "SSD-1": { "impacts": { "gwp": { "embedded": { "value": 0.0, "min": 0.0, "max": 0.0, "warnings": [ "End of life is not included in the calculation", "Uncertainty from technical characteristics is very important. Results should be interpreted with caution (see min and max values)" ] }, "use": "not implemented", "unit": "kgCO2eq", "description": "Total climate change" } }, "units": { "value": 0.0, "status": "ARCHETYPE", "min": 0.0, "max": 0.0 }, "capacity": { "value": 0.0, "status": "ARCHETYPE", "unit": "GB", "min": 0.0, "max": 0.0 }, "density": { "value": 54.8842105263158, "status": "COMPLETED", "unit": "GB/cm2", "source": "Average of 19 rows", "min": 16.4, "max": 128.0 }, "duration": { "value": 1.0, "unit": "hours" } }, "POWER_SUPPLY-1": { "impacts": { "gwp": { "embedded": { "value": 0.0041, "min": 0.001387, "max": 0.006935, "warnings": [ "End of life is not included in the calculation" ] }, "use": "not implemented", "unit": "kgCO2eq", "description": "Total climate change" } }, "units": { "value": 2.0, "status": "ARCHETYPE", "min": 2.0, "max": 2.0 }, "unit_weight": { "value": 2.99, "status": "ARCHETYPE", "unit": "kg", "min": 1.0, "max": 5.0 }, "duration": { "value": 1.0, "unit": "hours" } }, "CASE-1": { "impacts": { "gwp": { "embedded": { "value": 0.0043, "min": 0.002451, "max": 0.004281, "warnings": [ "End of life is not included in the calculation" ] }, "use": "not implemented", "unit": "kgCO2eq", "description": "Total climate change" } }, "units": { "value": 1, "status": "ARCHETYPE", "min": 1, "max": 1 }, "case_type": { "value": "rack", "status": "ARCHETYPE" }, "duration": { "value": 1.0, "unit": "hours" } }, "MOTHERBOARD-1": { "impacts": { "gwp": { "embedded": { "value": 0.001886, "min": 0.001886, "max": 0.001886, "warnings": [ "End of life is not included in the calculation" ] }, "use": "not implemented", "unit": "kgCO2eq", "description": "Total climate change" } }, "units": { "value": 1, "status": "ARCHETYPE", "min": 1, "max": 1 }, "duration": { "value": 1.0, "unit": "hours" } }, "avg_power": { "value": 2100.22428, "status": "COMPLETED", "unit": "W", "min": 1894.9391999999998, "max": 2526.5856000000003 }, "time_workload": { "value": [ { "time_percentage": 100.0, "load_percentage": 100.0 } ], "status": "INPUT", "unit": "%" }, "usage_location": { "value": "IRL", "status": "INPUT", "unit": "CodSP3 - NCS Country Codes - NATO" }, "use_time_ratio": { "value": 1.0, "status": "ARCHETYPE", "unit": "/1", "min": 1.0, "max": 1.0 }, "hours_life_time": { "value": 35040.0, "status": "ARCHETYPE", "unit": "hours", "min": 35040.0, "max": 35040.0 }, "other_consumption_ratio": { "value": 0.33, "status": "ARCHETYPE", "unit": "ratio /1", "min": 0.2, "max": 0.6 }, "instance_per_server": { "value": 4.0, "status": "ARCHETYPE", "min": 4.0, "max": 4.0 }, "gwp_factor": { "value": 0.384, "status": "COMPLETED", "unit": "kg CO2eq/kWh", "source": "https://www.sciencedirect.com/science/article/pii/S0306261921012149", "min": 0.384, "max": 0.384 }, "units": { "value": 1, "status": "ARCHETYPE", "min": 1, "max": 1 } } } ```

Notice how CPU.name and CPU.family is clearly completed although it is set in aws.csv. It might just be me not understanding completion right? I have tried to read the docs, but I would have expected that it at least completed to something from the right CPU.family which it didn't.

[da-ekchajzer]

CPU Completion

I wasn't clear enough in my explanations about completion. The API will first search cpu_specs.csv when given a CPU name. In your case, the API will retrieve the name and fuzzymatch the cpu in cpu_specs. Because cpu_specs.csv doesn't have, Xeon Platinum 6455B he will match an Intel Xeon Platinum 8124M. Matching sensitivity can be specified in the conf file.

You have two ways around this:

1. Don't enter the CPU name, just its characteristics and they will be used
2. Simply fill in the name and add an entry in cpu_specs if the cpu isn't there.

I prefer the second option, as everyone will be able to benefit from the addition of a CPU (especially if a new cloud provider is added, the contributor would only have to enter the name).

[da-ekchajzer]

Platform

Concerning the notion of "platform" I also told you some incomplete information. When a "metal" instance exists for an instance type, we can indeed consider this data as describing the "platform". When no "metal" instances exist, it's not correct to take the largest instance.

Maybe @github-benjamin-davy, who produced the first file, will be able to give you more details on how he did it, especially for :

• total RAM
• total vCPU

In the meantime you can check he's article : https://medium.com/teads-engineering/building-an-aws-ec2-carbon-emissions-dataset-3f0fd76c98ac and dataset : https://docs.google.com/spreadsheets/d/1DqYgQnEDLQVQm5acMAhLgHLD8xXCG9BIrk-_Nv6jF3k/edit

[JacobValdemar]

@da-ekchajzer

Regarding CPU Completion, I have added CPUs to cpu_spec.csv where I have enough information about the CPU (2780467).

Regarding the definition of Platform: I have read the blog post and visited the dataset, and I understand the current method is not correct. However, I still can't come up with better method to estimate the true platforms.

[da-ekchajzer]

Thanks for this update. I had a chat with @github-benjamin-davy who should look at the dataset you produced soon and give some help on how to retrieve platform info.

[JacobValdemar]

@da-ekchajzer I don't understand why in some cases the estimated GWP for two seemingly similar instance types can be so different. For example see the response for a1.xlarge and c7i.xlarge which both have 4 vCPU and 8 GiB memory. Why are the difference so large? And can it be true/correct or could it be caused by an error in the generation of new instance specs?

a1.xlarge

[a1.xlarge spec on vantage.sh](https://instances.vantage.sh/aws/ec2/a1.xlarge?region=eu-west-1&os=linux&cost_duration=hourly&reserved_term=Standard.noUpfront) POST `/v1/cloud/instance?verbose=true&duration=1&criteria=gwp` Request body: ```json { "provider": "aws", "instance_type": "a1.xlarge", "usage": { "usage_location": "IRL", "time_workload": [ { "time_percentage": 100, "load_percentage": 100 } ] } } ``` Response: ```json { "impacts": { "gwp": { "embedded": { "value": 0.0033, "min": 0.001866, "max": 0.004542, "warnings": [ "End of life is not included in the calculation" ] }, "use": { "value": 0.0064, "min": 0.005751, "max": 0.007668 }, "unit": "kgCO2eq", "description": "Total climate change" } }, "verbose": { "duration": { "value": 1.0, "unit": "hours" }, "ASSEMBLY-1": { "impacts": { "gwp": { "embedded": { "value": 0.0001906, "min": 0.0001906, "max": 0.0001906, "warnings": [ "End of life is not included in the calculation" ] }, "use": "not implemented", "unit": "kgCO2eq", "description": "Total climate change" } }, "units": { "value": 1, "status": "ARCHETYPE", "min": 1, "max": 1 }, "duration": { "value": 1.0, "unit": "hours" } }, "CPU-1": { "impacts": { "gwp": { "embedded": { "value": 0.00063, "min": 0.0004037, "max": 0.001362, "warnings": [ "End of life is not included in the calculation" ] }, "use": { "value": 0.01568, "min": 0.01568, "max": 0.01568 }, "unit": "kgCO2eq", "description": "Total climate change" } }, "units": { "value": 1.0, "status": "ARCHETYPE", "min": 1.0, "max": 1.0 }, "core_units": { "value": 16, "status": "COMPLETED", "source": "Completed from name name based on https://docs.google.com/spreadsheets/d/1DqYgQnEDLQVQm5acMAhLgHLD8xXCG9BIrk-_Nv6jF3k/edit#gid=224728652.", "min": 16, "max": 16 }, "die_size": { "value": 609.0, "status": "COMPLETED", "unit": "mm2", "source": "Average value of all families with 16 cores", "min": 205.0, "max": 1910.0 }, "model_range": { "value": "Graviton", "status": "COMPLETED", "source": "Completed from name name based on https://docs.google.com/spreadsheets/d/1DqYgQnEDLQVQm5acMAhLgHLD8xXCG9BIrk-_Nv6jF3k/edit#gid=224728652.", "min": "Graviton", "max": "Graviton" }, "manufacturer": { "value": "Annapurna Labs", "status": "COMPLETED", "source": "Completed from name name based on https://docs.google.com/spreadsheets/d/1DqYgQnEDLQVQm5acMAhLgHLD8xXCG9BIrk-_Nv6jF3k/edit#gid=224728652.", "min": "Annapurna Labs", "max": "Annapurna Labs" }, "family": { "value": "Graviton", "status": "COMPLETED", "source": "Completed from name name based on https://docs.google.com/spreadsheets/d/1DqYgQnEDLQVQm5acMAhLgHLD8xXCG9BIrk-_Nv6jF3k/edit#gid=224728652.", "min": "Graviton", "max": "Graviton" }, "name": { "value": "Annapurna Labs Graviton", "status": "COMPLETED", "source": "fuzzy match", "min": "Annapurna Labs Graviton", "max": "Annapurna Labs Graviton" }, "tdp": { "value": 40, "status": "COMPLETED", "unit": "W", "source": "Completed from name name based on https://docs.google.com/spreadsheets/d/1DqYgQnEDLQVQm5acMAhLgHLD8xXCG9BIrk-_Nv6jF3k/edit#gid=224728652.", "min": 40, "max": 40 }, "duration": { "value": 1.0, "unit": "hours" }, "avg_power": { "value": 40.83570291686707, "status": "COMPLETED", "unit": "W", "min": 40.83570291686707, "max": 40.83570291686707, "warnings": [ "value for one cpu unit" ] }, "time_workload": { "value": [ { "time_percentage": 100.0, "load_percentage": 100.0 } ], "status": "INPUT", "unit": "%" }, "usage_location": { "value": "IRL", "status": "INPUT", "unit": "CodSP3 - NCS Country Codes - NATO" }, "use_time_ratio": { "value": 1.0, "status": "ARCHETYPE", "unit": "/1", "min": 1.0, "max": 1.0 }, "hours_life_time": { "value": 35040.0, "status": "COMPLETED", "unit": "hours", "source": "from device", "min": 35040.0, "max": 35040.0 }, "workloads": { "value": [ { "load_percentage": 0, "power_watt": 4.8 }, { "load_percentage": 10, "power_watt": 12.8 }, { "load_percentage": 50, "power_watt": 30.0 }, { "load_percentage": 100, "power_watt": 40.8 } ], "status": "COMPLETED", "unit": "workload_rate:W" }, "params": { "value": { "a": 20.33917361602456, "b": 0.07222465114793339, "c": 20.451103179493458, "d": -3.1634217024047846 }, "status": "COMPLETED", "source": "From TDP" }, "gwp_factor": { "value": 0.384, "status": "COMPLETED", "unit": "kg CO2eq/kWh", "source": "https://www.sciencedirect.com/science/article/pii/S0306261921012149", "min": 0.384, "max": 0.384 } }, "RAM-1": { "impacts": { "gwp": { "embedded": { "value": 0.0019, "min": 0.001144, "max": 0.003513, "warnings": [ "End of life is not included in the calculation" ] }, "use": { "value": 0.00349, "min": 0.00349, "max": 0.00349 }, "unit": "kgCO2eq", "description": "Total climate change" } }, "units": { "value": 2.0, "status": "ARCHETYPE", "min": 2.0, "max": 2.0 }, "capacity": { "value": 16.0, "status": "ARCHETYPE", "unit": "GB", "min": 16.0, "max": 16.0 }, "density": { "value": 1.2443636363636363, "status": "COMPLETED", "unit": "GB/cm2", "source": "Average of 11 rows", "min": 0.625, "max": 2.375 }, "duration": { "value": 1.0, "unit": "hours" }, "avg_power": { "value": 4.544, "status": "COMPLETED", "unit": "W", "min": 4.544, "max": 4.544, "warnings": [ "value for one ram strip" ] }, "time_workload": { "value": [ { "time_percentage": 100.0, "load_percentage": 100.0 } ], "status": "INPUT", "unit": "%" }, "usage_location": { "value": "IRL", "status": "INPUT", "unit": "CodSP3 - NCS Country Codes - NATO" }, "use_time_ratio": { "value": 1.0, "status": "ARCHETYPE", "unit": "/1", "min": 1.0, "max": 1.0 }, "hours_life_time": { "value": 35040.0, "status": "COMPLETED", "unit": "hours", "source": "from device", "min": 35040.0, "max": 35040.0 }, "params": { "value": { "a": 4.544 }, "status": "COMPLETED", "source": "(ram_electrical_factor_per_go : 0.284) * (ram_capacity: 16.0) " }, "gwp_factor": { "value": 0.384, "status": "COMPLETED", "unit": "kg CO2eq/kWh", "source": "https://www.sciencedirect.com/science/article/pii/S0306261921012149", "min": 0.384, "max": 0.384 } }, "SSD-1": { "impacts": { "gwp": { "embedded": { "value": 0.0, "min": 0.0, "max": 0.0, "warnings": [ "End of life is not included in the calculation", "Uncertainty from technical characteristics is very important. Results should be interpreted with caution (see min and max values)" ] }, "use": "not implemented", "unit": "kgCO2eq", "description": "Total climate change" } }, "units": { "value": 0.0, "status": "ARCHETYPE", "min": 0.0, "max": 0.0 }, "capacity": { "value": 0.0, "status": "ARCHETYPE", "unit": "GB", "min": 0.0, "max": 0.0 }, "density": { "value": 54.8842105263158, "status": "COMPLETED", "unit": "GB/cm2", "source": "Average of 19 rows", "min": 16.4, "max": 128.0 }, "duration": { "value": 1.0, "unit": "hours" } }, "POWER_SUPPLY-1": { "impacts": { "gwp": { "embedded": { "value": 0.0041, "min": 0.001387, "max": 0.006935, "warnings": [ "End of life is not included in the calculation" ] }, "use": "not implemented", "unit": "kgCO2eq", "description": "Total climate change" } }, "units": { "value": 2.0, "status": "ARCHETYPE", "min": 2.0, "max": 2.0 }, "unit_weight": { "value": 2.99, "status": "ARCHETYPE", "unit": "kg", "min": 1.0, "max": 5.0 }, "duration": { "value": 1.0, "unit": "hours" } }, "CASE-1": { "impacts": { "gwp": { "embedded": { "value": 0.0043, "min": 0.002451, "max": 0.004281, "warnings": [ "End of life is not included in the calculation" ] }, "use": "not implemented", "unit": "kgCO2eq", "description": "Total climate change" } }, "units": { "value": 1, "status": "ARCHETYPE", "min": 1, "max": 1 }, "case_type": { "value": "rack", "status": "ARCHETYPE" }, "duration": { "value": 1.0, "unit": "hours" } }, "MOTHERBOARD-1": { "impacts": { "gwp": { "embedded": { "value": 0.001886, "min": 0.001886, "max": 0.001886, "warnings": [ "End of life is not included in the calculation" ] }, "use": "not implemented", "unit": "kgCO2eq", "description": "Total climate change" } }, "units": { "value": 1, "status": "ARCHETYPE", "min": 1, "max": 1 }, "duration": { "value": 1.0, "unit": "hours" } }, "avg_power": { "value": 66.39892, "status": "COMPLETED", "unit": "W", "min": 59.9088, "max": 79.8784 }, "time_workload": { "value": [ { "time_percentage": 100.0, "load_percentage": 100.0 } ], "status": "INPUT", "unit": "%" }, "usage_location": { "value": "IRL", "status": "INPUT", "unit": "CodSP3 - NCS Country Codes - NATO" }, "use_time_ratio": { "value": 1.0, "status": "ARCHETYPE", "unit": "/1", "min": 1.0, "max": 1.0 }, "hours_life_time": { "value": 35040.0, "status": "ARCHETYPE", "unit": "hours", "min": 35040.0, "max": 35040.0 }, "other_consumption_ratio": { "value": 0.33, "status": "ARCHETYPE", "unit": "ratio /1", "min": 0.2, "max": 0.6 }, "instance_per_server": { "value": 4.0, "status": "ARCHETYPE", "min": 4.0, "max": 4.0 }, "gwp_factor": { "value": 0.384, "status": "COMPLETED", "unit": "kg CO2eq/kWh", "source": "https://www.sciencedirect.com/science/article/pii/S0306261921012149", "min": 0.384, "max": 0.384 }, "units": { "value": 1, "status": "ARCHETYPE", "min": 1, "max": 1 } } } ```

c7i.xlarge

[c7i.xlarge spec on vantage.sh](https://instances.vantage.sh/aws/ec2/c7i.xlarge?region=eu-west-1&os=linux&cost_duration=hourly&reserved_term=Standard.noUpfront) POST `/v1/cloud/instance?verbose=true&duration=1&criteria=gwp` Request body: ```json { "provider": "aws", "instance_type": "c7i.xlarge", "usage": { "usage_location": "IRL", "time_workload": [ { "time_percentage": 100, "load_percentage": 100 } ] } } ``` Response: ```json { "impacts": { "gwp": { "embedded": { "value": 0.00069, "min": 0.0004004, "max": 0.001146, "warnings": [ "End of life is not included in the calculation" ] }, "use": { "value": 0.0039, "min": 0.003543, "max": 0.004724 }, "unit": "kgCO2eq", "description": "Total climate change" } }, "verbose": { "duration": { "value": 1.0, "unit": "hours" }, "ASSEMBLY-1": { "impacts": { "gwp": { "embedded": { "value": 0.0001906, "min": 0.0001906, "max": 0.0001906, "warnings": [ "End of life is not included in the calculation" ] }, "use": "not implemented", "unit": "kgCO2eq", "description": "Total climate change" } }, "units": { "value": 1, "status": "ARCHETYPE", "min": 1, "max": 1 }, "duration": { "value": 1.0, "unit": "hours" } }, "CPU-1": { "impacts": { "gwp": { "embedded": { "value": 0.001362, "min": 0.001362, "max": 0.001362, "warnings": [ "End of life is not included in the calculation" ] }, "use": { "value": 0.09985, "min": 0.09985, "max": 0.09985 }, "unit": "kgCO2eq", "description": "Total climate change" } }, "units": { "value": 1, "status": "ARCHETYPE", "min": 1, "max": 1 }, "die_size": { "value": 1910.0, "status": "COMPLETED", "unit": "mm2", "source": "Average value for Sapphire Rapids", "min": 1910.0, "max": 1910.0 }, "model_range": { "value": "xeon platinum", "status": "ARCHETYPE" }, "manufacturer": { "value": "Intel", "status": "ARCHETYPE" }, "family": { "value": "Sapphire Rapids", "status": "ARCHETYPE" }, "duration": { "value": 1.0, "unit": "hours" }, "avg_power": { "value": 260.02009570521255, "status": "COMPLETED", "unit": "W", "min": 260.02009570521255, "max": 260.02009570521255, "warnings": [ "value for one cpu unit" ] }, "time_workload": { "value": [ { "time_percentage": 100.0, "load_percentage": 100.0 } ], "status": "INPUT", "unit": "%" }, "usage_location": { "value": "IRL", "status": "INPUT", "unit": "CodSP3 - NCS Country Codes - NATO" }, "use_time_ratio": { "value": 1.0, "status": "ARCHETYPE", "unit": "/1", "min": 1.0, "max": 1.0 }, "hours_life_time": { "value": 35040.0, "status": "COMPLETED", "unit": "hours", "source": "from device", "min": 35040.0, "max": 35040.0 }, "params": { "value": { "a": 171.1813, "b": 0.0354, "c": 36.8953, "d": -10.1336 }, "status": "COMPLETED", "source": "From CPU model range" }, "gwp_factor": { "value": 0.384, "status": "COMPLETED", "unit": "kg CO2eq/kWh", "source": "https://www.sciencedirect.com/science/article/pii/S0306261921012149", "min": 0.384, "max": 0.384 } }, "RAM-1": { "impacts": { "gwp": { "embedded": { "value": 0.021, "min": 0.01194, "max": 0.04036, "warnings": [ "End of life is not included in the calculation" ] }, "use": { "value": 0.04188, "min": 0.04188, "max": 0.04188 }, "unit": "kgCO2eq", "description": "Total climate change" } }, "units": { "value": 12.0, "status": "ARCHETYPE", "min": 12.0, "max": 12.0 }, "capacity": { "value": 32.0, "status": "ARCHETYPE", "unit": "GB", "min": 32.0, "max": 32.0 }, "density": { "value": 1.2443636363636363, "status": "COMPLETED", "unit": "GB/cm2", "source": "Average of 11 rows", "min": 0.625, "max": 2.375 }, "duration": { "value": 1.0, "unit": "hours" }, "avg_power": { "value": 9.088, "status": "COMPLETED", "unit": "W", "min": 9.088, "max": 9.088, "warnings": [ "value for one ram strip" ] }, "time_workload": { "value": [ { "time_percentage": 100.0, "load_percentage": 100.0 } ], "status": "INPUT", "unit": "%" }, "usage_location": { "value": "IRL", "status": "INPUT", "unit": "CodSP3 - NCS Country Codes - NATO" }, "use_time_ratio": { "value": 1.0, "status": "ARCHETYPE", "unit": "/1", "min": 1.0, "max": 1.0 }, "hours_life_time": { "value": 35040.0, "status": "COMPLETED", "unit": "hours", "source": "from device", "min": 35040.0, "max": 35040.0 }, "params": { "value": { "a": 9.088 }, "status": "COMPLETED", "source": "(ram_electrical_factor_per_go : 0.284) * (ram_capacity: 32.0) " }, "gwp_factor": { "value": 0.384, "status": "COMPLETED", "unit": "kg CO2eq/kWh", "source": "https://www.sciencedirect.com/science/article/pii/S0306261921012149", "min": 0.384, "max": 0.384 } }, "SSD-1": { "impacts": { "gwp": { "embedded": { "value": 0.0, "min": 0.0, "max": 0.0, "warnings": [ "End of life is not included in the calculation", "Uncertainty from technical characteristics is very important. Results should be interpreted with caution (see min and max values)" ] }, "use": "not implemented", "unit": "kgCO2eq", "description": "Total climate change" } }, "units": { "value": 0.0, "status": "ARCHETYPE", "min": 0.0, "max": 0.0 }, "capacity": { "value": 0.0, "status": "ARCHETYPE", "unit": "GB", "min": 0.0, "max": 0.0 }, "density": { "value": 54.8842105263158, "status": "COMPLETED", "unit": "GB/cm2", "source": "Average of 19 rows", "min": 16.4, "max": 128.0 }, "duration": { "value": 1.0, "unit": "hours" } }, "POWER_SUPPLY-1": { "impacts": { "gwp": { "embedded": { "value": 0.0041, "min": 0.001387, "max": 0.006935, "warnings": [ "End of life is not included in the calculation" ] }, "use": "not implemented", "unit": "kgCO2eq", "description": "Total climate change" } }, "units": { "value": 2.0, "status": "ARCHETYPE", "min": 2.0, "max": 2.0 }, "unit_weight": { "value": 2.99, "status": "ARCHETYPE", "unit": "kg", "min": 1.0, "max": 5.0 }, "duration": { "value": 1.0, "unit": "hours" } }, "CASE-1": { "impacts": { "gwp": { "embedded": { "value": 0.0043, "min": 0.002451, "max": 0.004281, "warnings": [ "End of life is not included in the calculation" ] }, "use": "not implemented", "unit": "kgCO2eq", "description": "Total climate change" } }, "units": { "value": 1, "status": "ARCHETYPE", "min": 1, "max": 1 }, "case_type": { "value": "rack", "status": "ARCHETYPE" }, "duration": { "value": 1.0, "unit": "hours" } }, "MOTHERBOARD-1": { "impacts": { "gwp": { "embedded": { "value": 0.001886, "min": 0.001886, "max": 0.001886, "warnings": [ "End of life is not included in the calculation" ] }, "use": "not implemented", "unit": "kgCO2eq", "description": "Total climate change" } }, "units": { "value": 1, "status": "ARCHETYPE", "min": 1, "max": 1 }, "duration": { "value": 1.0, "unit": "hours" } }, "avg_power": { "value": 490.87108, "status": "COMPLETED", "unit": "W", "min": 442.89119999999997, "max": 590.5215999999999 }, "time_workload": { "value": [ { "time_percentage": 100.0, "load_percentage": 100.0 } ], "status": "INPUT", "unit": "%" }, "usage_location": { "value": "IRL", "status": "INPUT", "unit": "CodSP3 - NCS Country Codes - NATO" }, "use_time_ratio": { "value": 1.0, "status": "ARCHETYPE", "unit": "/1", "min": 1.0, "max": 1.0 }, "hours_life_time": { "value": 35040.0, "status": "ARCHETYPE", "unit": "hours", "min": 35040.0, "max": 35040.0 }, "other_consumption_ratio": { "value": 0.33, "status": "ARCHETYPE", "unit": "ratio /1", "min": 0.2, "max": 0.6 }, "instance_per_server": { "value": 48.0, "status": "ARCHETYPE", "min": 48.0, "max": 48.0 }, "gwp_factor": { "value": 0.384, "status": "COMPLETED", "unit": "kg CO2eq/kWh", "source": "https://www.sciencedirect.com/science/article/pii/S0306261921012149", "min": 0.384, "max": 0.384 }, "units": { "value": 1, "status": "ARCHETYPE", "min": 1, "max": 1 } } } ```

[github-benjamin-davy]

Hello @JacobValdemar and @da-ekchajzer I've reviewed the missing instances doc and modified / completed some of the fields (in yellow) in this Google Sheet i only looked at column A to M.

[JacobValdemar]

Thank you for your contribution @github-benjamin-davy! I have some questions about the changes:

• Why do you set Graviton 3's TDP to 220W? It has not been published by AWS, but people mostly speculate that the TDP is 100W: https://www.semianalysis.com/p/amazon-graviton-3-uses-chiplets-and https://www.nextplatform.com/2021/12/02/aws-goes-wide-and-deep-with-graviton3-server-chip/ https://min.news/en/tech/4deba013d8a09e928f65902948073b7e.html
• Why do you set Graviton 2's TDP to 150? Most online articles mention 80-110W: https://www.anandtech.com/show/15578/cloud-clash-amazon-graviton2-arm-against-intel-and-amd https://sonofsoil.medium.com/graviton2-on-fire-6da40ad8f18
• Do you have any data for Graviton3E that can be filled into crowdsourcing/cpu_specs.csv?
• In some cases you say AMD EPYC 9R14 has 280W cpu.TDP and in some cases you say it has 360W cpu.TDP. Only one can be correct, which one is it?
• How do you know AMD EPYC 9R14 has 48 cpu.core_units? And how to you know its TDP?

[github-benjamin-davy]

Hello @JacobValdemar, thanks for spotting this, I've fixed the doc for the 9R14, on a bare metal instance Turbostat indicates 280 Watts and lscpu a 96 cores CPU with one thread per core.

The assumptions for Graviton processors are based on ARM architecture documentation and are voluntarily conservative:

• Graviton 2: 150 Watts Estimated TDP based on ARM Neoverse N1 specifications
• Graviton 3: 220 Watts TDP estimated based on Neoverse V1 ARM presentation indicating Power-performance trade-off and +50% performance increase from Neoverse N1 architecture used by Graviton 2
• Graviton 3E is reported to be a higher-power version of Graviton3 so I would assume a 260 Watt TDP

I've listed the missing hardware platforms in the initial dataset

[da-ekchajzer]

@da-ekchajzer I don't understand why in some cases the estimated GWP for two seemingly similar instance types can be so different. For example see the response for a1.xlarge and c7i.xlarge which both have 4 vCPU and 8 GiB memory. Why are the difference so large? And can it be true/correct or could it be caused by an error in the generation of new instance specs? a1.xlarge c7i.xlarge

To answer this question. If you refer to the use impact, the main reason is the architecture of the CPU (ARM is more energy efficient). If you refer to the embodied impact, the main reason I see would be that a c7i.xlarge is way more mutualized (48 instances_per_server) than the a1.xlarge (4 instances_per_server). Thus, fixed machine costs (sockets, motherboard, powersupply, casing) are distributed over fewer instances. In principle the fixed costs of the ARM machine are overestimated, and inversely so for the Intel machine. To make this comparison more accurate, we'd need to verify the difference in fixed costs wich we are not able to do at the moment.

[JacobValdemar]

Hello again @github-benjamin-davy,

• Now you say Graviton3E has 260 Watt TDP, but in the aws missing instances spreadsheet you typed it as 250 Watt. Which one is it?

[github-benjamin-davy]

Hello @JacobValdemar that must be a typo, you can stick to 250 but again it's an assumption that could be debated :)

[JacobValdemar]

Alright then :) @da-ekchajzer I think this PR is ready 🚀

[da-ekchajzer]

Thank you both ! I'll merge and release on friday at the latest

[da-ekchajzer]

Could you "Allow edits from maintainers" on your fork ? I have some minor modification to make before merging.

[JacobValdemar]

Could you "Allow edits from maintainers" on your fork ? I have some minor modification to make before merging.

Sure, should be enabled now @da-ekchajzer

[da-ekchajzer]

I still have a permission denied :

To github.com:JacobValdemar/boaviztapi.git ! [remote rejected] JacobValdemar-add-aws-instances -> JacobValdemar-add-aws-instances (permission denied) error: failed to push some refs to 'github.com:JacobValdemar/boaviztapi.git'

[JacobValdemar]

You mean this thing right?

Maybe it thinks you are not a maintainer?:

[da-ekchajzer]

Everything seems in order ! I am merging and releasing. Thank you, @JacobValdemar and @github-benjamin-davy for your contributions.