Unable to get the data for carbonfootprint.

[TarunYadavG]

I'm using the same process as mentioned in the project. I'm trying to employ a specific method from https://github.com/aws-samples/experimental-programmatic-access-ccft/blob/main/MultiAccountApplication/lambda-functions/3_backfill_data/ccft_access.py to retrieve carbon footprint data. However, after converting it to Java, I encountered the following exception.

I am making API calls from the child account, not the master one, and I'm unsure about what might be missing here. I have added the IAM policy to my account and verified the validity of the credentials by listing EC2 instances, from which I successfully obtained data, indicating that credentials are not the issue. Perhaps AWS is restricting access to retrieve the data via API, but I'm uncertain.

Furthermore, I can confirm that I am able to view the carbon footprint data for my account, indicating that the data does exist.

IAM policy

{   
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "sustainability:GetCarbonFootprintSummary",
            "Resource": "*"
        }
    ]
}

Snippet

public static void extractEmissionsData(String startDate, String endDate, AwsSessionCredentials sessionCredentials, String accountId) throws IOException {
        String billingRegion = "us-east-1";

        String awsFederatedSigninEndpoint = "https://signin.aws.amazon.com/federation";
        Map<String, String> signinTokenParams = new HashMap<>();
        signinTokenParams.put("Action", "getSigninToken");

        JsonObject sessionJson = new JsonObject();
        sessionJson.addProperty("sessionId", sessionCredentials.accessKeyId());
        sessionJson.addProperty("sessionKey", sessionCredentials.secretAccessKey());
        sessionJson.addProperty("sessionToken", sessionCredentials.sessionToken());

        signinTokenParams.put("Session", gson.toJson(sessionJson));

        String signinTokenUrl = awsFederatedSigninEndpoint + "?" + getParamsString(signinTokenParams);
        URL url = new URL(signinTokenUrl);
        HttpURLConnection con = (HttpURLConnection) url.openConnection();
        con.setRequestMethod("GET");

        if (con.getResponseCode() != HttpURLConnection.HTTP_OK) {
            throw new IOException("Server returned HTTP response code: " + con.getResponseCode() + " for URL: " + signinTokenUrl);
        }

        BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
        JsonObject signinTokenResponse = gson.fromJson(in, JsonObject.class);
        in.close();
        String signinToken = signinTokenResponse.get("SigninToken").getAsString();
        Map<String, String> loginParams = new HashMap<>();
        loginParams.put("Action", "login");
        loginParams.put("Destination", "https://console.aws.amazon.com/");
        loginParams.put("SigninToken", signinToken);

        String loginUrl = awsFederatedSigninEndpoint + "?" + getParamsString(loginParams);

        con = (HttpURLConnection) new URL(loginUrl).openConnection();
        con.setRequestMethod("GET");
        con.getInputStream().close();
        con = (HttpURLConnection) new URL("https://console.aws.amazon.com/billing/home?state=hashArgs").openConnection();
        con.setRequestMethod("GET");
        con.getInputStream().close();
        String xsrfToken = con.getHeaderField("x-awsbc-xsrf-token");
        JsonObject cftRequest = new JsonObject();
        JsonObject headers = new JsonObject();
        headers.addProperty("Content-Type", "application/json");
        cftRequest.add("headers", headers);
        cftRequest.addProperty("path", "/get-carbon-footprint-summary");
        cftRequest.addProperty("method", "GET");
        cftRequest.addProperty("region", billingRegion);
        JsonObject params = new JsonObject();
        params.addProperty("startDate", startDate);
        params.addProperty("endDate", endDate);
        cftRequest.add("params", params);

        Map<String, String> cftHeaders = new HashMap<>();
        cftHeaders.put("x-awsbc-xsrf-token", xsrfToken);

        url = new URL("https://" + billingRegion + ".console.aws.amazon.com/billing/rest/api-proxy/carbonfootprint");
        con = (HttpURLConnection) url.openConnection();
        con.setRequestMethod("POST");
        con.setRequestProperty("Content-Type", "application/json");
        con.setRequestProperty("x-awsbc-xsrf-token", xsrfToken);
        con.setDoOutput(true);
        try (OutputStreamWriter writer = new OutputStreamWriter(con.getOutputStream())) {
            writer.write(gson.toJson(cftRequest));
        }

        in = new BufferedReader(new InputStreamReader(con.getInputStream()));
        JsonObject emissionsResponse = gson.fromJson(in, JsonObject.class);
        in.close();

        JsonObject emissionsData = new JsonObject();
        emissionsData.addProperty("accountId", accountId);
        JsonObject query = new JsonObject();
        query.addProperty("queryDate", LocalDate.now().format(DateTimeFormatter.ofPattern("yyyy-MM-dd")));
        query.addProperty("startDate", startDate);
        query.addProperty("endDate", endDate);
        emissionsData.add("query", query);
        emissionsData.add("emissions", emissionsResponse);

        System.out.println(gson.toJson(emissionsData));
    }

Exception:

java.io.IOException: Server returned HTTP response code: 401 for URL: https://us-east-1.console.aws.amazon.com/billing/rest/api-proxy/carbonfootprint
    at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:2022)
    at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1611)
    at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:224)
    at websocket.ExtractEmissionsData.extractEmissionsData(ExtractEmissionsData.java:172)
    at websocket.ExtractEmissionsData.main(ExtractEmissionsData.java:98)

Could anyone provide assistance?

[TarunYadavG]

@ktj-ph can you help me with this?

[ktj-ph]

Hi @TarunYadavG , thanks for your interest! What is the need to convert this into Java? From the error code it seems like the client has not provided valid credentials or authorization. "Perhaps AWS is restricting access to retrieve the data via API" > I can confirm that this works with the provided python code, so I think this has more to do with how you convert this to Java & getting the right signin token.

[TarunYadavG]

@ktj-ph Okay, let me recheck and see if I made any mistakes on my end during the conversion. I thought AWS was blocking.

[martenpoferl]

Using TypeScript, I'm getting the same 401 error when requesting /billing/rest/api-proxy/carbonfootprint.

Due to the fact that we want our future projects to be in TypeScript, Python is not our primary language.

[martenpoferl]

I have found the solution: In the example, all requests are sent in a (requests.)Session. In my TS Lambda, I'm using axios for the requests. But in this case, every request happens in its own session. For TS with axios, I added tough-cookie and http-cookie-agent packages. With those I can create a session in my lambda, where every request via client stores the information like cookies in the session and the Lambda is authorized to request the carbon emissions.

const jar = new CookieJar();
const client = axios.create({
  httpAgent: new HttpCookieAgent({cookies: {jar}}),
  httpsAgent: new HttpsCookieAgent({cookies: {jar}}),
})

const loginResponse = await client.get(signinTokenUrl); // use the client instead of axios

@TarunYadavG: I guess this would be the fix for you too.