Response is wrong json format resulting in conversion error

[lxorc]

using from yarn_api_client import ResourceManager to call the cluster_applications method, the returned json format is an error and an error is reported.

{
    "apps":{
        "app":[
            {
                "id":"application_1658305609026_207118",
                "user":"hadoop",
                "name":"Hive on Spark",
                "queue":"default",
                "state":"RUNNING",
                "finalStatus":"UNDEFINED",
                "progress":10,
                "trackingUI":"ApplicationMaster",
                "trackingUrl":"http://172.24.8.137:5004/proxy/application_1658305609026_207118/",
                "diagnostics":"",
                "clusterId":1658305609026,
                "applicationType":"SPARK",
                "applicationTags":"",
                "priority":0,
                "startedTime":1661170556597,
                "finishedTime":0,
                "elapsedTime":2666541,
                "amContainerLogs":"http://172.24.8.96:5008/node/containerlogs/container_e34_1658305609026_207118_01_000001/hadoop",
                "amHostHttpAddress":"172.24.8.96:5008",
                "amRPCAddress":"172.24.8.96:34473",
                "allocatedMB":21504,
                "allocatedVCores":5,
                "runningContainers":2,
                "memorySeconds":178827454,
                "vcoreSeconds":42957,
                "queueUsagePercentage":INF
            }
        ]
    }
}

eg: "queueUsagePercentage":INF

and

class Response(object):
    """
    Basic container for response dictionary

    :param requests.Response response: Response for call via requests lib
    """
    def __init__(self, response):
        #: Dictionary with response data.  Handle cases where content is empty
        # to prevent JSON decode issues
        if response.content:
            self.data = response.json()
        else:
            self.data = {}

response.json() Response is wrong json format resulting in conversion error

116

[dimon222]

This sounds like Hadoop platform bug and library is doing what is intended. Not a valid JSON.

[kevin-bates]

I agree - definitely a Hadoop issue. I suppose we could try to catch and log (or convert) as text, but the calling application can just do the same so I'm inclined to push this to the client (until Hadoop fixes this).

[dimon222]

Fixed in YARN itself. https://issues.apache.org/jira/browse/YARN-7699