Yelp / bravado

Bravado is a python client library for Swagger 2.0 services
Other
605 stars 117 forks source link

I have a new problem, actually I'm the return value is a standard dictionary, in dealing with the dictionary and bravado, tip me en error #490

Closed luoyq-Rockey closed 11 months ago

luoyq-Rockey commented 1 year ago

hey,guys,I have a new problem, actually I'm the return value is a standard dictionary, in dealing with the dictionary and bravado, tip me bravado_core. Exception. SwaggerMappingError: Expected type to be dict for value 2 to unmarshal to a <class 'dict'>.Was <class 'str'> instead.

luoyq-Rockey commented 1 year ago

return value is : { "id": 1891138, "retried": 0, "retry_at": "0001-01-01T00:00:00Z", "name": "9290-signature_test-1", "workflow_name": "", "namespace": "pipeline-public-270", "cluster": "cluster1", "uuid": "e861c280-15ed-4ccf-8b18-ff28b7dab44e", "status": "Succeeded", "synced": 0, "runId": "5e9317ce-5706-4afe-bf7b-6beba52d4c9b", "description": "", "params": { "a": "1", "b": "2", "c": "3" }, "projectId": 270, "serviceId": 9290, "serviceName": "signature_test", "versionId": 0, "versionName": "", "initedAt": "2023-07-10T19:56:36+08:00", "createdAt": "2023-07-10T19:56:36+08:00", "instanceCreatedAt": "2023-07-10T19:56:36+08:00", "readyAt": "2023-07-10T19:56:37+08:00", "endAt": "2023-07-10T19:56:49+08:00", "updatedAt": "2023-07-10T19:56:49+08:00", "ownerId": "01393547", "ownerName": "01393547", "label": "Develop", "bdpJobId": "", "experiment_id": "39b0ab26-c5ac-4785-8e17-266053c138eb", "create_source": 1, "resource_pool_id": 384, "resource_pool_name": "公共资源池", "new_version": true, "dag": { "name": "my-pipeline", "description": "", "nodes": [ { "name": "run", "container": { "name": "", "image": "artifactory-yun.sit.com/testdockersit/015inc-aiplat-core/base-python:1.0.6", "command": [ "sh", "-ec", "program_path=$(mktemp)\nprintf \"%s\" \"$0\" > \"$program_path\"\npython3 -u \"$program_path\" \"$@\"\n", "def run(a):\n print(a)\n\nimport argparse\n_parser = argparse.ArgumentParser(prog='Run', description='')\n_parser.add_argument(\"--a\", dest=\"a\", type=str, required=True, default=argparse.SUPPRESS)\n_parsed_args = vars(_parser.parse_args())\n\n_outputs = run(**_parsed_args)\n" ], "env": [ { "name": "env", "value": "sit" }, { "name": "user", "value": "01393547" } ], "resources": { "limits": { "cpu": "2", "memory": "8Gi" }, "requests": { "cpu": "2", "memory": "8Gi" } } }, "parameters": { "inputs": [ { "name": "a", "type": "", "from": [ { "key": "a", "source": "global" } ], "formalParam": "a", "isArg": true, "format": "{{inputs.parameters.a}}" } ] }, "dependencies": null, "cache": true, "execution": { "stepName": "run", "step_type": "Pod", "podName": "my-pipeline-szgxv-2356017870", "pod_namespace": "", "podPhase": "Succeeded", "LogUrl": "", "initLogUrl": "", "waitLogUrl": "", "started_at": "2023-07-10T19:56:36+08:00", "startTime": "2023-07-10 19:56:36 +0800 CST", "finishedAt": "2023-07-10 19:56:38 +0800 CST", "inputs": [ { "name": "a", "value": "1", "type": "parameter", "artifactKey": "", "artifactUrl": "", "artifactPreview": "", "container": "" } ], "outputs": [], "message": "", "retry_details": null }, "prefix": "--", "type": "pod" }, { "name": "run-2", "container": { "name": "", "image": "artifactory-yun.sit.com/testdockersit/015inc-aiplat-core/base-python:1.0.6", "command": [ "sh", "-ec", "program_path=$(mktemp)\nprintf \"%s\" \"$0\" > \"$program_path\"\npython3 -u \"$program_path\" \"$@\"\n", "def run(a):\n print(a)\n\nimport argparse\n_parser = argparse.ArgumentParser(prog='Run', description='')\n_parser.add_argument(\"--a\", dest=\"a\", type=str, required=True, default=argparse.SUPPRESS)\n_parsed_args = vars(_parser.parse_args())\n\n_outputs = run(**_parsed_args)\n" ], "env": [ { "name": "env", "value": "sit" }, { "name": "user", "value": "01393547" } ], "resources": { "limits": { "cpu": "2", "memory": "8Gi" }, "requests": { "cpu": "2", "memory": "8Gi" } } }, "parameters": { "inputs": [ { "name": "a", "type": "", "from": [ { "key": "b", "source": "global" } ], "formalParam": "a", "isArg": true, "format": "{{inputs.parameters.b}}" } ] }, "dependencies": null, "cache": true, "execution": { "stepName": "run-2", "step_type": "Pod", "podName": "my-pipeline-szgxv-3161600369", "pod_namespace": "", "podPhase": "Succeeded", "LogUrl": "", "initLogUrl": "", "waitLogUrl": "", "started_at": "2023-07-10T19:56:36+08:00", "startTime": "2023-07-10 19:56:36 +0800 CST", "finishedAt": "2023-07-10 19:56:43 +0800 CST", "inputs": [ { "name": "b", "value": "2", "type": "parameter", "artifactKey": "", "artifactUrl": "", "artifactPreview": "", "container": "" } ], "outputs": [], "message": "", "retry_details": null }, "prefix": "--", "type": "pod" }, { "name": "run-3", "container": { "name": "", "image": "artifactory-yun.sit..com/testdockersit/015inc-aiplat-core/base-python:1.0.6", "command": [ "sh", "-ec", "program_path=$(mktemp)\nprintf \"%s\" \"$0\" > \"$program_path\"\npython3 -u \"$program_path\" \"$@\"\n", "def run(a):\n print(a)\n\nimport argparse\n_parser = argparse.ArgumentParser(prog='Run', description='')\n_parser.add_argument(\"--a\", dest=\"a\", type=str, required=True, default=argparse.SUPPRESS)\n_parsed_args = vars(_parser.parse_args())\n\n_outputs = run(**_parsed_args)\n" ], "env": [ { "name": "env", "value": "sit" }, { "name": "user", "value": "01393547" } ], "resources": { "limits": { "cpu": "2", "memory": "8Gi" }, "requests": { "cpu": "2", "memory": "8Gi" } } }, "parameters": { "inputs": [ { "name": "a", "type": "", "from": [ { "key": "c", "source": "global" } ], "formalParam": "a", "isArg": true, "format": "{{inputs.parameters.c}}" } ] }, "dependencies": null, "cache": true, "execution": { "stepName": "run-3", "step_type": "Pod", "podName": "my-pipeline-szgxv-3144822750", "pod_namespace": "", "podPhase": "Succeeded", "LogUrl": "", "initLogUrl": "", "waitLogUrl": "", "started_at": "2023-07-10T19:56:36+08:00", "startTime": "2023-07-10 19:56:36 +0800 CST", "finishedAt": "2023-07-10 19:56:38 +0800 CST", "inputs": [ { "name": "c", "value": "3", "type": "parameter", "artifactKey": "", "artifactUrl": "", "artifactPreview": "", "container": "" } ], "outputs": [], "message": "", "retry_details": null }, "prefix": "--", "type": "pod" } ], "parameters": [ { "name": "a", "type": "String", "optional": true, "default": "1", "config": "" }, { "name": "b", "type": "String", "optional": true, "default": "2", "config": "" }, { "name": "c", "type": "String", "optional": true, "default": "3", "config": "" } ], "resourceLimit": { "CPU": 0, "Memory": 0 } }, "spark_applications": {} }

luoyq-Rockey commented 1 year ago

The unmarshall.py file has a method for checking

    if not is_dict_like(model_value):
        raise SwaggerMappingError(
            "Expected type to be dict for value {0} to unmarshal to a {1}."
            "Was {2} instead.".format(model_value, model_type, type(model_value)),
        )

When I debug this line of code, it says model_value = (str) '2', but actually, I don't have a key of '2' Now I do not know how to deal with this problem, I hope to get the author's help, so as to facilitate my next work, thank you

luoyq-Rockey commented 1 year ago

image

luoyq-Rockey commented 1 year ago

image

macisamuele commented 1 year ago

To me this somehow looks like a legit validation issue that eventually is not surfaced very cleanly to the user.

@luoyq-Rockey any chance that you could provide a minimal reproducible example? What I'm interested is are 1) swagger spec of the object and 2) object content.

With this two components I could try to support you a bit in determine what the underlying issue is.

If the issue does not persist anymore, feel free to close the issue.

luoyq-Rockey commented 1 year ago

已收到您的邮件,谢谢。

macisamuele commented 1 year ago

English please

luoyq-Rockey commented 1 year ago

English please

Sorry, this reply message is an automatic response from Github

macisamuele commented 1 year ago

I did try to cross-check the #489 and I do have only one suspect, that allOf is creating some odd interactions.

I would suggest to remove the allOf as it is a not needed complexity in the specs (according to the snippet shared). As the following

properties:
  property:
    allOf:
      - $ref: #/definitions/model.ProjectResource

is equivalent to

properties:
  property:
    $ref: #/definitions/model.ProjectResource

If still you want to use the allOf then I would recommend to give to the "types" an explicit model name such that bravado-core is able to generate the proper type and avoid the error you are experiencing. You can provide x-model attribute as described in https://bravado-core.readthedocs.io/en/stable/models.html

luoyq-Rockey commented 1 year ago

Thank you for your reply,as far as I know, allOf is a usage supported by Swagger, At this moment, I have addressed the Bravado parsing issue caused by allof using the following method image image

luoyq-Rockey commented 1 year ago

By handling it myself, I can avoid Bravado checking response data

Also, I found that I set 'validate' Responses': False, but it seems to have no effect. If it had any effect, would the question I mentioned above not have occurred because 'validate' When responses': False, the format of the returned data is not checked client = SwaggerClient.from_spec( load_file(swag), http_client = http_client, config = {'use_models': False, 'validate_responses': False, 'validate_requests': False})

macisamuele commented 11 months ago

Skipping validation is always a possibility... But ideally it would be worth being able to validate and with your current setup it seems that you cannot do so. The issue, as stated above, is caused by the usage of the bravado models. Models are handy wrappers to give you attribute access to the json fields, but the can be disabled if causing troubles. Furthermore, in the analysis above I've also mentioned what could be spec-corrective actions to continue using models.

Does it solve your issues? I would be close the issue in about 2 weeks if no further assistance is needed.

luoyq-Rockey commented 11 months ago

Thank you for your reply, i close this issue by myself, thank you