search

awslabs / aws-sdk-rust

AWS SDK for the Rust Programming Language
https://awslabs.github.io/aws-sdk-rust/
Apache License 2.0
2.96k stars 242 forks source link

Failure to parse json from response for aws_sdk_quicksight::Client::list_data_sources() #1095

Closed mjc3bb closed 4 months ago

mjc3bb commented 6 months ago

Describe the bug

In aws-models/quicksight.json DataSourceParameters has the union type, but the api appears to return a structure type with nulls for the other variants. Due to this, when calling aws_sdk_quicksight::Client::list_data_sources() it errors due to AmazonElasticsearchParameters being returned as null.

Expected Behavior

Calling aws_sdk_quicksight::Client::list_data_sources() should correctly parse the returned api response and provide a single enum variant of DataSourceParameters.

Current Behavior

The api appears to return a structure type instead of a union type, so it has a null value for all possible variants, except the variant it actually is. The generated code bounces off of the first variant check which is AmazonElasticsearchParameters.

https://github.com/awslabs/aws-sdk-rust/blob/main/sdk/quicksight/src/protocol_serde/shape_data_source_parameters.rs#L195

body of datasourceparameters from the trace logs:

{
  "AmazonElasticsearchParameters": null,
  "AmazonOpenSearchParameters": null,
  "AppFlowParameters": null,
  "AthenaParameters": null,
  "AuroraParameters": null,
  "AuroraPostgreSqlParameters": null,
  "AwsIotAnalyticsParameters": null,
  "BigQueryParameters": null,
  "DatabricksParameters": null,
  "Db2Parameters": null,
  "DenodoParameters": null,
  "DocumentDBParameters": null,
  "DremioParameters": null,
  "ExasolParameters": null,
  "GoogleAnalyticsParameters": null,
  "JiraParameters": null,
  "MariaDbParameters": null,
  "MongoAtlasParameters": null,
  "MongoDBParameters": null,
  "MySqlParameters": null,
  "OracleParameters": null,
  "PostgreSqlParameters": null,
  "PrestoParameters": null,
  "RdsParameters": null,
  "RedshiftParameters": null,
  "S3Parameters": {
    "IsUploaded": false,
    "ManifestFileLocation": {
      "Bucket": "deided-bucket.prod.us-east-1",
      "Key": "sales/manifest.json"
    },
    "RoleArn": null
  },
  "SalesforceParameters": null,
  "SapHanaParameters": null,
  "ServiceNowParameters": null,
  "SnowflakeParameters": null,
  "SparkParameters": null,
  "SqlServerParameters": null,
  "StarburstParameters": null,
  "TeradataParameters": null,
  "TrinoParameters": null,
  "TwitterParameters": null
}

stdout from test to reproduce issue

---- backup::reproduce_error stdout ----
thread 'backup::reproduce_error' panicked at src/backup.rs:175:47:
called `Result::unwrap()` on an `Err` value: service error

Caused by:
    0: unhandled error
    1: failed to parse JSON: value for 'AmazonElasticsearchParameters' cannot be null

Reproduction Steps

let config = aws_config::defaults(aws_config::BehaviorVersion::latest()).load.await;
let client = aws_sdk_quicksight::Client::new(&config);
let resp = client.list_data_sources().aws_account_id("<id here>").send.await;

resp will be a service error about failing to parse json.

Possible Solution

A quick fix that I validated works is to use a structure type instead of a union type in the aws-model, like boto3 does. This severely limits the usability of the type however since you can no longer match against it to determine which DataSourceParameters variant it used. It's possible this is more of a QuickSight api problem, as it returns surprising/unexpected data.

Additional Information/Context

No response

Version

├── aws-config v1.1.7
│   ├── aws-credential-types v1.1.7
│   │   ├── aws-smithy-async v1.1.7
│   │   ├── aws-smithy-runtime-api v1.1.7
│   │   │   ├── aws-smithy-async v1.1.7 (*)
│   │   │   ├── aws-smithy-types v1.1.7
│   │   ├── aws-smithy-types v1.1.7 (*)
│   ├── aws-runtime v1.1.7
│   │   ├── aws-credential-types v1.1.7 (*)
│   │   ├── aws-sigv4 v1.1.7
│   │   │   ├── aws-credential-types v1.1.7 (*)
│   │   │   ├── aws-smithy-http v0.60.6
│   │   │   │   ├── aws-smithy-runtime-api v1.1.7 (*)
│   │   │   │   ├── aws-smithy-types v1.1.7 (*)
│   │   │   ├── aws-smithy-runtime-api v1.1.7 (*)
│   │   │   ├── aws-smithy-types v1.1.7 (*)
│   │   ├── aws-smithy-async v1.1.7 (*)
│   │   ├── aws-smithy-http v0.60.6 (*)
│   │   ├── aws-smithy-runtime-api v1.1.7 (*)
│   │   ├── aws-smithy-types v1.1.7 (*)
│   │   ├── aws-types v1.1.7
│   │   │   ├── aws-credential-types v1.1.7 (*)
│   │   │   ├── aws-smithy-async v1.1.7 (*)
│   │   │   ├── aws-smithy-runtime-api v1.1.7 (*)
│   │   │   ├── aws-smithy-types v1.1.7 (*)
│   ├── aws-sdk-sso v1.15.0
│   │   ├── aws-credential-types v1.1.7 (*)
│   │   ├── aws-runtime v1.1.7 (*)
│   │   ├── aws-smithy-async v1.1.7 (*)
│   │   ├── aws-smithy-http v0.60.6 (*)
│   │   ├── aws-smithy-json v0.60.6
│   │   │   └── aws-smithy-types v1.1.7 (*)
│   │   ├── aws-smithy-runtime v1.1.7
│   │   │   ├── aws-smithy-async v1.1.7 (*)
│   │   │   ├── aws-smithy-http v0.60.6 (*)
│   │   │   ├── aws-smithy-runtime-api v1.1.7 (*)
│   │   │   ├── aws-smithy-types v1.1.7 (*)
│   │   ├── aws-smithy-runtime-api v1.1.7 (*)
│   │   ├── aws-smithy-types v1.1.7 (*)
│   │   ├── aws-types v1.1.7 (*)
│   ├── aws-sdk-ssooidc v1.15.0
│   │   ├── aws-credential-types v1.1.7 (*)
│   │   ├── aws-runtime v1.1.7 (*)
│   │   ├── aws-smithy-async v1.1.7 (*)
│   │   ├── aws-smithy-http v0.60.6 (*)
│   │   ├── aws-smithy-json v0.60.6 (*)
│   │   ├── aws-smithy-runtime v1.1.7 (*)
│   │   ├── aws-smithy-runtime-api v1.1.7 (*)
│   │   ├── aws-smithy-types v1.1.7 (*)
│   │   ├── aws-types v1.1.7 (*)
│   ├── aws-sdk-sts v1.15.0
│   │   ├── aws-credential-types v1.1.7 (*)
│   │   ├── aws-runtime v1.1.7 (*)
│   │   ├── aws-smithy-async v1.1.7 (*)
│   │   ├── aws-smithy-http v0.60.6 (*)
│   │   ├── aws-smithy-json v0.60.6 (*)
│   │   ├── aws-smithy-query v0.60.6
│   │   │   ├── aws-smithy-types v1.1.7 (*)
│   │   ├── aws-smithy-runtime v1.1.7 (*)
│   │   ├── aws-smithy-runtime-api v1.1.7 (*)
│   │   ├── aws-smithy-types v1.1.7 (*)
│   │   ├── aws-smithy-xml v0.60.6
│   │   ├── aws-types v1.1.7 (*)
│   ├── aws-smithy-async v1.1.7 (*)
│   ├── aws-smithy-http v0.60.6 (*)
│   ├── aws-smithy-json v0.60.6 (*)
│   ├── aws-smithy-runtime v1.1.7 (*)
│   ├── aws-smithy-runtime-api v1.1.7 (*)
│   ├── aws-smithy-types v1.1.7 (*)
│   ├── aws-types v1.1.7 (*)
├── aws-sdk-quicksight v1.17.0
│   ├── aws-credential-types v1.1.7 (*)
│   ├── aws-runtime v1.1.7 (*)
│   ├── aws-smithy-async v1.1.7 (*)
│   ├── aws-smithy-http v0.60.6 (*)
│   ├── aws-smithy-json v0.60.6 (*)
│   ├── aws-smithy-runtime v1.1.7 (*)
│   ├── aws-smithy-runtime-api v1.1.7 (*)
│   ├── aws-smithy-types v1.1.7 (*)
│   ├── aws-types v1.1.7 (*)


### Environment details (OS name and version, etc.)

macOS Ventura 13.6 and Pop!_OS 22.04 LTS

### Logs

_No response_
rcoh commented 5 months ago

seems like either a parsing bug or a service side bug. We're investigating and will get this fixed

rcoh commented 5 months ago

Bug on our side. Fixed here: https://github.com/smithy-lang/smithy-rs/pull/3481 It just missed the train for the new release of the code generator but it will go out in the next one or a patch.

mjc3bb commented 5 months ago

Thanks so much for the quick response and fix on this!

mjc3bb commented 5 months ago

Hi, just wanted to follow up on this issue. It wasn't high priority at the time I submitted this issue but it is starting to become a blocker for a project at work. I don't see a listed cadence, is there any expected timeline for a new release to be cut for smithy-rs so that this change can make it into aws-sdk-rust?

Edit: I see now the changes actually are in next, so hopefully that means we can expect a release with those changes relatively soon?

jdisanti commented 5 months ago

The fix was merged into smithy-rs, so it'll go out the next time we do a smithy-rs release, which should be pretty soon.

ysaito1001 commented 4 months ago

Confirmed the reproduction steps were executed successfully with aws-sdk-quicksight of version 1.26.0 (must've been fixed in an earlier version).

github-actions[bot] commented 4 months ago

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.