aws / aws-sdk

Landing page for the AWS SDKs on GitHub
https://aws.amazon.com/tools/
Other
71 stars 14 forks source link

Aws::SQS::Client in GovCloud fails for use of MD5 -> disabled for fips #74

Open Nandez89 opened 3 years ago

Nandez89 commented 3 years ago

When using a client from the aws-sqs-sdk on GovCloud environments there is an error generated when trying to produce outbox messages. When the SDK tries to validate the message body using md5s.rb plugin and that generates the following error

OpenSSL::Digest::DigestError: Digest initialization failed: disabled for fips

Posting it on this repo since the most similar issue I could find for the Ruby SDK repo pointed me here.

This is the stack trace:

2021-05-20 16:40:09/home/user/app/vendor/bundle/ruby/2.5.0/gems/aws-sdk-sqs-1.38.0/lib/aws-sdk-sqs/plugins/md5s.rb:87:in `md5_of_message_body'
2021-05-20 16:40:09/home/user/app/vendor/bundle/ruby/2.5.0/gems/aws-sdk-sqs-1.38.0/lib/aws-sdk-sqs/plugins/md5s.rb:61:in `validate_body'
2021-05-20 16:40:09/home/user/app/vendor/bundle/ruby/2.5.0/gems/aws-sdk-sqs-1.38.0/lib/aws-sdk-sqs/plugins/md5s.rb:54:in `validate_single_message'
2021-05-20 16:40:09/home/user/app/vendor/bundle/ruby/2.5.0/gems/aws-sdk-sqs-1.38.0/lib/aws-sdk-sqs/plugins/md5s.rb:48:in `block in validate_send_message_batch'
2021-05-20 16:40:09/home/user/app/vendor/bundle/ruby/2.5.0/gems/aws-sdk-sqs-1.38.0/lib/aws-sdk-sqs/plugins/md5s.rb:42:in `each'
2021-05-20 16:40:09/home/user/app/vendor/bundle/ruby/2.5.0/gems/aws-sdk-sqs-1.38.0/lib/aws-sdk-sqs/plugins/md5s.rb:42:in `validate_send_message_batch'
2021-05-20 16:40:09/home/user/app/vendor/bundle/ruby/2.5.0/gems/aws-sdk-sqs-1.38.0/lib/aws-sdk-sqs/plugins/md5s.rb:18:in `block in call'
2021-05-20 16:40:09/home/user/app/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.114.0/lib/seahorse/client/response.rb:46:in `block in on'
2021-05-20 16:40:09/home/user/app/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.114.0/lib/seahorse/client/http/response.rb:146:in `block in on_success'
2021-05-20 16:40:09/home/user/app/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.114.0/lib/seahorse/client/http/response.rb:173:in `block in listener'
2021-05-20 16:40:09/home/user/app/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.114.0/lib/seahorse/client/http/response.rb:137:in `on_done'
2021-05-20 16:40:09/home/user/app/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.114.0/lib/seahorse/client/http/response.rb:144:in `on_success'
2021-05-20 16:40:09/home/user/app/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.114.0/lib/seahorse/client/response.rb:45:in `on'
2021-05-20 16:40:09/home/user/app/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.114.0/lib/seahorse/client/response.rb:54:in `on_success'
2021-05-20 16:40:09/home/user/app/vendor/bundle/ruby/2.5.0/gems/aws-sdk-sqs-1.38.0/lib/aws-sdk-sqs/plugins/md5s.rb:13:in `call'
2021-05-20 16:40:09/home/user/app/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.114.0/lib/aws-sdk-core/plugins/param_validator.rb:26:in `call'
2021-05-20 16:40:09/home/user/app/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.114.0/lib/seahorse/client/plugins/raise_response_errors.rb:16:in `call'
2021-05-20 16:40:09/home/user/app/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.114.0/lib/aws-sdk-core/plugins/jsonvalue_converter.rb:22:in `call'
2021-05-20 16:40:09/home/user/app/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.114.0/lib/aws-sdk-core/plugins/idempotency_token.rb:19:in `call'
2021-05-20 16:40:09/home/user/app/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.114.0/lib/aws-sdk-core/plugins/param_converter.rb:26:in `call'
2021-05-20 16:40:09/home/user/app/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.114.0/lib/seahorse/client/plugins/request_callback.rb:71:in `call'
2021-05-20 16:40:09/home/user/app/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.114.0/lib/aws-sdk-core/plugins/response_paging.rb:12:in `call'
2021-05-20 16:40:09/home/user/app/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.114.0/lib/seahorse/client/plugins/response_target.rb:24:in `call'
2021-05-20 16:40:09/home/user/app/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.114.0/lib/seahorse/client/request.rb:72:in `send_request'
2021-05-20 16:40:09/home/user/app/vendor/bundle/ruby/2.5.0/gems/aws-sdk-sqs-1.38.0/lib/aws-sdk-sqs/client.rb:2166:in `send_message_batch'
...{our code call to client.send_message_batch}...
...

Are there any configs specific to the client besides the region and queue URL that tell it not to use md5, or is this something missing in the sdk? I haven't been able to find any other leads here on this so any pointers are greatly appreciated.

kdaily commented 3 years ago

Hi @Nandez89,

Thanks for posting. From my initial research this is something that would have to change on the service side. Similar requests have been made for the Python SDK and the S3 client. For example:

https://github.com/boto/botocore/issues/1700#issuecomment-476412391

I'll look into it some more to see if anything has changed with this scenario.

Nandez89 commented 3 years ago

Hi @kdaily and others, I thought I'd share an update on our workaround.

The issue happens specifically on receiving a response from SQS and when the client is verifying the response contents, so far our workaround was to disable the :verify_checksums option when creating a new Client in GovCloud environments. Since the SDK it's always using the MD5 plugin internally, this doesn't work on GovCloud environments because or FIPS restrictions. It would be great if the verify feature could be used with other algorithms that are FIPS compliant so the verify could be used on GovClous environments too, although I'm not sure if this is supported by SQS responses.

mzia commented 3 years ago

👋🏽 So this is no longer a govcloud issue. We at GSA for login.gov started using Ubuntu Pro for its maintained FIPS OpenSSL validated module to meet FedRAMP requirements and MD5 is not a government approve algorithm. We could def use an assist on this feature update for your future roadmap be it on the service end or not.

ghost commented 2 years ago

Hi Team, We are also facing the same issue. Is there any update on how to resolve the issue?

pkrishjobs commented 10 months ago

Unable to disable MD5 checks from the SDK side (using v3 of AWS SDK for JS). v2 of the AWS SDK for JS works fine to disable MD5 check via computechecksums property. Until v3 has this support working, can v2 SDK support be extended?