search

localstack / localstack

💻 A fully functional local AWS cloud stack. Develop and test your cloud & Serverless apps offline
https://localstack.cloud
Other
55.87k stars 3.98k forks source link

bug: SNS publish fails if filter policies match a String.Array with true/false/null #8304

Closed tonybajan closed 1 year ago

tonybajan commented 1 year ago

Is there an existing issue for this?

Current Behavior

When we publish some messages, localstack fails while trying to apply filter policies.

Where the message attribute is a String.Array that contains true, false or null (which is valid), localstack doesn't parse the attribute correctly, as it's using ast.literal_eval.

Expected Behavior

The message attribute should be parsed and the filter policy applied, as for string or boolean values.

How are you starting LocalStack?

With a docker-compose file

Steps To Reproduce

Client commands (e.g., AWS SDK code snippet, or sequence of "awslocal" commands)

aws --endpoint-url=http://localhost:4566 sns create-topic --name test-topic
aws --endpoint-url=http://localhost:4566 sqs create-queue --queue-name test-queue

aws --endpoint-url=http://localhost:4566 sns subscribe --topic-arn arn:aws:sns:eu-west-2:000000000000:test-topic --protocol sqs  --notification-endpoint arn:aws:sqs:eu-west-2:000000000000:test-queue

# Taking subscription ARN from results of previous command
aws --endpoint-url=http://localhost:4566 sns set-subscription-attributes --subscription-arn arn:aws:sns:eu-west-2:000000000000:test-topic:726531bf-ef59-405c-823d-b693cf0eaf0a  --attribute-name FilterPolicy --attribute-value '{"HAS_FOO": [true]}'

aws --endpoint-url=http://localhost:4566 sns publish --topic-arn arn:aws:sns:eu-west-2:000000000000:test-topic --message "hello"  --message-attributes '{"HAS_FOO": {"DataType": "String.Array", "StringValue":"[true]"}}'

Stack trace from logs

master-localstack-1  | 2023-05-11T15:49:34.120682588Z 2023-05-11T15:49:34.120  INFO --- [   asgi_gw_0] localstack.request.aws     : AWS sns.Publish => 500 (InternalError)
master-localstack-1  | 2023-05-11T15:49:34.696141630Z 2023-05-11T15:49:34.695 ERROR --- [   asgi_gw_2] l.aws.handlers.logging     : exception during call chain
master-localstack-1  | 2023-05-11T15:49:34.696181755Z Traceback (most recent call last):
master-localstack-1  | 2023-05-11T15:49:34.696185630Z   File "/opt/code/localstack/localstack/aws/chain.py", line 90, in handle
master-localstack-1  | 2023-05-11T15:49:34.696187838Z     handler(self, self.context, response)
master-localstack-1  | 2023-05-11T15:49:34.696189921Z   File "/opt/code/localstack/localstack/aws/handlers/service.py", line 123, in __call__
master-localstack-1  | 2023-05-11T15:49:34.696191921Z     handler(chain, context, response)
master-localstack-1  | 2023-05-11T15:49:34.696193838Z   File "/opt/code/localstack/localstack/aws/handlers/service.py", line 93, in __call__
master-localstack-1  | 2023-05-11T15:49:34.696195838Z     skeleton_response = self.skeleton.invoke(context)
master-localstack-1  | 2023-05-11T15:49:34.696210338Z   File "/opt/code/localstack/localstack/aws/skeleton.py", line 154, in invoke
master-localstack-1  | 2023-05-11T15:49:34.696216713Z     return self.dispatch_request(context, instance)
master-localstack-1  | 2023-05-11T15:49:34.696219088Z   File "/opt/code/localstack/localstack/aws/skeleton.py", line 166, in dispatch_request
master-localstack-1  | 2023-05-11T15:49:34.696221463Z     result = handler(context, instance) or {}
master-localstack-1  | 2023-05-11T15:49:34.696223380Z   File "/opt/code/localstack/localstack/aws/forwarder.py", line 60, in _call
master-localstack-1  | 2023-05-11T15:49:34.696225505Z     return handler(context, req)
master-localstack-1  | 2023-05-11T15:49:34.696227380Z   File "/opt/code/localstack/localstack/aws/skeleton.py", line 118, in __call__
master-localstack-1  | 2023-05-11T15:49:34.696229838Z     return self.fn(*args, **kwargs)
master-localstack-1  | 2023-05-11T15:49:34.696231921Z   File "/opt/code/localstack/localstack/services/sns/provider.py", line 655, in publish
master-localstack-1  | 2023-05-11T15:49:34.696234005Z     self._publisher.publish_to_topic(publish_ctx, topic_or_target_arn)
master-localstack-1  | 2023-05-11T15:49:34.696235755Z   File "/opt/code/localstack/localstack/services/sns/publisher.py", line 1141, in publish_to_topic
master-localstack-1  | 2023-05-11T15:49:34.696237921Z     if self._should_publish(ctx.store.subscription_filter_policy, ctx.message, subscriber):
master-localstack-1  | 2023-05-11T15:49:34.696240005Z   File "/opt/code/localstack/localstack/services/sns/publisher.py", line 1129, in _should_publish
master-localstack-1  | 2023-05-11T15:49:34.696242088Z     return self.subscription_filter.check_filter_policy_on_message_attributes(
master-localstack-1  | 2023-05-11T15:49:34.696244005Z   File "/opt/code/localstack/localstack/services/sns/publisher.py", line 922, in check_filter_policy_on_message_attributes
master-localstack-1  | 2023-05-11T15:49:34.696246046Z     if not self._evaluate_filter_policy_conditions_on_attribute(
master-localstack-1  | 2023-05-11T15:49:34.696247796Z   File "/opt/code/localstack/localstack/services/sns/publisher.py", line 1005, in _evaluate_filter_policy_conditions_on_attribute
master-localstack-1  | 2023-05-11T15:49:34.696249838Z     values = ast.literal_eval(val)
master-localstack-1  | 2023-05-11T15:49:34.696251671Z   File "/usr/local/lib/python3.10/ast.py", line 110, in literal_eval
master-localstack-1  | 2023-05-11T15:49:34.696253630Z     return _convert(node_or_string)
master-localstack-1  | 2023-05-11T15:49:34.696256421Z   File "/usr/local/lib/python3.10/ast.py", line 90, in _convert
master-localstack-1  | 2023-05-11T15:49:34.696258671Z     return list(map(_convert, node.elts))
master-localstack-1  | 2023-05-11T15:49:34.696277505Z   File "/usr/local/lib/python3.10/ast.py", line 109, in _convert
master-localstack-1  | 2023-05-11T15:49:34.696280255Z     return _convert_signed_num(node)
master-localstack-1  | 2023-05-11T15:49:34.696282130Z   File "/usr/local/lib/python3.10/ast.py", line 83, in _convert_signed_num
master-localstack-1  | 2023-05-11T15:49:34.696284088Z     return _convert_num(node)
master-localstack-1  | 2023-05-11T15:49:34.696285963Z   File "/usr/local/lib/python3.10/ast.py", line 74, in _convert_num
master-localstack-1  | 2023-05-11T15:49:34.696287921Z     _raise_malformed_node(node)
master-localstack-1  | 2023-05-11T15:49:34.696289880Z   File "/usr/local/lib/python3.10/ast.py", line 71, in _raise_malformed_node
master-localstack-1  | 2023-05-11T15:49:34.696292130Z     raise ValueError(msg + f': {node!r}')
master-localstack-1  | 2023-05-11T15:49:34.696310130Z ValueError: malformed node or string on line 1: <ast.Name object at 0xffff48786b90>

Environment

- LocalStack: 2.0.1 from dockerhub

Anything else?

No response

localstack-bot commented 1 year ago

Welcome to LocalStack! Thanks for reporting your first issue and our team will be working towards fixing the issue for you or reach out for more background information. We recommend joining our Slack Community for real-time help and drop a message to LocalStack Pro Support if you are a Pro user! If you are willing to contribute towards fixing this issue, please have a look at our contributing guidelines and our contributing guide.