Pinpoint: removeAttributes API not deleting the user attributes

[deepakpc]

Confirm by changing [ ] to [x] below to ensure that it's a bug:

• [x] I've gone through Developer Guide and API reference
• [x] I've checked AWS Forums and StackOverflow for answers
• [x] I've searched for previous similar issues and didn't find any solution

Describe the bug I have been trying the removeAttributes API for Pinpoint to remove a specific user attribute from all the endpoints. But, whenever I call this API, I get a response with blacklisted attributes which are all the fields except for what I have given in the request. But, when I get the endpoint using getEndpoint, I can see all the attributes intact. I tried updating to the latest version of the SDK. But nothing changed.

Is the issue in the browser/Node.js? Node.js

If on Node.js, are you running this on AWS Lambda? Yes

Details of the browser/Node.js version Paste output of npx envinfo --browsers or node -v Using the Node.js 10.x Runtime for Lambda

SDK version number v2.643.0

To Reproduce (observed behavior)

const params = {
        ApplicationId: 'Pinpoint-AppId', /* required */
        AttributeType: 'endpoint-user-attributes', /* required */
        UpdateAttributesRequest: { /* required */
            Blacklist: ['uuid1(user-attribute-1)', 'uuid2(user-attribute-2)']
        }
};
const removeAttributesResp = await pinpoint.removeAttributes(params).promise();

Expected behavior It should delete the user attributes given in the 'Blacklist' field from all endpoints.

Screenshots If applicable, add screenshots to help explain your problem.

Additional context Add any other context about the problem here.

[ajredniwja]

Have you followed the steps to use the latest SDK when working with Lambda?

Can you please check the version by printing the output of AWS.VERSION in your code where you have AWS = require("aws-sdk");

It might be the case that you are using an older version of the SDK if you have not bundled the SDK to use in your application.

[deepakpc]

@ajredniwja I'm using a Lambda Layer with the latest version of aws-sdk. The output of AWS.VERSION is 2.646.0. Updated and tried today. Not working still.

[ajredniwja]

@deepakpc In order for removeAttribute to work with blacklisted attribute you have to specify exact name of each attribute to remove or the Blacklist array can specify a glob pattern that an attribute name must match in order for the attribute to be removed. Can you make sure that you are sending the attribute in correct pattern?

https://docs.aws.amazon.com/pinpoint/latest/apireference/apps-application-id-attributes-attribute-type.html#apps-application-id-attributes-attribute-typeput

[deepakpc]

@ajredniwja Yes. I am sending the exact attribute name in the Blacklist array. In my case, the attribute name that I gave is a UUID. For example: "081fb620-7244-11ea-a350-b7f82e08a699".

[deepakpc]

@ajredniwja Any new updates on this issue?

[fotherda71]

I have exactly the same problem.

The docs are wrong - they claim that it should return the removed attributes but in fact it returns the remaining attributes.

I have tried this operation using the:

1. AWS Java SDK
2. AWS CLI
3. AWS REST API

The results are identical in every case.

Please can we get an update on this?

[pnikhil]

@fotherda71 @deepakpc Hey guys, were you able to resolve this issue? I notice the same in Python boto SDK.

[deepakpc]

@fotherda71 @deepakpc Hey guys, were you able to resolve this issue? I notice the same in Python boto SDK.

No. I had contacted AWS developer support and they said it was an issue on their end after contacting the Pinpoint development team. They suggested me a "workaround" which was to delete the endpoints that have the attributes to be removed and re-create them without those attributes.

[pnikhil]

@deepakpc Oh, alright! Thank you for your quick response 👍

[edvanr]

Any update on this one? We are discovering this now and it is becoming a pretty urgent issue for us as we need to remove stale attributes from all our users and it is not feasible to do it on a user by users basis as we have millions of endpoints.

[ajredniwja]

@edvanr reaching out to the pinpoint team to see what the issue was and whats going on.

P49565825

[deepakpc]

@ajredniwja Will this get fixed any time soon or is the Pinpoint team still figuring out what is going on?

[ajredniwja]

Pinpoint team was able to provide a workaround but not an idle solution, please see if this works for you.

You follow the following sequence of commands:

1. Create a test endpoint with "UserAttributes.subscriptions" as a user attribute:

   aws pinpoint update-endpoint --cli-input-json '{
   "ApplicationId": "xyz",
   "EndpointId": "endpoint-test",
   "EndpointRequest": {
   "Address": "rest@west.com",
   "User": {
     "UserAttributes": {
       "subscriptions": ["prime" ],
       "topics": ["science", "politics", "cats", "dogs"]
     }
   },
   "ChannelType": "EMAIL"
   }
   }'

2. Remove all values for the user attribute "subscriptions". Note that excluding "subscriptions" itself without the empty list will NOT remove any values for subscriptions nor will it remove the attribute.

   aws pinpoint update-endpoint --cli-input-json '{
   "ApplicationId": "xyz",
   "EndpointId": "endpoint-test",
   "EndpointRequest": {
   "Address": "xx@dodo.com",
   "User": {
     "UserAttributes": {
       "subscriptions": [],
       "topics": ["science", "politics", "cats", "dogs"]
     }
   },
   "ChannelType": "EMAIL"
   }
   }'

3. If you want to remove the "subscriptions" attribute entirely, first delete the endpoint:

   aws pinpoint delete-endpoint --application-id xyz --endpoint-id endpoint-test

4. Then recreate it, without the "subscriptions" attribute:

   aws pinpoint update-endpoint --cli-input-json '{
   "ApplicationId": "xyz",
   "EndpointId": "endpoint-test",
   "EndpointRequest": {
   "Address": "22@zz.com",
   "User": {
     "UserAttributes": {
       "topics": ["science", "politics", "cats", "dogs"]
     }
   },
   "ChannelType": "EMAIL"
   }
   }'

[github-actions[bot]]

This issue has not received a response in 1 week. If you still think there is a problem, please leave a comment to avoid the issue from automatically closing.

[deepakpc]

@ajredniwja I understand what the Pinpoint team is trying to provide as a workaround. When I try to update an endpoint with an empty array, it is working fine for an endpoint.

But, the real issue is not that. The "removeAttributes" API is for removing that attribute from all the endpoints within that Pinpoint Application. With this workaround, we need to get the endpoints and update them one by one. But, the AWS-SDK for Javascript only has "getEndpoint" API, which is only to get a single endpoint. There is no way to get a list of the endpoints other than doing a "createExportJob". This makes the suggested workaround very hard to achieve in a large project in real-time.

Is there any suggestion on the exact issue?

[vrieni]

Is there an update on this? I'm having the exact same problem with custom endpoint attributes using boto3.

[adrian5991]

Any updates? Still experiencing the same issue that other users have pointed out above.

[mint673]

Any updates? Still experiencing the same issue that other users have pointed out above.

[longility]

This is still an issue as we ran into this.

[stwonary]

We are facing the limit of attributes in production and getting this error:

ERROR   BadRequestException: Too many total additional attributes

The workaround mentioned before of deleting all users and reinsert is not doable for us. Please fix this issue asap.

In the mean time we are requesting to increase the soft limit from 250 to 500. But seems like this change is not instantly that require code push from the AWS team and require a few days to apply it.

[AleksandarGT]

Still having this issue in 2023. I fail to understand what does removeAttributes function even do at this point.

[RanVaknin]

Hi all,

This is not an issue with the SDK. This is an issue with this operation being incorrectly documented. I have reached out to the Pinpoint service team and asked for clarifications and this is what I heard back:

The main usage of this API is when a application reaches the limit of attributes per application, the customer can call this API to remove the attribute key and ingests a new one. (The old one will still be in the endpoint, but update/create new value for the removed attribute key will be rejected if it breaches the limit.)

I have submitted a request to update the documentation and clarify this matter. Since this is not an issue with the SDK and is not actionable by the SDK team, I'll be closing this.

Thanks again for your continuous engagement on the matter.

All the best, Ran~

[deepakpc]

Thanks for the reply. Too bad it took 4 years for the team to find this out.