search

Azure / azure-sdk-for-python

This repository is for active development of the Azure SDK for Python. For consumers of the SDK we recommend visiting our public developer docs at https://learn.microsoft.com/python/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-python.
MIT License
4.59k stars 2.8k forks source link

Sample Update TTL doesnt work (comes back with cna't change partition key) #15683

Closed bekraupov closed 3 years ago

bekraupov commented 3 years ago

Describe the bug I am trying to run sample command described in pypi to update TTL of the container

To Reproduce Steps to reproduce the behavior: database.replace_container( container, partition_key=PartitionKey(path="/productName"), default_ttl=10, )

Expected behavior It should udpate the TTL but in my case it is giving an error as per below: azure.cosmos.exceptions.CosmosHttpResponseError: (BadRequest) Message: {"serializedCollection":"{\"Errors\":[\"Document collection partition key cannot be changed.\"]}","serializedSourceCollectionForMaterializedView":"","serializedOffer":"","serializedPartitionKeyRanges":[],"serializedPartitions":[],"collectionRemoteStorageSecurityIdentifier":"00000000-0000-0000-c014-e146cc010000","collectionChildResourceNameLimitInBytes":-1,"collectionChildResourceContentLengthLimitInKB":-1,"uniqueIndexNameEncodingMode":0,"uniqueIndexReIndexingState":0} ActivityId: 0fb27df8-444a-4892-9655-1cab99e03c3b, Request URI: /apps/29ee1020-5171-4c4e-8bb0-26e77ead2474/services/8bc5e8f8-9062-49f8-808a-e8093b414a15/partitions/dc35cf73-721c-483b-9246-2cb73cddd2f6/replicas/132500272630426623p, RequestStats: RequestStartTime: 2020-12-07T15:19:50.3866416Z, RequestEndTime: 2020-12-07T15:19:50.3866416Z, Number of regions attempted:1 ResponseTime: 2020-12-07T15:19:50.3866416Z, StoreResult: StorePhysicalAddress: rntbd://10.0.0.31:11300/apps/29ee1020-5171-4c4e-8bb0-26e77ead2474/services/8bc5e8f8-9062-49f8-808a-e8093b414a15/partitions/dc35cf73-721c-483b-9246-2cb73cddd2f6/replicas/132500272630426623p, LSN: 3121, GlobalCommittedLsn: 3121, PartitionKeyRangeId: , IsValid: True, StatusCode: 400, SubStatusCode: 0, RequestCharge: 1.24, ItemLSN: -1, SessionToken: -1#3121, UsingLocalLSN: False, TransportException: null, ResourceType: Collection, OperationType: Replace , SDK: Microsoft.Azure.Documents.Common/2.11.0

Tried on my collegue's box as well so he is having similar issue. We tried to retrieve Parition Key from container and use the same (not hardcode), same issue

Screenshots If applicable, add screenshots to help explain your problem.

Additional context Add any other context about the problem here.

ghost commented 3 years ago

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @southpolesteve, @zfoster

zfoster commented 3 years ago

@bekraupov I'll take a look

zfoster commented 3 years ago

@bekraupov is this the sample you're trying to use? https://github.com/Azure/azure-sdk-for-python/blob/fc38db1cea8e6b7f22b29f3218c7bcf0f24668ca/sdk/cosmos/azure-cosmos/samples/examples.py

I was able to run that multiple times with multiple TTLs and I'm not seeing errors.

Is the partition key you're using "/productName"? - I see the code you shared is using that

bekraupov commented 3 years ago

yes I am, canyou confirm yo uhave run for exsting container which wa there already (and had TTL turned off)

bekraupov commented 3 years ago

see attached upload sampel code where you can see i am reading container and getting its existing PartitionKey

image

bekraupov commented 3 years ago

Full exception stack:

Error Traceback (most recent call last): File "C:\UBS\Dev\Bek\Conda\wcat-build\lib\unittest\case.py", line 59, in testPartExecutor yield File "C:\UBS\Dev\Bek\Conda\wcat-build\lib\unittest\case.py", line 605, in run testMethod() File "C:\UBS\Dev\Bek\GitWorkspace\wcat-build\python_scripts\test_cosmos_common.py", line 21, in test_SetTTL response = cosmosService.SetTTL("WCATMultiRole", "dataset", 500) File "C:\UBS\Dev\Bek\GitWorkspace\wcat-build\python_scripts\cosmos_common.py", line 46, in SetTTL dbClient.replace_container(containerClient, partition_key=PartitionKey(path= partitionKey), default_ttl=ttlInSeconds) File "C:\UBS\Dev\Bek\Conda\wcat-build\lib\site-packages\azure\core\tracing\decorator.py", line 83, in wrapper_use_tracer return func(args, kwargs) File "C:\UBS\Dev\Bek\Conda\wcat-build\lib\site-packages\azure\cosmos\database.py", line 508, in replace_container container_link, collection=parameters, options=request_options, kwargs File "C:\UBS\Dev\Bek\Conda\wcat-build\lib\site-packages\azure\cosmos_cosmos_client_connection.py", line 448, in ReplaceContainer return self.Replace(collection, path, "colls", collection_id, None, options, kwargs) File "C:\UBS\Dev\Bek\Conda\wcat-build\lib\site-packages\azure\cosmos_cosmos_client_connection.py", line 2133, in Replace result, self.last_response_headers = self.__Put(path, request_params, resource, headers, kwargs) File "C:\UBS\Dev\Bek\Conda\wcat-build\lib\site-packages\azure\cosmos_cosmos_client_connection.py", line 2269, in __Put kwargs File "C:\UBS\Dev\Bek\Conda\wcat-build\lib\site-packages\azure\cosmos_synchronized_request.py", line 218, in SynchronizedRequest kwargs File "C:\UBS\Dev\Bek\Conda\wcat-build\lib\site-packages\azure\cosmos_retry_utility.py", line 73, in Execute result = ExecuteFunction(function, global_endpoint_manager, args, *kwargs) File "C:\UBS\Dev\Bek\Conda\wcat-build\lib\site-packages\azure\cosmos_retry_utility.py", line 130, in ExecuteFunction return function(args, **kwargs) File "C:\UBS\Dev\Bek\Conda\wcat-build\lib\site-packages\azure\cosmos_synchronized_request.py", line 158, in _Request raise exceptions.CosmosHttpResponseError(message=data, response=response) azure.cosmos.exceptions.CosmosHttpResponseError: (BadRequest) Message: {"serializedCollection":"{\"Errors\":[\"Document collection partition key cannot be changed.\"]}","serializedSourceCollectionForMaterializedView":"","serializedOffer":"","serializedPartitionKeyRanges":[],"serializedPartitions":[],"collectionRemoteStorageSecurityIdentifier":"00000000-0000-0000-40dd-625f3b010000","collectionChildResourceNameLimitInBytes":-1,"collectionChildResourceContentLengthLimitInKB":-1,"uniqueIndexNameEncodingMode":0,"uniqueIndexReIndexingState":0} ActivityId: 2943b563-ec1e-44ed-856c-193054a1837c, Request URI: /apps/29ee1020-5171-4c4e-8bb0-26e77ead2474/services/8bc5e8f8-9062-49f8-808a-e8093b414a15/partitions/dc35cf73-721c-483b-9246-2cb73cddd2f6/replicas/132499165541179298p, RequestStats: RequestStartTime: 2020-12-12T11:19:47.6890629Z, RequestEndTime: 2020-12-12T11:19:47.6890629Z, Number of regions attempted:1 ResponseTime: 2020-12-12T11:19:47.6890629Z, StoreResult: StorePhysicalAddress: rntbd://10.0.0.33:11000/apps/29ee1020-5171-4c4e-8bb0-26e77ead2474/services/8bc5e8f8-9062-49f8-808a-e8093b414a15/partitions/dc35cf73-721c-483b-9246-2cb73cddd2f6/replicas/132499165541179298p, LSN: 3126, GlobalCommittedLsn: 3126, PartitionKeyRangeId: , IsValid: True, StatusCode: 400, SubStatusCode: 0, RequestCharge: 1.24, ItemLSN: -1, SessionToken: -1#3126, UsingLocalLSN: False, TransportException: null, ResourceType: Collection, OperationType: Replace , SDK: Microsoft.Azure.Documents.Common/2.11.0

zfoster commented 3 years ago

@bekraupov I still can't reproduce unfortunately - I've tried pretty much every configuration of container and TTL. Can you try just running this code with the partition key hard coded? It seems like the key and the container aren't matching

bekraupov commented 3 years ago

the fix was around passing version=1 into the mthod... looksl ike if partition was created with 100bytes less then it will be version1, otherwise version 2

zfoster commented 3 years ago

I see - version 1 is quite rare and usually old, it makes sense the samples wouldn't reproduce the issue. Glad you solved it!

MohanrajBalakrishnan commented 2 years ago

the fix was around passing version=1 into the mthod... looksl ike if partition was created with 100bytes less then it will be version1, otherwise version 2

@bekraupov I am facing the same issue. Can you explain clearly what you done to fix that? What is version =1 and version =2?

MohanrajBalakrishnan commented 2 years ago

@bekraupov I am facing the same issue. Can you explain clearly what you done to fix that? What is version =1 and version =2?