databricks / databricks-sdk-py

Databricks SDK for Python (Beta)
https://databricks-sdk-py.readthedocs.io/
Apache License 2.0
369 stars 124 forks source link

AccountClient.groups.patch operation not working #187

Open althrussell opened 1 year ago

althrussell commented 1 year ago

Not 100% sure the issue here. I cannot manage to get groups.patch operation to work. Error is "'str' object has no attribute 'value'"

My Code>>

patches = [] patch = Patch() patch.op = "add" patch.path = "members" #Tried this>patch.value = [ComplexValue(value=user.id)] #and Tried this> patch.value = user.id patch.value = [{"value": user.id}] patches.append(patch) account_client.groups.patch(id=group.id,operations=patches)

Looking at the API payload it's [ { "op": "add", "path": "members", "value": [ { "value": "3780913032932404" } ] } ] Is it my implementation or is value not getting encoded correctly?

nfx commented 1 year ago

@althrussell please share a full stack trace. I suspect it's related to using an incorrect type.

althrussell commented 1 year ago

File "databricksonaws/env/lib/python3.11/site-packages/databricks/sdk/service/iam.py", line 1029, in patch body = request.as_dict() ^^^^^^^^^^^^^^^^^ File "databricksonaws/env/lib/python3.11/site-packages/databricks/sdk/service/iam.py", line 497, in as_dict if self.operations: body['operations'] = [v.as_dict() for v in self.operations] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "databricksonaws/env/lib/python3.11/site-packages/databricks/sdk/service/iam.py", line 497, in if self.operations: body['operations'] = [v.as_dict() for v in self.operations] ^^^^^^^^^^^ File "databricksonaws/env/lib/python3.11/site-packages/databricks/sdk/service/iam.py", line 513, in as_dict if self.op is not None: body['op'] = self.op.value ^^^^^^^^^^^^^ AttributeError: 'str' object has no attribute 'value'

starlancer999 commented 1 year ago

Using patch.op = PatchOp.add solved this for me (at least for the WorkspaceClient which I'm using).

With this I'm able to make the API call but it's not working. I've tried the following 2 approaches:

  1. Passing the request directly operations = [Patch(op=PatchOp.add, path="members", value=el) for el in user_ids] w.groups.patch(id=dbx_group_id, request=PartialUpdate(operations=operations))
  2. Passing operations operations = [Patch(op=PatchOp.add, path="members", value=el) for el in user_ids] w.groups.patch(id=dbx_group_id, operations=operations)

For both approaches I get the following error message: Traceback (most recent call last): File "...\user_management.py", line 74, in add_missing_user_to_dbx w.groups.patch(id=dbx_group_id, operations=operations) File "...\venv\Lib\site-packages\databricks\sdk\service\iam.py", line 1854, in patch self._api.do('PATCH', f'/api/2.0/preview/scim/v2/Groups/{request.id}', body=body) File "...\venv\Lib\site-packages\databricks\sdk\core.py", line 932, in do raise self._make_nicer_error(status_code=response.status_code, **payload) from None databricks.sdk.core.DatabricksError: None Error in performing the patch operation on group resource.

(Hope it's ok that I'm adding my error message to this thread. I believe it's the same issue.)

jdreyesp commented 1 year ago

As a workaround, I overcame this issue by:

from enum import Enum
import requests

class DBAPI_Scope(Enum):
    GROUP = "Groups"
    USER = "Users"
    SP = "ServicePrincipals"

def patch_op(scope: DBAPI_Scope, object_id: str, operations: List[Patch]) -> None:

    def extract_ops(operations: list[Patch]) -> dict:
        for operation in operations:
            if operation.path in ["members", "groups", "roles", "entitlements"]:
                op_value = [{"value": operation.value}]
            else:
                op_value = operation.value
        return [{"op": operation.op.value, "path": operation.path, "value": op_value} for operation in operations]

    resp = requests.patch(
     url=f"https://accounts.azuredatabricks.net/api/2.0/accounts/<<REPLACE_YOUR_ACCOUNT_HERE>>/scim/v2/{scope.value}/{object_id}",
        headers={"Content-Type": "application/json", "Authorization": f"Bearer {ACCESS_TOKEN}"},
        json={
            "schemas":
                ["urn:ietf:params:scim:api:messages:2.0:PatchOp"],
            "Operations": extract_ops(operations)
        })

    if resp.status_code != 200:
        raise Exception(f"Error patching: {resp.json()}")
jaina15 commented 1 year ago

I was trying to use the patch method to patch the Databricks group from the Active directory, but it is removing all the members from that group instead of removing the selected members.

To Replicate:

I tried using @jdreyesp your code but that's also doing the same thing i.e., removing all members from the group instead of selected members.

Also, I found one documentation from databricks that states: Group membership cannot be updated through PATCH/PUT requests.

jdreyesp commented 1 year ago

I use the code I posted with op=add and path=members and it adds those members while keeping the others

jaina15 commented 1 year ago

I use the code I posted with op=add and path=members and it adds those members while keeping the others

Members addition is working fine with patch, the problem occurs when we try to remove some members from the group.