query format at OPAL_DATA_CONFIG_SOURCES

[HonorBn]

Hello.

I met an error when using postgres fetch provider.

ValueError: OPA Client: unexpected status code: 400, error: {'reason': 'You have malformed a bad request', 'description': 'The content in the request does not match the specifications: entities deserialization error: invalid type: string "{}", expected a map', 'code': 400}

I think the query may be wrong, but I couldn't find right format.

Is there any guide? Or what is the correct sql format including that of attrs and parents if they are empty or filled?

Thank you.

[danyi1212]

Hey @HonorBn, thank you for reporting this issue 💎

The error suggests a serialization problem, potentially due to an empty data update represented by {}. To better assist, could you provide details on your OPAL_DATA_CONFIG_SOURCES configuration and any other relevant configs? Additionally, if you have any specific SQL queries related to this, please share those as well.

Looking forward to your response! Let me know if you’d like further help troubleshooting this.

[maya-barak]

Hey @HonorBn :) Also, This doc might be relevant - https://docs.opal.ac/tutorials/write_your_own_fetch_provider/

[HonorBn]

Hi, @danyi1212, @maya-barak.

I'd like to know more than what is in the docs.

The query format seems to have to be the result of a Select statement with columns named uid, attrs, and parents. The uid may be recognized well. However, in the case of attrs and parents, no matter how many times I try, an error occurs such as invalid type: string "{}", expected a map. I tried as the following value; , '{}'::json, json_build_object(), json_build_object('aaa', 'bbb'), and etc.

And here is my ConfigMap. The query as the below. is, SELECT CONCAT('{"type": "User", "id": "', account, '"}') as uid, ''{}'' as attrs, ''[]''::json as parents FROM public.users;

[danyi1212]

It is possible that the issue is the json data type in the query.

Now knowing that you ingest it into Cedar, I've found exactly what raised the deserialization error. Here are the available entities with types that are available in Cedar. https://docs.cedarpolicy.com/auth/entities-syntax.html

In Cedar Agent (Stateful Web Server for Cedar) we directly use the Entities.from_json_value() to serialize the PUT /v1/data HTTP Request JSON body. https://docs.rs/cedar-policy/latest/cedar_policy/struct.Entities.html#method.from_json_value

@omer9564 might have some further insights about how Cedar Agent ingests data.

The serialization error is raised from this method due to the invalid values in the Request's JSON body, which was generated from the SQL Query result.

Hope that will give you a lead in resolving that issue. Let me know if you’d like further help troubleshooting this.

[HonorBn]

@danyi1212 I know that Cedar Agent collects data using JSON format. But I'd like to know why opal_client.policy.fetcher or opal_client.data.updator recognizes JSON as String even though I have offered json_build_object().

[danyi1212]

Maybe the Postgres provider is loading the SQL Query result as String and not as JSON, so when it provides it to Cedar Agent it gets the error.

I would recommend running the OPAL locally and debugging it, checking what exactly is returned from the Postgres query and what is sent in the data update.

Let me know if you’d like code references or any further help troubleshooting this.