cloudquery / cloudquery

The open source high performance ELT framework powered by Apache Arrow
https://cloudquery.io
Mozilla Public License 2.0
5.72k stars 503 forks source link

bug: Missing `properties.keyVaultKeyUri` in `azure_cosmos_database_accounts` #11622

Closed code-vise closed 1 year ago

code-vise commented 1 year ago

Describe the resource

properties.keyVaultKeyUri in https://www.cloudquery.io/docs/plugins/sources/azure/tables/azure_cosmos_database_accounts this table

Use Case

want to check if cosmosdb account is using encryption_at_rest_using_cmk

Link to API documentation

https://learn.microsoft.com/en-us/rest/api/cosmos-db-resource-provider/2022-11-15-preview/database-accounts/list?tabs=HTTP&tryIt=true#code-try-0

Additional Context

No response

Pull request (optional)

candiduslynx commented 1 year ago

@code-vise do you miss this info from the properties? Or you want this to be a top-level field in the azure_cosmos_database_accounts resource?

code-vise commented 1 year ago

yes we are missing this info from the properties. and no we dont want it to be a top-level field, Its just missing in the properties column of the table

candiduslynx commented 1 year ago

yes we are missing this info from the properties. and no we dont want it to be a top-level field, Its just missing in the properties column of the table

@code-vise could you verify that you can get the info you seek via az cli?

code-vise commented 1 year ago

Yes sure, using this az command, we get this WhatsApp Image 2023-06-16 at 1 58 24 PM

Screenshot 2023-06-16 at 1 58 26 PM
candiduslynx commented 1 year ago

@code-vise We use this object from Azure SDK and the spec for the keyVaultKeyUri is

// The URI of the key vault
KeyVaultKeyURI *string `json:"keyVaultKeyUri,omitempty"`

so the null value from Azure API will be just omitted & won't be present in the output.

Kindly verify that you can see the value if you assign some Key Vault Key URI, and, if so, this just works as designed and you'll need to account for possible missing values in your queries. Missing value == null in this case.

Consider this example for PostgreSQL:

postgres=# create table gh_11622_test(a jsonb);
CREATE TABLE

postgres=# insert into gh_11622_test values('{"a":123}');                                                                                                                 INSERT 0 1

postgres=# select * from gh_11622_test;                                                                                                                                        a      
------------
 {"a": 123}
(1 row)

postgres=# select a->>'a' from gh_11622_test;
 ?column? 
----------
 123
(1 row)

postgres=# select a->>'b' from gh_11622_test;
 ?column? 
----------

(1 row)

postgres=# select (a->>'b' IS NULL) as b_is_null from gh_11622_test;
 b_is_null 
-----------
 t
(1 row)

postgres=# select (a->>'a' IS NULL) as a_is_null, (a->>'b' IS NULL) as b_is_null from gh_11622_test;
 a_is_null | b_is_null 
-----------+-----------
 f         | t
(1 row)