Deploying VDS that have Wiki details

[MaWilly]

Hello,

I am using the dremio-cloner script to deploy my dremio environment and I have some virtual datasets that have Wiki details written. I encounter the following errors during the deployment:

DEBUG:2024-02-06 14:13:41,939:_write_wiki: processing wiki: {'entity_id': 'xxx', 'path': ['SELF_SERVICE_PROJECTS', 'DCOG_PROJECT', 'SIMILARWEB', 'SEGMENT_TRAFFIC_AND_ENGAGEMENT'], 'text': ''} DEBUG:2024-02-06 14:13:41,959:https://xxx.com:443 "GET /api/v3/catalog/by-path/SELF_SERVICE_PROJECTS/DCOG_PROJECT/SIMILARWEB/SEGMENT_TRAFFIC_AND_ENGAGEMENT HTTP/1.1" 404 148 INFO:2024-02-06 14:13:41,959:get_catalog_entity_by_path: received HTTP Response Code 404 for : <api/v3/catalog/by-path/SELF_SERVICE_PROJECTS/DCOG_PROJECT/SIMILARWEB/SEGMENT_TRAFFIC_AND_ENGAGEMENT> errorMessage: Could not find entity with path [[SELF_SERVICE_PROJECTS, DCOG_PROJECT, SIMILARWEB, SEGMENT_TRAFFIC_AND_ENGAGEMENT]] moreInfo: ERROR:2024-02-06 14:13:41,959:_write_wiki: Unable to resolve wiki's dataset for {'entity_id': 'xxx', 'path': ['SELF_SERVICE_PROJECTS', 'DCOG_PROJECT', 'SIMILARWEB', 'SEGMENT_TRAFFIC_AND_ENGAGEMENT'], 'text': ''} ERROR:2024-02-06 14:13:41,959:_write_wiki: Unable to resolve wiki's dataset for {'entity_id': 'xxx', 'path': ['SELF_SERVICE_PROJECTS', 'DCOG_PROJECT', 'SIMILARWEB', 'SEGMENT_TRAFFIC_AND_ENGAGEMENT'], 'text': ''}

[mxmarg]

Hi @MaWilly, It seems like the following function call for retrieving the existing wiki fails: self._find_existing_dataset_by_path(self._utils.normalize_path(wiki_path)) (https://github.com/deane-dremio/dremio-cloner/blob/master/src/DremioWriter.py#L1224)

Can you please verify that the specified object "SELF_SERVICE_PROJECTS"."DCOG_PROJECT"."SIMILARWEB"."SEGMENT_TRAFFIC_AND_ENGAGEMENT" exists in your Dremio instance and that the request GET /api/v3/catalog/by-path/SELF_SERVICE_PROJECTS/DCOG_PROJECT/SIMILARWEB/SEGMENT_TRAFFIC_AND_ENGAGEMENT is able to retrieve the Dremio catalog object definition?

[MaWilly]

Hi @mxmarg ,

Yes the specified object does exist in my Dremio instance but for the previous example the request was not able to retrieve the dremio object definition.

I did the test again today;

• create a vds "VW_TEST_WIKI_DEPLOYEMENT" in my first dremio instance
• add some wiki to it
• deploy to another dremio instance

My vds with the wiki was correctly deployed to my new dremio instance, however the Code 404 is still there. Here is the log result regarding the request :

` 2024-02-20T06:23:48.9326718Z DEBUG:2024-02-20 06:23:48,932:https://xxx.com:443 "GET /api/v3/catalog/by-path/IT_RAW_DATA/VW_TEST_WIKI_DEPLOYEMENT HTTP/1.1" 404 106

INFO:2024-02-20 06:23:48,932:get_catalog_entity_by_path: received HTTP Response Code 404 for : <api/v3/catalog/by-path/IT_RAW_DATA/VW_TEST_WIKI_DEPLOYEMENT> errorMessage: Could not find entity with path [[IT_RAW_DATA, VW_TEST_WIKI_DEPLOYEMENT]] moreInfo: `

[mxmarg]

You say that "My vds with the wiki was correctly deployed to my new dremio instance, however the Code 404 is still there." Does that mean the VDS deployment (including wiki) works in principle and the log entry is simply misleading?

[MaWilly]

Yes, so far the log entry is misleading but the deployment works.