Closed dayalus closed 1 year ago
Hi,
Is anyone having issues deploying datasets? Linked services, dataflows, etc. work fine. However, of late, all our datasets are not getting deployed.
Here's a snippet from the logs:
2023-06-01T06:19:17.9446572Z Azure Data Factory exists. 2023-06-01T06:19:17.9485210Z =================================================================================== 2023-06-01T06:19:17.9495870Z STEP: Reading Azure Data Factory from JSON files... 2023-06-01T06:19:18.2580680Z IntegrationRuntimes: 3 object(s) loaded. 2023-06-01T06:19:19.8060934Z LinkedServices: 54 object(s) loaded. 2023-06-01T06:21:14.0226392Z Pipelines: 188 object(s) loaded. 2023-06-01T06:21:36.2502433Z DataSets: 521 object(s) loaded. 2023-06-01T06:21:44.9889263Z DataFlows: 99 object(s) loaded. 2023-06-01T06:21:46.3668523Z Triggers: 63 object(s) loaded. 2023-06-01T06:21:46.3768432Z Managed VNet: 1 object(s) loaded. 2023-06-01T06:21:46.3961335Z Managed Private Endpoints: 1 object(s) loaded. 2023-06-01T06:21:46.3997584Z Credentials: 0 object(s) loaded. 2023-06-01T06:21:46.4295994Z Factories: 1 object(s) loaded. 2023-06-01T06:21:46.5155202Z =================================================================================== 2023-06-01T06:21:46.5167793Z STEP: Pre-deployment 2023-06-01T06:21:46.5178828Z =================================================================================== 2023-06-01T06:21:46.5189996Z STEP: Replacing all properties environment-related... 2023-06-01T06:21:46.5218214Z Stage parameter was not provided - action skipped. 2023-06-01T06:21:46.5229377Z =================================================================================== 2023-06-01T06:21:46.5240996Z STEP: Determining the objects to be deployed... 2023-06-01T06:21:47.4702225Z # Number of objects marked as to be deployed: 2/931 2023-06-01T06:21:47.4861645Z - [dataflow].[DataFlowxxx] 2023-06-01T06:21:47.4876918Z - [dataflow].[DataFlowyyy] 2023-06-01T06:21:47.4889025Z =================================================================================== 2023-06-01T06:21:47.4899210Z STEP: Stopping triggers... 2023-06-01T06:21:47.4924110Z Operation skipped as publish option 'StopStartTriggers' = false 2023-06-01T06:21:47.4935837Z =================================================================================== 2023-06-01T06:21:47.5057167Z STEP: Deployment of all ADF objects... 2023-06-01T06:21:47.7922421Z Start deploying object: [dataflow].[DataFlowxx] (4 dependency/ies) 2023-06-01T06:21:47.9616294Z ##[warning]ADFT0006: Referenced object [Dataset].[AzureDataLakeStorexx] was not found. No error raised as user wanted to carry on. 2023-06-01T06:21:48.7922961Z ##[error]BadRequest : The document creation or update failed because of invalid reference 'azuredatalakestoreyyy'. CorrelationId: 5896fc05-f0d3-49ea-bbbb-ee293c138af8
Please ignore it for now. We think it may be an error on our side with naming the JSON files..Will close this if confirmed
Yes, it was a dataset type error from our side. My apologies!
Hi,
Is anyone having issues deploying datasets? Linked services, dataflows, etc. work fine. However, of late, all our datasets are not getting deployed.
Here's a snippet from the logs:
2023-06-01T06:19:17.9446572Z Azure Data Factory exists. 2023-06-01T06:19:17.9485210Z =================================================================================== 2023-06-01T06:19:17.9495870Z STEP: Reading Azure Data Factory from JSON files... 2023-06-01T06:19:18.2580680Z IntegrationRuntimes: 3 object(s) loaded. 2023-06-01T06:19:19.8060934Z LinkedServices: 54 object(s) loaded. 2023-06-01T06:21:14.0226392Z Pipelines: 188 object(s) loaded. 2023-06-01T06:21:36.2502433Z DataSets: 521 object(s) loaded. 2023-06-01T06:21:44.9889263Z DataFlows: 99 object(s) loaded. 2023-06-01T06:21:46.3668523Z Triggers: 63 object(s) loaded. 2023-06-01T06:21:46.3768432Z Managed VNet: 1 object(s) loaded. 2023-06-01T06:21:46.3961335Z Managed Private Endpoints: 1 object(s) loaded. 2023-06-01T06:21:46.3997584Z Credentials: 0 object(s) loaded. 2023-06-01T06:21:46.4295994Z Factories: 1 object(s) loaded. 2023-06-01T06:21:46.5155202Z =================================================================================== 2023-06-01T06:21:46.5167793Z STEP: Pre-deployment 2023-06-01T06:21:46.5178828Z =================================================================================== 2023-06-01T06:21:46.5189996Z STEP: Replacing all properties environment-related... 2023-06-01T06:21:46.5218214Z Stage parameter was not provided - action skipped. 2023-06-01T06:21:46.5229377Z =================================================================================== 2023-06-01T06:21:46.5240996Z STEP: Determining the objects to be deployed... 2023-06-01T06:21:47.4702225Z # Number of objects marked as to be deployed: 2/931 2023-06-01T06:21:47.4861645Z - [dataflow].[DataFlowxxx] 2023-06-01T06:21:47.4876918Z - [dataflow].[DataFlowyyy] 2023-06-01T06:21:47.4889025Z =================================================================================== 2023-06-01T06:21:47.4899210Z STEP: Stopping triggers... 2023-06-01T06:21:47.4924110Z Operation skipped as publish option 'StopStartTriggers' = false 2023-06-01T06:21:47.4935837Z =================================================================================== 2023-06-01T06:21:47.5057167Z STEP: Deployment of all ADF objects... 2023-06-01T06:21:47.7922421Z Start deploying object: [dataflow].[DataFlowxx] (4 dependency/ies) 2023-06-01T06:21:47.9616294Z ##[warning]ADFT0006: Referenced object [Dataset].[AzureDataLakeStorexx] was not found. No error raised as user wanted to carry on. 2023-06-01T06:21:48.7922961Z ##[error]BadRequest : The document creation or update failed because of invalid reference 'azuredatalakestoreyyy'. CorrelationId: 5896fc05-f0d3-49ea-bbbb-ee293c138af8