Bad body serialization creating a Schedule Trigger in Data Factory

[csotomon]

• Package Name: azure-mgmt-datafactory
• Package Version: 1.1.0
• Operating System: Windows
• Python Version: 3.8

Describe the bug Using the library to create a Data Factory trigger, calling DataFactoryManagementClient.triggers.create_or_update method is not serializing the request body correctly

Serialized example:

{
    "type": "ScheduleTrigger",
    "description": "My scheduler trigger",
    "pipelines": [
        {
            "pipelineReference": {
                "type": "PipelineReference",
                "referenceName": "PL_ADW_IMPORT_MASTER"
            },
            "parameters": {
                "datacenter_name": "datacenter1",
                "schedule_name": "schedule1"
            }
        }
    ],
    "typeProperties": {
        "recurrence": {
            "frequency": "Minute",
            "interval": 15,
            "startTime": "2022-12-12T04:00:00.000Z",
            "endTime": "2022-12-12T05:00:00.000Z",
            "timeZone": "UTC"
        }
    }
}

The correct serealized body is:

{
    "properties": {
        "type": "ScheduleTrigger",
        "description": "My scheduler trigger",
        "pipelines": [
            {
                "pipelineReference": {
                    "type": "PipelineReference",
                    "referenceName": "PL_ADW_IMPORT_MASTER"
                },
                "parameters": {
                    "datacenter_name": "datacenter1",
                    "schedule_name": "schedule1"
                }
            }
        ],
        "typeProperties": {
            "recurrence": {
                "frequency": "Minute",
                "interval": 15,
                "startTime": "2022-12-12T04:00:00.000Z",
                "endTime": "2022-12-12T05:00:00.000Z",
                "timeZone": "UTC"
            }
        }
    }
}

To Reproduce Steps to reproduce the behavior:

from azure.identity import DefaultAzureCredential 
from azure.mgmt.resource import ResourceManagementClient
from azure.mgmt.datafactory import DataFactoryManagementClient
from azure.mgmt.datafactory.models import *

def main():
    credential = DefaultAzureCredential()
    adf_client = DataFactoryManagementClient(credential, 'AAAAA-VVVV-SWESDDS-XCV3-3bb145207'

    # Create a trigger
    rg_name = 'EUNPADWCANRSG01'
    df_name = 'EUNPADWCANDFA01'
    tr_name = 'mytrigger'
    scheduler_recurrence = ScheduleTriggerRecurrence(frequency='Minute', interval='15',start_time='2022-12-12T04:00:00Z', end_time='2022-12-12T05:00:00Z', time_zone='UTC')
    pipeline_parameters = {'datacenter_name':'datacenter1', 'schedule_name':'schedule1'}
    pipelines_to_run = []
    pipeline_reference = PipelineReference(reference_name='PL_ADW_IMPORT_MASTER')
    pipelines_to_run.append(TriggerPipelineReference(pipeline_reference=pipeline_reference, parameters=pipeline_parameters))
    tr_properties = ScheduleTrigger(description='My scheduler trigger', pipelines = pipelines_to_run, recurrence=scheduler_recurrence)    
    adf_client.triggers.create_or_update(rg_name, df_name, tr_name, tr_properties)

    # Start the trigger
    adf_client.triggers.start(rg_name, df_name, tr_name)

if __name__ == "__main__":
    main()

Expected behavior Running this code,

\azure\mgmt\datafactory\operations_triggers_operations.py trigger an Exception because the response status code is 400

Screenshots If applicable, add screenshots to help explain your problem.

Additional context Add any other context about the problem here.

[swathipil]

@csotomon Thanks for your report! Directing this to the right people.

[msyyc]

According to https://github.com/Azure/azure-sdk-for-python/blob/7c4cfe236d113c59e66515b6315def256219aa6b/sdk/datafactory/azure-mgmt-datafactory/azure/mgmt/datafactory/operations/_triggers_operations.py#L192-L200,

You should use 'TriggerResource' instead of 'ScheduleTrigger'.

[csotomon]

According to

https://github.com/Azure/azure-sdk-for-python/blob/7c4cfe236d113c59e66515b6315def256219aa6b/sdk/datafactory/azure-mgmt-datafactory/azure/mgmt/datafactory/operations/_triggers_operations.py#L192-L200

, You should use 'TriggerResource' instead of 'ScheduleTrigger'.

I am following the example in https://docs.microsoft.com/en-us/azure/data-factory/how-to-create-schedule-trigger#python-sdk. That example creas an object from ScheduleTrigger class. What’s the correct way?

[msyyc]

according to the python package code, it should be tr_properties = TriggerResource(description='My scheduler trigger', pipelines = pipelines_to_run, recurrence=scheduler_recurrence) instead of tr_properties = ScheduleTrigger(description='My scheduler trigger', pipelines = pipelines_to_run, recurrence=scheduler_recurrence)

[csotomon]

Using TriggerResource class instead of ScheduleTrigger, i got this error tr_properties = TriggerResource( TypeError: __init__() missing 1 required keyword-only argument: 'properties'

And the problem is not with the Trigger class, the problem is that TriggersOperations class (this class is used by DataFactoryManagementClient class) is not evolving the JSON message into a properties attribute

[ghost]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @shawnxzq, @lmy269, @Jingshu923.

Issue Details

---

- **Package Name**: azure-mgmt-datafactory - **Package Version**: 1.1.0 - **Operating System**: Windows - **Python Version**: 3.8 **Describe the bug** Using the library to create a Data Factory trigger, calling DataFactoryManagementClient.triggers.create_or_update method is not serializing the request body correctly Serialized example: ```json { "type": "ScheduleTrigger", "description": "My scheduler trigger", "pipelines": [ { "pipelineReference": { "type": "PipelineReference", "referenceName": "PL_ADW_IMPORT_MASTER" }, "parameters": { "datacenter_name": "datacenter1", "schedule_name": "schedule1" } } ], "typeProperties": { "recurrence": { "frequency": "Minute", "interval": 15, "startTime": "2022-12-12T04:00:00.000Z", "endTime": "2022-12-12T05:00:00.000Z", "timeZone": "UTC" } } } ``` The correct serealized body is: ```json { "properties": { "type": "ScheduleTrigger", "description": "My scheduler trigger", "pipelines": [ { "pipelineReference": { "type": "PipelineReference", "referenceName": "PL_ADW_IMPORT_MASTER" }, "parameters": { "datacenter_name": "datacenter1", "schedule_name": "schedule1" } } ], "typeProperties": { "recurrence": { "frequency": "Minute", "interval": 15, "startTime": "2022-12-12T04:00:00.000Z", "endTime": "2022-12-12T05:00:00.000Z", "timeZone": "UTC" } } } } ``` **To Reproduce** Steps to reproduce the behavior: ```python from azure.identity import DefaultAzureCredential from azure.mgmt.resource import ResourceManagementClient from azure.mgmt.datafactory import DataFactoryManagementClient from azure.mgmt.datafactory.models import * def main(): credential = DefaultAzureCredential() adf_client = DataFactoryManagementClient(credential, 'AAAAA-VVVV-SWESDDS-XCV3-3bb145207' # Create a trigger rg_name = 'EUNPADWCANRSG01' df_name = 'EUNPADWCANDFA01' tr_name = 'mytrigger' scheduler_recurrence = ScheduleTriggerRecurrence(frequency='Minute', interval='15',start_time='2022-12-12T04:00:00Z', end_time='2022-12-12T05:00:00Z', time_zone='UTC') pipeline_parameters = {'datacenter_name':'datacenter1', 'schedule_name':'schedule1'} pipelines_to_run = [] pipeline_reference = PipelineReference(reference_name='PL_ADW_IMPORT_MASTER') pipelines_to_run.append(TriggerPipelineReference(pipeline_reference=pipeline_reference, parameters=pipeline_parameters)) tr_properties = ScheduleTrigger(description='My scheduler trigger', pipelines = pipelines_to_run, recurrence=scheduler_recurrence) adf_client.triggers.create_or_update(rg_name, df_name, tr_name, tr_properties) # Start the trigger adf_client.triggers.start(rg_name, df_name, tr_name) if __name__ == "__main__": main() ``` **Expected behavior** Running this code, [\azure\mgmt\datafactory\operations\_triggers_operations.py ](https://github.com/Azure/azure-sdk-for-python/blob/7c4cfe236d113c59e66515b6315def256219aa6b/sdk/datafactory/azure-mgmt-datafactory/azure/mgmt/datafactory/operations/_triggers_operations.py#L254) trigger an Exception because the response status code is 400 **Screenshots** If applicable, add screenshots to help explain your problem. **Additional context** Add any other context about the problem here.

Author:	csotomon
Assignees:	msyyc
Labels:	`bug`, `Data Factory`, `Service Attention`, `Mgmt`, `customer-reported`
Milestone:	-

[lmazuel]

Hi @csotomon I'm closing your PR https://github.com/Azure/azure-sdk-for-python/pull/18963, since this code is auto-generated, meaning we can't fix the code directly. Fix needs to go in OpenAPI description here: https://github.com/Azure/azure-rest-api-specs/tree/master/specification/datafactory/resource-manager

The ADF team was not tagged correctly in your issue and I'm sorry about that, they are tagged now and hopefully should see it and fix the OpenAPI spec. Feel free to email me at microsoft (my email is the same as github alias) if you don't see things moving in a few days.

Thanks,

[shawnxzq]

@lrtoyou1223 Looks like the document for trigger in Python doesn't work as expected, could you please help have a look at this? Thanks!

@csotomon We will check this and update here later, thanks for your patience.

[shawnxzq]

@chez-charlie could you please help on this? It's about creating trigger using python SDK following the doc below, thanks!

https://docs.microsoft.com/en-us/azure/data-factory/how-to-create-schedule-trigger#python-sdk

@csotomon I tagged the wrong owner before, sorry for that.

[csotomon]

@shawnxzq and @chez-charlie how can I help you?

[shawnxzq]

@chez-charlie any update on this? Thanks a lot!

[ghost]

Hi @csotomon. Thank you for opening this issue and giving us the opportunity to assist. We believe that this has been addressed. If you feel that further discussion is needed, please add a comment with the text “/unresolve” to remove the “issue-addressed” label and continue the conversation.

[ghost]

Hi @csotomon, since you haven’t asked that we “/unresolve” the issue, we’ll close this out. If you believe further discussion is needed, please add a comment “/unresolve” to reopen the issue.

[csotomon]

/unresolve

[kazrael2119]

Hi @csotomon ,you can try to use

tr_properties = TriggerResource(properties=ScheduleTrigger(description='My schedulertrigger',pipelines=pipelines_to_run,recurrence=scheduler_recurrence))

instead of

tr_properties = TriggerResource(description='My scheduler trigger', pipelines = pipelines_to_run, recurrence=scheduler_recurrence)

[ghost]

Hi, we're sending this friendly reminder because we haven't heard back from you in a while. We need more information about this issue to help address it. Please be sure to give us your input within the next 7 days. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!

[ahuja-himanshu]

/unresolve

[ahuja-himanshu]

Hi @csotomon ,you can try to use

tr_properties = TriggerResource(properties=ScheduleTrigger(description='My schedulertrigger',pipelines=pipelines_to_run,recurrence=scheduler_recurrence))

instead of

tr_properties = TriggerResource(description='My scheduler trigger', pipelines = pipelines_to_run, recurrence=scheduler_recurrence)

This is throwing ValueError: Unknown type of azure resource: TriggerResource, I've imported TriggerResource from azure.mgmt.datafactory.models