search

Azure-Player / azure.synapse.tools

PowerShell module to deploy Synapse workspace (and more) in Microsoft Azure.
MIT License
20 stars 8 forks source link

Unsupported Type: SparkConfigurations #32

Open jreitz43 opened 6 months ago

jreitz43 commented 6 months ago

Deployment throws error "ASWT0029: Unknown object type: SparkConfiguration" when deploying a notebook (example shown below) that has a reference to a custom spark configuration. I believe this may be a very similar issue to #11 where SparkConfiguration would need to be added to the allowed valid types in private/!SynapseObject.class.ps1 as well as private/Get-SynapseObjectByName.ps1 so that it can pretend that the object referenced is valid and exists. Notebook:

{
    "name": "nb_example",
    "properties": {
        "folder": {
            "name": "utility"
        },
        "nbformat": 4,
        "nbformat_minor": 2,
        "bigDataPool": {
            "referenceName": "sparkSm",
            "type": "BigDataPoolReference"
        },
        "targetSparkConfiguration": {
            "referenceName": "CustomSparkConfig",
            "type": "SparkConfigurationReference"
        },

Failure:

**mVERBOSE: A***lyzing notebook dependencies...
**mVERBOSE: Folder: D:\a\***\drop\***\notebook
**mVERBOSE: - nb_example.json
Failure occurred while publishing artifacts to ***
##[debug]Error record:
##[debug]Exception: D:\a\_temp\a560ab36-63de-480f-888a-bc877***f23f0a.ps***:38
##[debug]     |  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
##[debug]     | ASWT0029: Unknown object type: SparkConfiguration.

azure.synapse.tools Version 0.23.000