search

Azure / azure-sdk-for-python

This repository is for active development of the Azure SDK for Python. For consumers of the SDK we recommend visiting our public developer docs at https://learn.microsoft.com/python/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-python.
MIT License
4.61k stars 2.83k forks source link

Azure Search IndexingParametersConfiguration needs parameters to be optional to create Indexer with Advanced Properties #33382

Open esgraham opened 11 months ago

esgraham commented 11 months ago

Describe the bug Azure Search IndexingParametersConfiguration needs to be changed to optional parameters.

When using azure-search-documents to create an Indexer, advanced properties could not be set for an indexer with a data connection of Azure Blob or Azure SQL Server.

This has blocked multiple teams when using JSON files in that the JSON values in the file cannot be used with Output Field Mappings. Instead, the values are added to storage blobs as metadata fields, but that could be a security risk and the metadata fields are not available to all Azure Resources when creating blobs (ie, ADF and Logic Apps).

To Reproduce Steps to reproduce the behavior: The following code returned the error: Configuration property 'queryTimeout' is not supported for the data source of type 'azureblob'.

      parameters_config = IndexingParametersConfiguration(parsing_mode="json")
      index_parameters = IndexingParameters(configuration=parameters_config)

      indexer = SearchIndexer( 
          name=indexer_name,  
          skillset_name=skillset_name,  
          target_index_name= index_name,  
          data_source_name= blob_data_source.name,
          parameters=index_parameters)

Conversely, the following code returned the error: Configuration property 'parsingMode' is not supported for the data source of type 'azuresql'

  parameters_config = IndexingParametersConfiguration(query_timeout="00:06:00")
  index_parameters = IndexingParameters(configuration=parameters_config)

  indexer = SearchIndexer(  
      name=indexer_name,  
      skillset_name=skillset_name,  
      target_index_name= index_name,  
      data_source_name= sql_data_source.name,
      parameters=index_parameters)

Expected behavior To be able to create Azure Search Indexers with Advanced Properties

Additional context I created pull request with the changes required and live tests, however, the are auto-generated the changes will not be saved. To see the changes requried view PR #33357

tho-actrecipe commented 11 months ago

@xiangyan99 I encountered the same issue. Can we expect a fix soon?

kboom commented 10 months ago

I'm getting this for a standard configuration from this official tutorial

HttpResponseError: () Configuration property 'queryTimeout' is not supported for the data source of type 'azureblob'.
Code: 
Message: Configuration property 'queryTimeout' is not supported for the data source of type 'azureblob'.
YannInTech commented 10 months ago

I replicated the issue using azureblob data source and azure_search_documents 11.4.0 A workaround is to pass a None query_timeout value configuration = IndexingParametersConfiguration(parsing_mode="jsonArray", query_timeout=None)

ogimgio commented 1 month ago

ok but thats not a solution actually, what if i want to increase the query_timeout