Add support for 2024-05-01-preview

[abhahn]

Motivation and Context

The primary goal of this PR is to add support for the latest preview API version.

Other changes include:

• Refactoring of settings into a separate module using pydantic-settings to do validation and serialization to response payload.
• Added some unit tests and integration tests to ensure refactoring changes didn't break anything.

Description

Contribution Checklist

• [ ] I have built and tested the code locally and in a deployed app
• [ ] For frontend changes, I have pulled the latest code from main, built the frontend, and committed all static files.
• [ ] This is a change for all users of this app. No code or asset is specific to my use case or my organization.
• [ ] I didn't break any existing functionality :smile:

[junan-trustarc]

@abhahn This PR broke datasource as request body doesn't contain datasource. Please check asap

[junan-trustarc]

@abhahn This PR broke datasource as request body doesn't contain datasource. Please check asap

I can see why, Default value of datasource_type was "AzureCognitiveSearch" but this PR changed to None. Can we change back to AzureCognitiveSearch for backward compatibility ?