No custom options for env parameter in ingestions

[SergeyMaslikhov]

Describe the bug I have several instances of Kafka running in different geographic regions. Some of them have the same naming of datasets, so without using env parameter only one of those datasets can be fetched. However env has only these options : {'STG', 'NON_PROD', 'RVW', 'UAT', 'PROD', 'TEST', 'PRE', 'CORP', 'QA', 'DEV', 'EI'} and trying to set another value in config leads to value_error. The issue is that I need to separate Kafka(and any other platform) environments by other value such as "GER", "FRA", etc., otherwise choosing from the listed values misleads users as all of the Kafka instances are actually 'PROD' but in its own region. To Reproduce Steps to reproduce the behavior:

1. Go to Ingestion
2. Click on Create new Kafka source
3. In connection configuration choose YAML and set env : GER
4. See error

Expected behavior Setting custom env parameter without type_error

Screenshots

[deepgarg-visa]

@SergeyMaslikhov, As you mention that they all are PROD but differ in regions. You can actually leverage "platform_instance" key for this.

source:
    type: kafka
    config:
        platform_instance: GER
        env: PROD
        connection:
            bootstrap: ' '
            schema_registry_url: ''

source:
    type: kafka
    config:
        platform_instance: FRA
        env: PROD
        connection:
            bootstrap: ' '
            schema_registry_url: ''

Then the expected urn for the dataset becomes:

urn:li:dataset:(urn:li:dataPlatform:kafka,GER.my-dataset, PROD)

urn:li:dataset:(urn:li:dataPlatform:kafka,FRA.my-dataset, PROD)

[SergeyMaslikhov]

Thank you so much, must've missed in docs