apache / seatunnel

SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.
https://seatunnel.apache.org/
Apache License 2.0
7.87k stars 1.78k forks source link

[Bug] [Seatunnel-Web] Conditional Options are not parsed correctly during Dynamic Form Generation #7807

Open shashwatsai opened 5 hours ago

shashwatsai commented 5 hours ago

Search before asking

What happened

While working on contributing to seatunnel-web HDFS datasource interface, as per HdfsFileSourceFactory#OptionRule we should allow users to provide user defined schema for TEXT, CSV, JSON, EXCEL and XML, during dynamic form generation, only for file_format_type TEXT, we are able to provide the schema.

Reason: for conditional options which have an expression with multiple conditions like file_format_type==TEXT || file_format_type==CSV, during form generation, only the first expression is parsed.

.conditional(
    BaseSourceConfigOptions.FILE_FORMAT_TYPE,
    Arrays.asList(
            FileFormat.TEXT,
            FileFormat.JSON,
            FileFormat.EXCEL,
            FileFormat.CSV,
            FileFormat.XML),
    TableSchemaOptions.SCHEMA)

This issue was observed while working on https://github.com/apache/seatunnel/issues/7767

SeaTunnel Version

seatunnel: 2.3.7 seatunnel-web: 1.0.1

SeaTunnel Config

Issue during generation of seatunnel config for HDFS.

Running Command

1. Navigate to seatunnel-web deployment.
2. Try choose HdfsFile as source or sink.
3. choose a file_format_type, only for TEXT, the schema field gets rendered.

Error Exception

Unable to select schema for file format types other than TEXT

Zeta or Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

schema field, actual: Screen Grab:

image

Rendered: For Text:

image

For CSV: No Schema field to provide input.

image

schema field expected: Screen Grab:

image

Expected: (For CSV as well we should see Schema field)

image

Are you willing to submit PR?

Code of Conduct

shashwatsai commented 5 hours ago

CC: @arshadmohammad

shashwatsai commented 4 hours ago

https://github.com/apache/seatunnel/issues/7767 must be handled first.