[Feature][Connector-V2][Connector-File] NOT NULL Constraints on All Columns When Using JDBC Sink generate_sink_sql with CREATE_SCHEMA_WHEN_NOT_EXIST Option #7979
[X] I had searched in the feature and found no similar feature requirement.
Description
In the File Source to JDBC Sink transmission, we often rely on the generate_auto_sql option. Currently, this option automatically applies a NOT NULL constraint to all generated columns.
However, when data contains null values, the transmission fails due to a NOT NULL constraint violation. I believe this behavior may not be appropriate for the File-to-JDBC case, as it prevents successful data transmission when null values are present in the source data.
If there is a specific reason for setting nullable: false by default, please let me know. Understanding the intent behind this configuration would be very helpful.
Proposed Improvement:
I would like to propose modifying the logic in BaseFileSourceConfig, specifically in the newCatalogTable method, where PhysicalColumn instances are created. This adjustment would set the nullable option to true by default, allowing null values in the data.
This is my first contribution to open source, and I would appreciate any guidance on how to improve my contribution. Thank you!
Usage Scenario
Migration of File Source (Local, S3 ...) to JDBC Sink.
Search before asking
Description
In the File Source to JDBC Sink transmission, we often rely on the generate_auto_sql option. Currently, this option automatically applies a NOT NULL constraint to all generated columns.
However, when data contains null values, the transmission fails due to a NOT NULL constraint violation. I believe this behavior may not be appropriate for the File-to-JDBC case, as it prevents successful data transmission when null values are present in the source data.
If there is a specific reason for setting nullable: false by default, please let me know. Understanding the intent behind this configuration would be very helpful.
Proposed Improvement:
I would like to propose modifying the logic in BaseFileSourceConfig, specifically in the newCatalogTable method, where PhysicalColumn instances are created. This adjustment would set the nullable option to true by default, allowing null values in the data.
This is my first contribution to open source, and I would appreciate any guidance on how to improve my contribution. Thank you!
Usage Scenario
Migration of File Source (Local, S3 ...) to JDBC Sink.
Related issues
No response
Are you willing to submit a PR?
Code of Conduct