wgzhao / Addax

Addax is a versatile open-source ETL tool that can seamlessly transfer data between various RDBMS and NoSQL databases, making it an ideal solution for data migration.
https://wgzhao.github.io/Addax/
Apache License 2.0
1.2k stars 304 forks source link

[Bug]: null is written to csv and becomes \N,null #1081

Closed liuqian1990 closed 3 months ago

liuqian1990 commented 3 months ago

What happened?

A bug happened! The original table has only 29 columns

Version

4.1.3 (Default)

OS Type

Linux (Default)

Java JDK Version

Oracle JDK 1.8.0

Relevant log output

Error while reading table: xima.test, error message: CSV table references column position 32, but line contains only 32 columns.; byte_offset_to_start_of_line: 32490922446 column_index: 32 column_name: "timestamp_field_32" column_type: TIMESTAMP File: gs://test/test.csv_0d99df356bb6451dacc008a2c5aa7792
liuqian1990 commented 3 months ago

After removing this line of code, the data is normal

  delete sj.add(this.nullFormat) code;

        private String record2String(Record record) {
            StringJoiner sj = new StringJoiner(this.fieldDelimiter + "");
            int columnNum = record.getColumnNumber();
            for (int i=0; i < columnNum; i++) {
                Column column = record.getColumn(i);
                if (column == null || column.asString() == null) {
                    //sj.add(this.nullFormat);
                }
                assert column != null;
                Column.Type type = column.getType();
                if (type == Column.Type.DATE) {
                    SimpleDateFormat sdf = new SimpleDateFormat(this.dateFormat);
                    sj.add(sdf.format(column.asDate()));
                } else {
                    sj.add(column.asString());
                }
            }
            return sj.toString();
        }
wgzhao commented 3 months ago

Please paste the complete running output here

liuqian1990 commented 3 months ago

Sorry addax's log has been deleted. Writing is normal, but reading is wrong now.