rdagumampan / yuniql

Free and open source schema versioning and database migration made natively with .NET/6. NEW THIS MAY 2022! v1.3.15 released!
https://yuniql.io
Apache License 2.0
417 stars 63 forks source link

csv data does not distinguish between empty and null strings #284

Open spaquette opened 1 year ago

spaquette commented 1 year ago

I have a simple table

create table [yuniqltest](
    [Id] bigint primary key not null ,
    [testdata] nvarchar(max)
)

I import data in it; one empty line and one null.

"Id","testdata"
"1",""
"2",

However, the result I have in the end is:

Id                   testdata
-------------------- ----------
1                    NULL
2                    NULL

The expected result would be for id 1 to not be null but empty

rdagumampan commented 1 year ago

@spaquette thanks for reaching out and I have limited time lately to respond to issues. Meanwhile, I think it will work if you put NULL as value without quote/double quote. I hope you manage to find a work around anyway. Please reach out if you did still manage to get things to work. Hope to learn more of your use case.

Br, Rodel

spaquette commented 1 year ago

@rdagumampan Thank you for your answer. However, the issue is the double quote results in NULL rather than an empty string.

Abdallah-Darwish commented 1 year ago

@rdagumampan Hey, I want to pick this issue up if you think that it should be fixed, I took a look at the code and I think the issue is coming from the bulk importers where you do:

if (fieldData[i] == "" || fieldData[i] == "NULL")
{
    fieldData[i] = null;
}

So in order to solve this we have to let CsvTextFieldParser.ReadFields() return string?[] and remove the previous check from the importers.

Please let me know WDYT, and if I should dig deeper as I still have some questions about what the new behaviour should be.