search

ceumicrodata / mETL

mito ETL tool
161 stars 41 forks source link

Is it possible to populate not null varchar fields with empty strings? #13

Closed e3krisztian closed 11 years ago

e3krisztian commented 11 years ago

There is a .csv file with explicit empty string values in it (...,"","","",...). The target is a postgres database, where the fields are defined as not null.

Running metl will die with 

sqlalchemy.exc.IntegrityError: (IntegrityError) null value in column "..." violates not-null constraint

The relevant code converting the empty string to NULL is at https://github.com/ceumicrodata/mETL/blob/master/metl/fieldtype/base.py#L83

I have managed to temporarily get over this problem with an insert trigger coalescing the problematic fields to '', but I am not really comfortable with this solution, it is a problem during import, so it should be solved within mETL.

korenmiklos commented 11 years ago

@krisztianfekete, can you patch it right there? If I understand correctly, "" is not Null so we should be able to insert it.

bfaludi commented 11 years ago

Will be a new version soon where It will be a new attribute called "nullable" for every field. It will be true by default, but you will able to set to false and will work well with not null constraint.

bfaludi commented 11 years ago

Added the nullable option to the field.