petl-developers / petl

Python Extract Transform and Load Tables of Data
MIT License
1.25k stars 193 forks source link

call function in etl.convert is not working #484

Open BouchaaraAdil opened 4 years ago

BouchaaraAdil commented 4 years ago

Minimal, reproducible code sample, a copy-pastable example if possible

            temp_table1 = etl.convert(temp_table, 'startTimeTS', lambda v, row: self.calc_starttime(row.created_at, row.startTime), pass_row=True)

Problem description

I'm working on data migration from MySQL to Postgres and wanted to convert NULL fields for rows I coded a separated function that I need to call in etl.convert,

Version and installation information

alimanfoo commented 4 years ago

Hi @BouchaaraAdil, you could try:

temp_table1 = etl.convert(temp_table, 'startTimeTS', lambda v, row: self.calc_starttime(row.created_at, row.startTime), pass_row=True, failonerror=True)

This will cause petl to propagate an except if one is raised by the conversion function.

alimanfoo commented 4 years ago

The failonerror argument can be False (default), True (raise any exception immediately) or "inline" (use exceptions as values in the table). If you want to set the behaviour globally, you can do:

petl.config.failonerror = True
juarezr commented 4 years ago

Reopening: not related to #484