I am trying to use Import-DbaCsv to import large files, and it works great most of the time. But we sometimes get malformed data... I passed the -ParseErrorAction 'AdvanceToNextLine' argument however I am still getting an error and it stops my script.
WARNING: [19:25:55][Import-DbaCsv] Failure | Input string was not in a correct format.
This is my whole command line FWIW.
Import-DbaCsv -path $tmpfileName -SqlInstance $currentServer -ParseErrorAction 'AdvanceToNextLine' -Database 'PWSETL' -Schema $split[0] -Table $split[1] -Truncate -Delimiter '|' -NullValue '' -NoProgress
I am using dbatools v1.1.117.
If I understand the help file correctly, passing the -ParseErrorAction should cause Import-DbaCsv to ignore any individual errors and keep loading, either it doesn't work or I misunderstood. If I misunderstood, can we possibly add such an ability? The files I process are large, usually multi-gigabyte, losing a few rows due to malformed data isn't a big deal.
mike347
commented
2 years ago
Hello.
I am trying to use Import-DbaCsv to import large files, and it works great most of the time. But we sometimes get malformed data... I passed the -ParseErrorAction 'AdvanceToNextLine' argument however I am still getting an error and it stops my script.
WARNING: [19:25:55][Import-DbaCsv] Failure | Input string was not in a correct format.This is my whole command line FWIW.
Import-DbaCsv -path $tmpfileName -SqlInstance $currentServer -ParseErrorAction 'AdvanceToNextLine' -Database 'PWSETL' -Schema $split[0] -Table $split[1] -Truncate -Delimiter '|' -NullValue '' -NoProgressI am using dbatools v1.1.117.
If I understand the help file correctly, passing the -ParseErrorAction should cause Import-DbaCsv to ignore any individual errors and keep loading, either it doesn't work or I misunderstood. If I misunderstood, can we possibly add such an ability? The files I process are large, usually multi-gigabyte, losing a few rows due to malformed data isn't a big deal.