dataplat / dbatools

🚀 SQL Server automation and instance migrations have never been safer, faster or freer
https://dbatools.io
MIT License
2.45k stars 797 forks source link

[Import-DbaCsv] Failure | field header not found. #6553

Closed csdunaway closed 3 years ago

csdunaway commented 4 years ago

While attempting to import a .csv file into a database, I have received this error for the last 3 versions. I am currently running version 1.0.111

This is the full command I am using: Import-DbaCsv -Path 'C:\Temp\TempData.csv' -SqlInstance $ServerName -Database FirePowerData -ParseErrorAction AdvanceToNextLine -SkipEmptyLine -Table TempData

Environmental information

#### PowerShell version:

Name                           Value                                               
----                           -----                                               
PSVersion                      5.1.17763.1007                                      
PSEdition                      Desktop                                             
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}                             
BuildVersion                   10.0.17763.1007                                     
CLRVersion                     4.0.30319.42000                                     
WSManStackVersion              3.0                                                 
PSRemotingProtocolVersion      2.3                                                 
SerializationVersion           1.1.0.1                                             

#### dbatools Module version:

Name    : dbatools
Path    : C:\Program Files\WindowsPowerShell\Modules\dbatools\1.0.111\dbatools.psd1
Version : 1.0.111
# Replace with output of Get-Culture

SQL Server:

/* REPLACE WITH output of @@VERSION */

Microsoft SQL Server 2016 (SP2-CU12) (KB4536648) - 13.0.5698.0 (X64) Feb 15 2020 01:47:30 Copyright (c) Microsoft Corporation Enterprise Edition: Core-based Licensing (64-bit) on Windows Server 2019 Standard 10.0 (Build 17763: )

/* REPLACE WITH output of @@LANGUAGE */

Report

Yes, I have tested this in the Powershell command.

Host used

If anything other than powershell.exe was used, please confirm that you can duplicate the issue with powershell.exe

Errors Received


PSMessageDetails      : 
Exception             : System.Management.Automation.RuntimeException: You cannot call a method on a 
                        null-valued expression.
                           at System.Management.Automation.ExceptionHandlingOps.CheckActionPreference(Functi
                        onContext funcContext, Exception exception)
                           at System.Management.Automation.Interpreter.ActionCallInstruction`2.Run(Interpret
                        edFrame frame)
                           at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(I
                        nterpretedFrame frame)
                           at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(I
                        nterpretedFrame frame)
TargetObject          : 
CategoryInfo          : InvalidOperation: (:) [], RuntimeException
FullyQualifiedErrorId : InvokeMethodOnNull
ErrorDetails          : 
InvocationInfo        : System.Management.Automation.InvocationInfo
ScriptStackTrace      : at Import-DbaCsv<End>, C:\Program 
                        Files\WindowsPowerShell\Modules\dbatools\1.0.108\allcommands.ps1: line 38891
                        at <ScriptBlock>, <No file>: line 1
PipelineIterationInfo : {}

Steps to Reproduce

Import-DbaCsv -Path 'C:\Temp\TempData.csv' -SqlInstance 'RWR-SQL-00-IG' -Database FirePowerData -ParseErrorAction AdvanceToNextLine -SkipEmptyLine -Table TempData

Expected Behavior

I expected success. It used to work with earlier builds, but the last 3 builds have been broken.

Actual Behavior

WARNING: [10:41:06][Import-DbaCsv] Failure | '"DestinationPort"' field header not found. Parameter name: name

csdunaway commented 4 years ago

The last build that worked with this command, is 1.0.60

jaxnoth commented 4 years ago

There is an easy code fix I haven't had time to submit as a bug fix Replace $firstline -split $Delimiter | ForEach-Object { $ColumnMap.Add($PSItem, $PSItem) } with $firstline = Get-Content -Path $file -TotalCount 1 -ErrorAction Stop $ColumnMapping = @{ } $firstline -split $Delimiter | ForEach-Object { $ColumnMapping.Add($, $) } $ColumnMap += $ColumnMapping

wsmelton commented 4 years ago

Can you clarify your post @csdunaway ?

but the last 3 builds have been broken

So were you using build 1.0.108 and it worked? Your latest post states that it doesn't work after build 1.0.60...that is from 7+ months ago.

wsmelton commented 4 years ago

If this worked in previous builds then the test should be updated to prevent it from breaking again...

dpo007 commented 4 years ago

Same problem here. Upgraded from 1.0.61 to 1.0.112 and when I try to import a CSV it gives me the same header error. Reverted back to .61 and it's fine.

dsolodow commented 3 years ago

Verified that it still exists in 1.0.124 and 1.0.128. Observed in Windows PowerShell 5.1 and PowerShell 7.0.3

wsmelton commented 3 years ago

Please provide a csv file that receives this error so that we can probably reproduce it.

dsolodow commented 3 years ago

completed_learners.zip

Here you go.

jpomfret commented 3 years ago

Hey @dsolodow, @dpo007 and @csdunaway could you test out PR #7210 and let me know if it fixes your issues - and that there are no regressions!

Thanks

potatoqualitee commented 3 years ago

This is fixed fa sho in v1.1.2