rdagumampan / yuniql

Free and open source schema versioning and database migration made natively with .NET/6. NEW THIS MAY 2022! v1.3.15 released!
https://yuniql.io
Apache License 2.0
419 stars 64 forks source link

VARBINARY(MAX) not supported for Sql Server's bulk import #307

Open carlvincentld opened 1 year ago

carlvincentld commented 1 year ago

We're using YuniQL to import various data from CSV files into Sql Server, except, when importing hex number in lieu of varbinary(max), we get an error indicating the hex number is expected to be Int16. Except, the hex number we are trying to import is expected to be at least 256 bits (32 bytes). Would it be possible to add support for varbinary(max)?

Yuniql version: 1.3.15

ERR   2023-09-28 18:25:33Z   Failed to execute run function. Input string was not in a correct format.Couldn't store <0x0123456789ABCDFE> in CredentialId Column.  Expected type is Int16.
Diagnostics stack trace captured a System.ArgumentException:
 ---> System.FormatException: Input string was not in a correct format.
   at System.Number.ThrowOverflowOrFormatException(ParsingStatus , TypeCode )
   at System.Int16.Parse(ReadOnlySpan`1 s, NumberStyles style, NumberFormatInfo info)
   at System.Int16.Parse(String s, IFormatProvider provider)
   at System.Convert.ToInt16(String value, IFormatProvider provider)
   at System.String.System.IConvertible.ToInt16(IFormatProvider provider)
   at System.Data.Common.Int16Storage.Set(Int32 , Object )
   at System.Data.DataColumn.set_Item(Int32 , Object )
   --- End of inner exception stack trace ---
   at System.Data.DataColumn.set_Item(Int32 , Object )
   at System.Data.DataTable.NewRecordFromArray(Object[] )
   at System.Data.DataRowCollection.Add(Object[] )
   at Yuniql.SqlServer.SqlServerBulkImportService.ParseCsvFile(String csvFileFullPath, Dictionary`2 columnTypes, String bulkSeparator)
   at Yuniql.SqlServer.SqlServerBulkImportService.Run(IDbConnection connection, IDbTransaction transaction, String fileFullPath, String bulkSeparator, Nullable`1 bulkBatchSize, Nullable`1 commandTimeout, List`1 tokens)
   at Yuniql.Core.MigrationService.<>c__DisplayClass18_0.<RunBulkImportScripts>b__1(String csvFile)
   at System.Collections.Generic.List`1.ForEach(Action`1 )
   at Yuniql.Core.MigrationService.RunBulkImportScripts(IDbConnection connection, IDbTransaction transaction, String workspace, String scriptDirectory, String bulkSeparator, Nullable`1 bulkBatchSize, Nullable`1 commandTimeout, String environment, List`1 tokens)
   at Yuniql.Core.MigrationService.<>c__DisplayClass12_0.<RunVersionDirectories>g__RunVersionDirectoriesInternal|8(IDbConnection connection, IDbTransaction transaction, List`1 scriptSubDirectories, String scriptDirectory, String versionDirectory, Stopwatch stopwatch)
   at Yuniql.Core.MigrationService.<>c__DisplayClass12_0.<RunVersionDirectories>b__7(<>f__AnonymousType0`2 version)   at System.Collections.Generic.List`1.ForEach(Action`1 )
   at Yuniql.Core.MigrationService.RunVersionDirectories(IDbConnection connection, IDbTransaction transaction, List`1 appliedVersions, String workspace, String targetVersion, TransactionContext transactionContext, List`1 tokens, String bulkSeparator, String metaSchemaName, String metaTableName, Nullable`1 commandTimeout, Nullable`1 bulkBatchSize, String appliedByTool, String appliedByToolVersion, String environment, String transactionMode)
   at Yuniql.Core.MigrationService.<Run>g__RunAllInternal|11_3(IDbConnection connection, IDbTransaction transaction, Boolean isRequiredClearedDraft, <>c__DisplayClass11_0& )
   at Yuniql.Core.MigrationService.Run(String workspace, String targetVersion, Nullable`1 isAutoCreateDatabase, List`1 tokens, Nullable`1 isVerifyOnly, String bulkSeparator, String metaSchemaName, String metaTableName, Nullable`1 commandTimeout, Nullable`1 bulkBatchSize, String appliedByTool, String appliedByToolVersion, String environment, Nullable`1 isContinueAfterFailure, String transactionMode, Boolean isRequiredClearedDraft)
   at Yuniql.Core.MigrationService.Run()
   at Yuniql.CLI.CommandLineService.RunRunOption(RunOption opts)
If you think this is a bug, please report an issue here https://github.com/rdagumampan/yuniql/issues.

Thanks in advance!