microsoft / DacFx

DacFx, SqlPackage, and other SQL development libraries enable declarative database development and database portability across SQL versions and environments. Share feedback here on dacpacs, bacpacs, and SQL projects.
https://aka.ms/sqlpackage-ref
MIT License
313 stars 19 forks source link

SSMS hits out of memory errors during bacpac export where nvarchar(max) column has extremely long string #438

Open dzsquared opened 5 months ago

dzsquared commented 5 months ago

x86 apps (SSMS 19, 20) hit out of memory errors during dacfx export where nvarchar(max) column has extremely long string

Steps to Reproduce:

  1. Database with nvarchar(max) column and string at fully maximum size
  2. Run export from SSMS 19/20
  3. out of memory error from SSMS
  4. Run export form sqlpackage dotnet tool
  5. successfully exports bacpac

Did this occur in prior versions? If not - which version(s) did it work in? Repros in all DacFx versions

Data plan execution failed with message One or more errors occurred. (Microsoft.SqlServer.Dac)

------------------------------
Program Location:

   at Microsoft.SqlServer.Dac.Deployment.PlanExecutor.Execute(DeploymentPlan plan, SqlConnectionFactory connectionFactory, Boolean isAzureTarget, LoggingContext loggingContext, CancellationToken cancelToken)
   at Microsoft.SqlServer.Dac.DeployOperation.UpdateDatabaseData(DacDeployOptions dacDeployOptions, LoggingContext loggingContext, CancellationToken token)
   at Microsoft.SqlServer.Dac.DeployOperation.<>c__DisplayClass27_1.<CreatePlanExecutionOperation>b__1()
   at Microsoft.Data.Tools.Schema.Sql.Dac.OperationLogger.Capture(Action action)
   at Microsoft.SqlServer.Dac.DeployOperation.<>c__DisplayClass27_0.<CreatePlanExecutionOperation>b__0(Object operation, CancellationToken token)
   at Microsoft.SqlServer.Dac.Operation.Microsoft.SqlServer.Dac.IOperation.Run(OperationContext context)
   at Microsoft.SqlServer.Dac.ReportMessageOperation.Microsoft.SqlServer.Dac.IOperation.Run(OperationContext context)
   at Microsoft.SqlServer.Dac.OperationExtension.CompositeOperation.Microsoft.SqlServer.Dac.IOperation.Run(OperationContext context)
   at Microsoft.SqlServer.Dac.DeployOperation.Microsoft.SqlServer.Dac.IOperation.Run(OperationContext context)
   at Microsoft.SqlServer.Dac.OperationExtension.Execute(IOperation operation, DacLoggingContext loggingContext, CancellationToken cancellationToken)
   at Microsoft.SqlServer.Dac.DacServices.InternalDeploy(IPackageSource packageSource, Boolean isDacpac, String targetDatabaseName, DacDeployOptions options, CancellationToken cancellationToken, DacLoggingContext loggingContext, Action`3 reportPlanOperation, Boolean executePlan, DacTask action)
   at Microsoft.SqlServer.Dac.DacServices.ImportBacpac(BacPackage package, String targetDatabaseName, DacImportOptions importOptions, Nullable`1 cancellationToken)
   at Microsoft.SqlServer.Dac.DacServices.ImportBacpac(BacPackage package, String targetDatabaseName, DacAzureDatabaseSpecification creationDefaults, Nullable`1 cancellationToken)
   at Microsoft.SqlServer.Management.Dac.DacWizard.ImportDatabase.DoWork()
   at Microsoft.SqlServer.Management.TaskForms.SimpleWorkItem.Run()

===================================

One or more errors occurred. (mscorlib)

------------------------------
Program Location:

   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at Microsoft.Data.Tools.Schema.Sql.Dac.Data.Import.ImportBacpacHelper.ImportBacpac()
   at Microsoft.Data.Tools.Schema.Sql.Dac.Data.ImportTablesStep.Execute()
   at Microsoft.Data.Tools.Schema.Sql.Deployment.Steps.DacBulkCopyStep.Execute(IDbConnection conn, CancellationToken token, LoggingContext loggingContext)
   at Microsoft.Data.Tools.Schema.Sql.Deployment.SqlPlanDeployment.Execute(DeploymentPlanHandle plan, SqlConnectionFactory connectionFactory, String targetDatabaseName, Boolean IsAzureTarget, String databaseScriptPath, String masterScriptPath, CancellationToken cancelToken, LoggingContext loggingContext)
   at Microsoft.SqlServer.Dac.Deployment.PlanExecutor.Execute(DeploymentPlan plan, SqlConnectionFactory connectionFactory, Boolean isAzureTarget, LoggingContext loggingContext, CancellationToken cancelToken)

===================================

One or more errors occurred.

===================================

One or more errors occurred.

===================================

Exception of type 'System.OutOfMemoryException' was thrown. (mscorlib)

------------------------------
Program Location:

   at System.Text.StringBuilder.ExpandByABlock(Int32 minBlockCharCount)
   at System.Text.StringBuilder.Append(Char* value, Int32 valueCount)
   at System.Text.StringBuilder.Append(Char[] value)
   at Microsoft.Data.Tools.Schema.Sql.SqlClient.Bcp.TextTypesSerializer.ReadDataChunksFromBCP(BinaryReader reader)
   at Microsoft.Data.Tools.Schema.Sql.SqlClient.Bcp.TextTypesSerializer.TryGetValue(BinaryReader reader, Object& value)
   at Microsoft.Data.Tools.Schema.Sql.SqlClient.Bcp.ColumnSerializer.GetValue(BinaryReader reader)
   at Microsoft.Data.Tools.Schema.Sql.SqlClient.Bcp.BcpDataReader.GetValue(Int32 i)
   at Microsoft.Data.Tools.Schema.Sql.Dac.Data.Import.ImportTablePartHelper.ReadBCPDataRowsBatch(BcpDataReader bcpReader, Stream stream, BatchInfo batchInfo)
   at Microsoft.Data.Tools.Schema.Sql.Dac.Data.Import.ImportTablePartHelper.<>c__DisplayClass7_1.<ImportTableParts>b__1(BatchInfo batchInfo)
   at Microsoft.Data.Tools.Schema.Sql.Dac.Data.Import.ImportTablePartHelper.StartImportTablePart(Func`2 readDataRowsBatch, Stream stream, Uri partUri, Progress partProgress, Boolean isLastPart)
   at Microsoft.Data.Tools.Schema.Sql.Dac.Data.Import.ImportTablePartHelper.ImportTableParts()
   at Microsoft.Data.Tools.Schema.Sql.Dac.Data.Import.ImportTableHelper.ImportTableParts(ImportTableMetadata tableMetadata, IEnumerable`1 partUris)
   at Microsoft.Data.Tools.Schema.Sql.Dac.Data.Import.ImportTableHelper.<>c__DisplayClass57_0.<StartImportTableParts>b__0()
   at Microsoft.Data.Tools.Schema.Sql.Dac.Logging.DacLogContext.<>c__DisplayClass22_0.<AddContext>b__0()
   at Microsoft.Data.Tools.Schema.Sql.Dac.Data.Import.ImportBacpacHelper.PackageReaderTaskFactory.<>c__DisplayClass5_0.<GetExecutableAction>b__0()
   at Microsoft.Data.Tools.Schema.Sql.Dac.Logging.DacLogContext.<>c__DisplayClass22_0.<AddContext>b__0()
   at System.Threading.Tasks.Task.InnerInvoke()
   at System.Threading.Tasks.Task.Execute()