sanamhub / postgresql-to-mssql

Migrate postgresql data to sql server on the fly!
20 stars 2 forks source link

Npgsql.NpgsqlException: Exception while reading from stream #20

Open OProf77 opened 1 month ago

OProf77 commented 1 month ago

Thanks for the application. While I was doing the migration I got this error...

06/07/2024 06:37:34 || LOG: Fetched data from public.ana_co_visure table of postgresql...
Npgsql.NpgsqlException: Exception while reading from stream
System.TimeoutException: Timeout during reading attempt

at async ValueTask Npgsql.Internal.NpgsqlConnector.ReadMessageLong(bool async, DataRowLoadingMode dataRowLoadingMode, bool
readingNotifications, bool isReadingPrependedMessage)
at TResult System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder1.StateMachineBox1.System.Threading.Tasks.Sources.IValueTaskSource. GetResult(short token)
at async Task Npgsql.NpgsqlDataReader.Read(bool async, CancellationToken cancellationToken)
at bool Npgsql.NpgsqlDataReader.Read()
at int System.Data.Common.DataAdapter.FillLoadDataRow(SchemaMapping mapping)
at int System.Data.Common.DataAdapter.FillFromReader(DataSet dataset, DataTable datatable, string srcTable, DataReaderContainer dataReader, int startRecord, int maxRecords, DataColumn parentChapterColumn, object parentChapterValue)
at int System.Data.Common.DataAdapter.Fill(DataTable[] dataTables, IDataReader dataReader, int startRecord, int maxRecords)
at void System.Data.DataTable.Load(IDataReader reader, LoadOption loadOption, FillErrorEventHandler errorHandler)
at void Application.Service.<>cDisplayClass3_0.b0(StatusContext ctx) in /postgresql-to-mssql/Services/
Service.cs:95
Npgsql.NpgsqlOperationInProgressException: A command is already in progress: SELECT * FROM public.ana_co_visure at void Npgsql.ThrowHelper.ThrowNpgsqlOperationInProgressException(NpgsqlCommand command)
at UserAction Npgsql.Internal.NpgsqlConnector.g
DoStartUserAction|2790(ConnectorState newState, NpgsqlCommand command, CancellationToken cancellationToken, bool attemptPgCancellation)
at async ValueTask Npgsql.NpgsqlCommand.ExecuteReader(bool async, CommandBehavior behavior, CancellationToken cancellationToken)
at async ValueTask Npgsql.NpgsqlCommand.ExecuteReader(bool async, CommandBehavior behavior, CancellationToken cancellationToken)
at DbDataReader Dapper.SqlMapper.ExecuteReaderWithFlagsFallback(IDbCommand cmd, bool wasClosed, CommandBehavior behavior) in /
/Dapper/SqlMapper.cs:1156
at IEnumerable Dapper.SqlMapper.QueryImpl(IDbConnection cnn, CommandDefinition command, Type effectiveType) in //Dapper/SqlMapper.cs:1184
at System.Collections.Generic.List`1..ctor(IEnumerable collection)
at IEnumerable Dapper.SqlMapper.Query(IDbConnection cnn, string sql, object param, IDbTransaction transaction, bool buffered, int? commandTimeout,
CommandType? commandType) in //Dapper/SqlMapper.cs:737
at void Application.Service.<>c__DisplayClass3_0.b0(StatusContext ctx) in /postgresql-to-mssql/Services/
Service.cs:76
at Task Spectre.Console.Status.<>cDisplayClass14_0.b_0(StatusContext ctx) in //src/Spectre.Console/Live/Status/Status.cs:44
at void Spectre.Console.Status.<>c
DisplayClass16_0.<b0>d.MoveNext() in /_/src/Spectre.Console/Live/Status/Status.cs:79
at void Spectre.Console.Status.<>c__DisplayClass17_0`1.<b
0>d.MoveNext() in /_/src/Spectre.Console/Live/Status/Status.cs:120
at void Spectre.Console.Progress.<>c__DisplayClass28_0`1.<b_0>d.MoveNext() in //src/Spectre.Console/Live/Progress/Progress.cs:133
at async Task Spectre.Console.Internal.DefaultExclusivityMode.RunAsync(Func<Task> func) in //src/Spectre.Console/Internal/
DefaultExclusivityMode.cs:40
at async Task Spectre.Console.Progress.StartAsync(Func<ProgressContext, Task> action) in /
/src/Spectre.Console/Live/Progress/Progress.cs:116
at async Task Spectre.Console.Status.StartAsync(string status, Func<StatusContext, Task> func) in //src/Spectre.Console/Live/Status/Status.cs:117 at async Task Spectre.Console.Status.StartAsync(string status, Func<StatusContext, Task> action) in //src/Spectre.Console/Live/Status/Status.cs:77
at void Spectre.Console.Status.Start(string status, Action action) in /_/src/Spectre.Console/Live/Status/Status.cs:48
at void Application.Service.Migrate() in /postgresql-to-mssql/Services/Service.cs:37
List of failed migration
table/views
Describe the bug A clear and concise description of what the bug is.

Desktop (please complete the following information):

  • I'm using Visual Studio Code on Mac
OProf77 commented 1 month ago

I think that error appears on views

"06/07/2024 06:37:34 || LOG: Fetched data from public.ana_co_visure table of postgresql..."

because "public.ana_co_visure" it's a view and not a table.

Can I specify to migrate only tables and table data?

Thanks

OProf77 commented 4 weeks ago

I have modified this line in Service.cs file: var getTablesQuery = $"SELECT table_name FROM information_schema.tables WHERE table_schema = '{sourceSchema}'"; with this: var getTablesQuery = $"SELECT table_name FROM information_schema.tables WHERE table_schema = '{sourceSchema}' and table_type = 'BASE TABLE'";

...and now all works fine and I have migrated all my tables(only tables without views or more) with data from Postgres to MSSQL. 2,5GB size of Postgres

sanamhub commented 4 weeks ago

Thanks @OProf77 for creating an issue, sorry cannot reach you early but happy you got the solution. I'll keep this issue open and resolve with better approach what I can find.