zzzprojects / Bulk-Operations

C# SQL Bulk Operations | High-performance C# bulk insert, update, delete and merge for SQL Server, SQL Azure, SQL Compact, MySQL, and SQLite.
https://bulk-operations.net
142 stars 34 forks source link

unexpected relation error on PostgresSQL #75

Closed rajkamal0610 closed 1 year ago

rajkamal0610 commented 1 year ago

Hi @JonathanMagnan ,

Am getting the following error, I can't be able to fix this.

Would be please help on the issue

Exception: Npgsql.PostgresException (0x80004005): 42P01: relation "zzzprojects_fa6c7719_f0d8_4b19_aae1_cab71e343aed" does not exist at Npgsql.NpgsqlConnector.<g__ReadMessageLong|194_0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Npgsql.NpgsqlDataReader.d44.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Npgsql.NpgsqlDataReader.NextResult() at Npgsql.NpgsqlCommand.d105.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Npgsql.NpgsqlCommand.d105.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Npgsql.NpgsqlCommand.d93.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Npgsql.NpgsqlCommand.ExecuteNonQuery() at ?.?(DbCommand ?, BulkOperation ?, Int32 ?) at ?.?(? ?, DbCommand ?) at ?.Execute(List`1 actions) at Z.BulkOperations.BulkOperation.Execute()

rajkamal0610 commented 1 year ago

Addition information

We are processing time series data ( vtq ) values in a periodic time using quartz.net.

Each job contains 5000-7000 records and we are using bulk insert options and main objective that all data will get inserted in one single table

JonathanMagnan commented 1 year ago

Hello @rajkamal0610 ,

Do you think you could create a runnable project with the issue? It doesn’t need to be your project, just a new solution with the minimum code to reproduce the issue. You can send it in private here: info@zzzprojects.com

Best Regards,

Jon

rajkamal0610 commented 1 year ago

Hi @JonathanMagnan,

I tried to reproduce the same but the error was appearing in the new project.

I need a few clarifications that the bulk operation method is trying to create a temporary file that's what I understand from the error.

Is there any way to play with the settings for a temporary table

JonathanMagnan commented 1 year ago

Hello @rajkamal0610 ,

I tried to reproduce the same but the error was appearing in the new project.

I believe here you wanted to say that the error WAS NOT appearing in the new project? Otherwise, we will surely want the project.

I need a few clarifications that the bulk operation method is trying to create a temporary file

A bulk insert is made into a temporary table. Later we use this temporary table to perform an UPDATE statement for example, and this is where the temporary table doesn't look to exist anymore and cause this issue.

Is there any way to play with the settings for a temporary table

It depends on what exactly you want to do? You can disable for example the temporary table insert by setting the bulk.TemporaryTableMinRecord = int.MaxValue; options to a very high value (so it will never happen)

Best Regards,

Jon

rajkamal0610 commented 1 year ago

Hi @JonathanMagnan,

We found that the issue is with the pgbouncer setting on pooling mode =transaction, once we changed into session, the error went off

Thanks

Rajkamal S

JonathanMagnan commented 1 year ago

Hello @rajkamal0610 ,

Awesome, thank you for letting us know. It makes sense when I read how this configuration work.

Best Regards,

Jon