Exception while reading from stream

[BrunoSeixas]

I was changing between Sqlite to Postgres and when i swap "UseSqlite" to "UseNpgsql" it started showing me this error

Basically im following a course and ive repeated every step and even talked with the instructor and we can't understand what is really happening here

watch : Started
Building...
fail: API.Program[0]
      An error occured during migration
      Npgsql.NpgsqlException (0x80004005): Exception while reading from stream
       ---> System.IO.EndOfStreamException: Attempted to read past the end of the stream.
         at Npgsql.NpgsqlReadBuffer.<Ensure>g__EnsureLong|40_0(NpgsqlReadBuffer buffer, Int32 count, Boolean async, Boolean readingNotifications)
         at Npgsql.NpgsqlReadBuffer.<Ensure>g__EnsureLong|40_0(NpgsqlReadBuffer buffer, Int32 count, Boolean async, Boolean readingNotifications)
         at Npgsql.NpgsqlConnector.<ReadMessage>g__ReadMessageLong|194_0(NpgsqlConnector connector, Boolean async, DataRowLoadingMode dataRowLoadingMode, Boolean readingNotifications, Boolean isReadingPrependedMessage)
         at Npgsql.NpgsqlConnector.Authenticate(String username, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken)
         at Npgsql.NpgsqlConnector.Open(NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken)
         at Npgsql.NpgsqlConnection.<>c__DisplayClass41_0.<<Open>g__OpenAsync|0>d.MoveNext()
      --- End of stack trace from previous location ---
         at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.OpenInternalAsync(Boolean errorsExpected, CancellationToken cancellationToken)
         at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.OpenInternalAsync(Boolean errorsExpected, CancellationToken cancellationToken)
         at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.OpenAsync(CancellationToken cancellationToken, Boolean errorsExpected)
         at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlDatabaseCreator.Exists(Boolean async, CancellationToken cancellationToken)
         at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlDatabaseCreator.Exists(Boolean async, CancellationToken cancellationToken)
         at Microsoft.EntityFrameworkCore.Migrations.HistoryRepository.ExistsAsync(CancellationToken cancellationToken)
         at Microsoft.EntityFrameworkCore.Migrations.Internal.Migrator.MigrateAsync(String targetMigration, CancellationToken cancellationToken)
         at API.Program.Main(String[] args) in /Users/6100842/Desktop/Personal/EcoCampo/ecoApi/API/Program.cs:line 30
info: Microsoft.Hosting.Lifetime[0]
      Now listening on: https://localhost:5024
info: Microsoft.Hosting.Lifetime[0]
      Now listening on: http://localhost:5025
info: Microsoft.Hosting.Lifetime[0]
      Application started. Press Ctrl+C to shut down.
info: Microsoft.Hosting.Lifetime[0]
      Hosting environment: Development
info: Microsoft.Hosting.Lifetime[0]
      Content root path: /Users/6100842/Desktop/Personal/EcoCampo/ecoApi/API

Further technical details

Npgsql version: 5.0.7

[vonzshik]

Hello, there was a similar issue raised no so long ago (#3942). Most of the time it happens whenever there is a problem on pg's side - the best thing to do is to check the database logs.

[BrunoSeixas]

@vonzshik Where shall i use that snippet im kinda lost and my brain cells are all dead by now

[vonzshik]

@BrunoSeixas if you're using the official postgres image you might want to raise this issue with them, since it doesn't look like npgsql is at fault here.

[BrunoSeixas]

Ive posted there and im getting sent to other forums etc what i really need is someone able to help me out ive read like a lot of threads and the most equal one was the one u sent and i cant figure it out how to fix it..

[codeex]

ooop， It's terrible

[roji]

For anyone hitting this, what we need in order to investigate and fix this error is a proper code sample which reproduces the error - a simple error message or stack trace simply aren't sufficient. This is especially important as this error could be a result of incorrect user code, where the same connection is accidentally used concurrently (this isn't supported).

Am going to close this for now, but if a proper code sample is submitted I can reopen.

[clane2812]

Hello - i ran into the same issue and solved it, so perhaps i have some informations for other users:

Firstly i had a npgsql-client (6.0.3) working fine with a server with normal connection without ssl. When i started to secure the connection i got the above exception directly when i passed a client-certificate in the 'SSL Certificate' parameter. The exception raised in the call with the same callstack like described from @BrunoSeixas

await serviceProvider.GetService<ConfigurationDataInternal>().Database.MigrateAsync(); Commenting out the migration raised no more exception, but the next call to 'CanConnectAsync' failed ...

looking in wireshark i saw that the TLS-connection was established, the certificate exchanged, but than an 'encrypted alert' was send from the server - i guess this message leads to the exception.

going on i found the following message in the postgres-log

"Common-Name im SSL-Zertifikat enthält Null-Byte"

Looking in the client-certificate in wireshark i saw that really the used certificate had 0 bytes in the CN, cause it was obviously coded with bmpStrings instead of utf8-strings https://stackoverflow.com/questions/5136198/what-strings-are-allowed-in-the-common-name-attribute-in-an-x-509-certificate/5142550#5142550

After some research i found the reason why postgres skips such certificates (funny that e.g. rabbitMq accepts it ...) https://nvd.nist.gov/vuln/detail/CVE-2009-4034

After creating new certificates with the right coding everything worked perfect and i could establish a connection with SSL Mode=VerifyFull and mTLS-Authentication on both sides. The login with the client-certificate instead of user/pw worked fine, too.

So - i hope that i can point someone hitting this issue in the right direction. Perhaps @roji you are interested in the faulty certificate which leads to the exception to debug it. Or you can just present a invalid certificate to your server - think this should lead to the same effect.

[roji]

@clane2812 thanks, yeah - if you can provide such a faulty certificate that would allow us to reproduce the error, we may be able to make Npgsql throw a clearer error.

[clane2812]

@roji ok - i attached the test certificates to reproduce the exception. The SSL Password is empty. cert.zip

[nromano32]

Hello All - I'm getting a very similar error. We are using .net 6.0.7 Npgsql 6.0.5 AWS Arora with RDS proxy. It is appearing in both our staging and prod environments. What is strange is it mainly appears in a simple query on a ver small table (prod ~20 rows staging ~5 rows) and the query is based on primary key. Our load can be high on prod at times but is extremely low in staging.

query select * from public.companies where id = 'GUID'

What is more strange is that it is not a repeatable event. In other words, it only happens sometimes. We have been looking for the issue for about a month now. Any help would be appreciated. It is always this query ... very strange not to see it on random queries if it was a connection issue. We have check the Postgres logs and see no errors of note.

Thanks - Nick

Full error

XXXXX.Base.NpgsqlDB In    at Npgsql.Internal.NpgsqlReadBuffer.<Ensure>g__EnsureLong|41_0(NpgsqlReadBuffer buffer, Int32 count, Boolean async, Boolean readingNotifications)
   at Npgsql.Internal.NpgsqlConnector.<ReadMessage>g__ReadMessageLong|211_0(NpgsqlConnector connector, Boolean async, DataRowLoadingMode dataRowLoadingMode, Boolean readingNotifications, Boolean isReadingPrependedMessage)
   at Npgsql.Internal.NpgsqlConnector.<ReadMessage>g__ReadMessageLong|211_0(NpgsqlConnector connector, Boolean async, DataRowLoadingMode dataRowLoadingMode, Boolean readingNotifications, Boolean isReadingPrependedMessage)
   at Npgsql.NpgsqlDataReader.NextResult(Boolean async, Boolean isConsuming, CancellationToken cancellationToken)
   at Npgsql.NpgsqlDataReader.NextResult()
   at Npgsql.NpgsqlCommand.ExecuteReader(CommandBehavior behavior, Boolean async, CancellationToken cancellationToken)
   at Npgsql.NpgsqlCommand.ExecuteReader(CommandBehavior behavior, Boolean async, CancellationToken cancellationToken)
   at Npgsql.NpgsqlCommand.ExecuteReader(CommandBehavior behavior)
   at Npgsql.NpgsqlCommand.ExecuteDbDataReader(CommandBehavior behavior)
   at System.Data.Common.DbCommand.System.Data.IDbCommand.ExecuteReader(CommandBehavior behavior)
   at System.Data.Common.DbDataAdapter.FillInternal(DataSet dataset, DataTable[] datatables, Int32 startRecord, Int32 maxRecords, String srcTable, IDbCommand command, CommandBehavior behavior)
   at System.Data.Common.DbDataAdapter.Fill(DataSet dataSet, Int32 startRecord, Int32 maxRecords, String srcTable, IDbCommand command, CommandBehavior behavior)
   at System.Data.Common.DbDataAdapter.Fill(DataSet dataSet, String srcTable)
   at CapTrav.Base.NpgsqlDb.Execute_QueryWithOpenConnection(String sqlStr, String tableName, DataSet ds, NpgsqlConnection conn) in C:\Users\nromano\source\repos\CapTrav_core\CapTrav.Base\NpgsqlDB.cs:line 118 with message = Exception while reading from stream in CapTravel with Inner Exception = Attempted to read past the end of the stream. sent message = select * from public.companies  where id = 'GUID'

Code - We open one connection and use it to fill multiple tables. The first query is the one that fails.

public bool FillAllCompanies(CompanyDS companyDS)
        {
            NpgsqlConnection conn = NpgsqlDb.GetConnection(true);

            try
            {
                conn.Open();

                NpgsqlDb.Execute_QueryWithOpenConnection(SqlStrCompanies, companyDS.companies.TableName, companyDS, conn);
                NpgsqlDb.Execute_QueryWithOpenConnection(SqlStrDomains, companyDS.domains.TableName, companyDS, conn);
                NpgsqlDb.Execute_QueryWithOpenConnection(SqlStrDomainRulesLiterals, companyDS.domain_rules_literals.TableName, companyDS, conn);
                NpgsqlDb.Execute_QueryWithOpenConnection(SqlStrDomainRules, companyDS.domain_rules.TableName, companyDS, conn);
                return NpgsqlDb.Execute_QueryWithOpenConnection(SqlStrDomainAuths, companyDS.domain_auths.TableName, companyDS, conn);
            }
            catch (Exception ex) { ErrorHelper.WriteException(ex, ClassName); }
            finally
            {
                if (conn.State == ConnectionState.Open) conn.Close();
                conn.Dispose();
                conn = null;
            }
            return false;
        }

Next level of Code

        public static bool Execute_QueryWithOpenConnection(string sqlStr, string tableName, DataSet ds, NpgsqlConnection conn)
        {

            try
            {
                using (NpgsqlDataAdapter npgsqlDataAdapter = new NpgsqlDataAdapter(sqlStr, conn))
                {
                    npgsqlDataAdapter.Fill(ds, tableName);
                    return true;
                }
            }
            catch (NpgsqlException npge) 
            {
                if (ErrorHelper.WriteException(npge, ClassName, sqlStr))
                    WriteDataSetErrors(ds);
            }
            catch (Exception e) 
            {
                if (ErrorHelper.WriteException(e, ClassName, sqlStr))
                    WriteDataSetErrors(ds);
            }

            return false;
        }

[nromano32]

Hey All - As a follow up, I think we have made some progress. We are using RSD proxy with RDS on AWS. We had set the proxy to a time out of 15 minutes. When we looked at our time to live settings in Postgres 12 they where set at 300 sec (the default for RDS). We are not quit sure how to properly reconcile this mismatch. We believe the proxy has a handle to a connection being closed so the client fails intermittently. The Postgres logs show

[1054]:LOG: could not receive data from client: Connection timed out AND [6321]:LOG: could not receive data from client: Connection reset by peer

We are novice level with Postgres but have extensive experience with MS SQL. We think the issue is caused by the mismatch. Any pointers would be helpful.

Best - Nick

[roji]

@nromano32 you probably want to take a look at keepalives.

[roji]

Closing as no response was provided.

[IndexGit]

Faced with the same issue after deploing to prod, where postgre with Patroni and Odyssey, having server connection pooling. By default Npgsql has pooling enabled. To resolve it you shoudl follow instruction: https://www.npgsql.org/doc/compatibility.html#pgbouncer

In my case we disabled client pooling by adding Pooling=false; to connection string.