search

cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
29.9k stars 3.78k forks source link

sql: pgwire OID error during EF Core / Npgsql migration #24173

Closed rkdrnfds closed 5 years ago

rkdrnfds commented 6 years ago

When used ef core cli commands dotnet ef database update to auto generate database tables, it generates error as below.

fail: Microsoft.EntityFrameworkCore.Database.Command[20102]
      Failed executing DbCommand (18ms) [Parameters=[], CommandType='Text', CommandTimeout='30']
      SELECT EXISTS (SELECT 1 FROM pg_catalog.pg_class c JOIN pg_catalog.pg_namespace n ON n.oid=c.relnamespace WHERE c.relname='__EFMigrationsHistory');
System.NotSupportedException: The field 'EXISTS (SELECT 1 FROM pg_catalog.pg_class AS c JOIN pg_catalog.pg_namespace AS n ON n.oid = c.relnamespace WHERE c.relname = '__EFMigrationsHistory')' has a type currently unknown to Npgsql (OID 16). You can retrieve it as a string by marking it as unknown, please see the FAQ.
   at Npgsql.NpgsqlDataReader.GetValue(Int32 ordinal)
   at Npgsql.NpgsqlCommand.<ExecuteScalar>d__87.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at System.Runtime.CompilerServices.ValueTaskAwaiter`1.GetResult()
   at Npgsql.NpgsqlCommand.ExecuteScalar()
   at Microsoft.EntityFrameworkCore.Storage.Internal.RelationalCommand.Execute(IRelationalConnection connection, DbCommandMethod executeMethod, IReadOnlyDictionary`2 parameterValues)

v2.0-alpha.20180129 did not generate those error when doing the same job, so I guess there's some compatibility issue in v2.0-beta.20180319 with postgres driver(Npgsql).

jordanlewis commented 6 years ago

Hi @rkdrnfds,

I don't understand this one. Why does npgsql not know about oid 16, the Postgres bool type? As you can see, the query emitted is a SELECT EXISTS query, which returns a boolean, so this is expected. We haven't changed our pg_type mapping recently so I don't see why this would have changed.

roji commented 6 years ago

Can you guys provide some more info? Which version of Npgsql is being used, and when did this start occurring - on upgrade of an Npgsql version (from which to which), an upgrade of Cockroach...

I'm going to assume this is a Cockroach-side issue for now (since PostgreSQL doesn't seem to be affected), but I'll monitor this issue in case you guys need help/guidance.

FYI the upcoming Npgsql 4.0 will have an extensible type loading API, which allows you to bypass pg_type entirely and load types your own way (or just hardcode them) - take a look at https://github.com/npgsql/npgsql/issues/1486#issuecomment-371192471. Full PostgreSQL compatibility via pg_type (and related tables) is still the best possibility, but this could be relevant for Cockroach. Let me know if you need more details.

rkdrnfds commented 6 years ago

This occured under cockroach v2.0-beta.20180319, Npgsql.EntityFrameworkCore.PostgreSQL 2.1.0-preview1

When upgraded cockroach from 201801xx to 20180319 version.

Additionally, I'm not experiencing this problem since using cockroach 2.0 stable version.

kierenj commented 6 years ago

I've only been using the cockroachdb/cockroach:v2.0.0 docker image and have this issue, unfortunately (docker images doesn't show anything else)

Oops: and <PackageReference Include="Npgsql.EntityFrameworkCore.PostgreSQL" Version="2.0.1" />

kierenj commented 6 years ago

I'm keen to help if I can, it's a bit of a blocker for me. Would it be possible to establish where the issue lies?

(I also get the issue on Npgsql.EntityFrameworkCore.PostgreSQL version 2.1.0-preview1)

To me it sounds like the pg_type table is populated correctly with an oid and typname for (16/bool), and Npgsql agrees on that convention (https://github.com/npgsql/npgsql/blob/dev/src/Npgsql/NpgsqlTypes/NpgsqlDbType.cs#L102).

Npgsql must be assigning an Unknown type handler for this type for some reason.

Am I right in thinking it's either a broken implementation of bools in the wire protocol from CockroachDB, or something about setting up mappings in Npgsql is misinterpreting something? Can anyone share any hints for how to diagnose this?

roji commented 6 years ago

FYI for the Npgsql side I'm away for the next 1.5 weeks or so but will be back and available to discuss

jordanlewis commented 6 years ago

@kierenj it'd be great if you could start your app and get a log of all the metadata queries that get sent to the database. That usually helps narrow things down. You can start cockroach with --vmodule=conn_executor=2 --logtostderr to see lots of detail in stderr about what's going on with the client connection, what's being sent, etc.

kierenj commented 6 years ago

Ok cool, here we are. Note that I'm pretty much following https://www.cockroachlabs.com/docs/stable/start-a-local-cluster-in-docker.html#os-windows, though only 2 nodes, called repro1 and repro2 (and this is output from repro1 via docker logs repro1).

(Running that first query which is looking for types manually does return a row for bool, with oid 16.)

I180415 14:08:46.188352 93 server/status/runtime.go:219  [n1] runtime stats: 148 MiB RSS, 116 goroutines, 104 MiB/58 MiB/175 MiB GO alloc/idle/total, 14 MiB/20 MiB 
CGO alloc/total, 74.70cgo/sec, 0.01/0.00 %(u/s)time, 0.00 %gc (0x)
I180415 14:08:48.442289 1957 sql/conn_executor.go:1816  [n1,client=172.18.0.1:58662,user=kierenj] [NoTxn pos:0] executing PrepareStmt: SELECT ns.nspname, a.typname, 
a.oid, a.typrelid, a.typbasetype, CASE WHEN pg_proc.proname = 'array_recv' THEN 'a' ELSE a.typtype END AS type, CASE WHEN pg_proc.proname = 'array_recv' THEN 
a.typelem WHEN a.typtype = 'r' THEN rngsubtype ELSE 0 END AS elemoid, CASE WHEN pg_proc.proname IN ('array_recv', 'oidvectorrecv') THEN 3 WHEN a.typtype = 'r' THEN 2 
WHEN a.typtype = 'd' THEN 1 ELSE 0 END AS ord FROM pg_type AS a JOIN pg_namespace AS ns ON (ns.oid = a.typnamespace) JOIN pg_proc ON pg_proc.oid = a.typreceive LEFT 
JOIN pg_type AS b ON (b.oid = a.typelem) LEFT JOIN pg_range ON (pg_range.rngtypid = a.oid) WHERE ((a.typtype IN ('b', 'r', 'e', 'd')) AND ((b.typtype IS NULL) OR 
(b.typtype IN ('b', 'r', 'e', 'd')))) OR ((a.typname IN ('record', 'void')) AND (a.typtype = 'p')) ORDER BY ord
I180415 14:08:48.443081 1957 sql/conn_executor.go:1816  [n1,client=172.18.0.1:58662,user=kierenj] [NoTxn pos:1] executing BindStmt: ""->""

I180415 14:08:48.443210 1957 sql/conn_executor.go:1816  [n1,client=172.18.0.1:58662,user=kierenj] [NoTxn pos:2] executing Describe: ""
I180415 14:08:48.443280 1957 sql/conn_executor.go:1816  [n1,client=172.18.0.1:58662,user=kierenj] [NoTxn pos:3] executing ExecPortal name: ""
I180415 14:08:48.443373 1957 sql/conn_executor.go:928  [n1,client=172.18.0.1:58662,user=kierenj] portal resolved to: SELECT ns.nspname, a.typname, a.oid, a.typrelid, 
a.typbasetype, CASE WHEN pg_proc.proname = 'array_recv' THEN 'a' ELSE a.typtype END AS type, CASE WHEN pg_proc.proname = 'array_recv' THEN a.typelem WHEN a.typtype = 
'r' THEN rngsubtype ELSE 0 END AS elemoid, CASE WHEN pg_proc.proname IN ('array_recv', 'oidvectorrecv') THEN 3 WHEN a.typtype = 'r' THEN 2 WHEN a.typtype = 'd' THEN 1 
ELSE 0 END AS ord FROM pg_type AS a JOIN pg_namespace AS ns ON (ns.oid = a.typnamespace) JOIN pg_proc ON pg_proc.oid = a.typreceive LEFT JOIN pg_type AS b ON (b.oid = 
a.typelem) LEFT JOIN pg_range ON (pg_range.rngtypid = a.oid) WHERE ((a.typtype IN ('b', 'r', 'e', 'd')) AND ((b.typtype IS NULL) OR (b.typtype IN ('b', 'r', 'e', 
'd')))) OR ((a.typname IN ('record', 'void')) AND (a.typtype = 'p')) ORDER BY ord

I180415 14:08:48.443503 1957 sql/conn_executor.go:1816  [n1,client=172.18.0.1:58662,user=kierenj] [Open pos:3] executing ExecPortal name: ""
I180415 14:08:48.443585 1957 sql/conn_executor.go:928  [n1,client=172.18.0.1:58662,user=kierenj] portal resolved to: SELECT ns.nspname, a.typname, a.oid, a.typrelid, 
a.typbasetype, CASE WHEN pg_proc.proname = 'array_recv' THEN 'a' ELSE a.typtype END AS type, CASE WHEN pg_proc.proname = 'array_recv' THEN a.typelem WHEN a.typtype = 
'r' THEN rngsubtype ELSE 0 END AS elemoid, CASE WHEN pg_proc.proname IN ('array_recv', 'oidvectorrecv') THEN 3 WHEN a.typtype = 'r' THEN 2 WHEN a.typtype = 'd' THEN 1 
ELSE 0 END AS ord FROM pg_type AS a JOIN pg_namespace AS ns ON (ns.oid = a.typnamespace) JOIN pg_proc ON pg_proc.oid = a.typreceive LEFT JOIN pg_type AS b ON (b.oid = 
a.typelem) LEFT JOIN pg_range ON (pg_range.rngtypid = a.oid) WHERE ((a.typtype IN ('b', 'r', 'e', 'd')) AND ((b.typtype IS NULL) OR (b.typtype IN ('b', 'r', 'e', 
'd')))) OR ((a.typname IN ('record', 'void')) AND (a.typtype = 'p')) ORDER BY ord
I180415 14:08:48.448348 1975 storage/node_liveness.go:667  [n1,s1,r20/1:/Table/{23-50}] incremented n3 liveness epoch to 2
I180415 14:08:48.450270 67 storage/replica_proposal.go:201  [n1,s1,r20/1:/Table/{23-50}] new range lease repl=(n1,s1):1 seq=4 start=1523801328.447061400,0 epo=2 
pro=1523801328.447065900,0 following repl=(n3,s3):3 seq=3 start=1523786865.495545200,1 epo=1 pro=1523787088.316466100,0
I180415 14:08:48.452290 1957 sql/conn_executor.go:1816  [n1,client=172.18.0.1:58662,user=kierenj] [NoTxn pos:4] executing Sync
I180415 14:08:48.487384 1957 sql/conn_executor.go:1816  [n1,client=172.18.0.1:58662,user=kierenj] finishing connExecutor

I180415 14:08:48.530546 2004 sql/conn_executor.go:1816  [n1,client=172.18.0.1:58664,user=kierenj] [NoTxn pos:0] executing PrepareStmt: SELECT ns.nspname, a.typname, 
a.oid, a.typrelid, a.typbasetype, CASE WHEN pg_proc.proname = 'array_recv' THEN 'a' ELSE a.typtype END AS type, CASE WHEN pg_proc.proname = 'array_recv' THEN 
a.typelem WHEN a.typtype = 'r' THEN rngsubtype ELSE 0 END AS elemoid, CASE WHEN pg_proc.proname IN ('array_recv', 'oidvectorrecv') THEN 3 WHEN a.typtype = 'r' THEN 2 
WHEN a.typtype = 'd' THEN 1 ELSE 0 END AS ord FROM pg_type AS a JOIN pg_namespace AS ns ON (ns.oid = a.typnamespace) JOIN pg_proc ON pg_proc.oid = a.typreceive LEFT 
JOIN pg_type AS b ON (b.oid = a.typelem) LEFT JOIN pg_range ON (pg_range.rngtypid = a.oid) WHERE ((a.typtype IN ('b', 'r', 'e', 'd')) AND ((b.typtype IS NULL) OR 
(b.typtype IN ('b', 'r', 'e', 'd')))) OR ((a.typname IN ('record', 'void')) AND (a.typtype = 'p')) ORDER BY ord
I180415 14:08:48.531043 2004 sql/conn_executor.go:1816  [n1,client=172.18.0.1:58664,user=kierenj] [NoTxn pos:1] executing BindStmt: ""->""
I180415 14:08:48.531076 2004 sql/conn_executor.go:1816  [n1,client=172.18.0.1:58664,user=kierenj] [NoTxn pos:2] executing Describe: ""
I180415 14:08:48.531093 2004 sql/conn_executor.go:1816  [n1,client=172.18.0.1:58664,user=kierenj] [NoTxn pos:3] executing ExecPortal name: ""
I180415 14:08:48.531104 2004 sql/conn_executor.go:928  [n1,client=172.18.0.1:58664,user=kierenj] portal resolved to: SELECT ns.nspname, a.typname, a.oid, a.typrelid, 
a.typbasetype, CASE WHEN pg_proc.proname = 'array_recv' THEN 'a' ELSE a.typtype END AS type, CASE WHEN pg_proc.proname = 'array_recv' THEN a.typelem WHEN a.typtype = 
'r' THEN rngsubtype ELSE 0 END AS elemoid, CASE WHEN pg_proc.proname IN ('array_recv', 'oidvectorrecv') THEN 3 WHEN a.typtype = 'r' THEN 2 WHEN a.typtype = 'd' THEN 1 
ELSE 0 END AS ord FROM pg_type AS a JOIN pg_namespace AS ns ON (ns.oid = a.typnamespace) JOIN pg_proc ON pg_proc.oid = a.typreceive LEFT JOIN pg_type AS b ON (b.oid = 
a.typelem) LEFT JOIN pg_range ON (pg_range.rngtypid = a.oid) WHERE ((a.typtype IN ('b', 'r', 'e', 'd')) AND ((b.typtype IS NULL) OR (b.typtype IN ('b', 'r', 'e', 
'd')))) OR ((a.typname IN ('record', 'void')) AND (a.typtype = 'p')) ORDER BY ord
I180415 14:08:48.531140 2004 sql/conn_executor.go:1816  [n1,client=172.18.0.1:58664,user=kierenj] [Open pos:3] executing ExecPortal name: ""

I180415 14:08:48.531163 2004 sql/conn_executor.go:928  [n1,client=172.18.0.1:58664,user=kierenj] portal resolved to: SELECT ns.nspname, a.typname, a.oid, a.typrelid, 
a.typbasetype, CASE WHEN pg_proc.proname = 'array_recv' THEN 'a' ELSE a.typtype END AS type, CASE WHEN pg_proc.proname = 'array_recv' THEN a.typelem WHEN a.typtype = 
'r' THEN rngsubtype ELSE 0 END AS elemoid, CASE WHEN pg_proc.proname IN ('array_recv', 'oidvectorrecv') THEN 3 WHEN a.typtype = 'r' THEN 2 WHEN a.typtype = 'd' THEN 1 
ELSE 0 END AS ord FROM pg_type AS a JOIN pg_namespace AS ns ON (ns.oid = a.typnamespace) JOIN pg_proc ON pg_proc.oid = a.typreceive LEFT JOIN pg_type AS b ON (b.oid = 
a.typelem) LEFT JOIN pg_range ON (pg_range.rngtypid = a.oid) WHERE ((a.typtype IN ('b', 'r', 'e', 'd')) AND ((b.typtype IS NULL) OR (b.typtype IN ('b', 'r', 'e', 
'd')))) OR ((a.typname IN ('record', 'void')) AND (a.typtype = 'p')) ORDER BY ord
I180415 14:08:48.534586 2004 sql/conn_executor.go:1816  [n1,client=172.18.0.1:58664,user=kierenj] [NoTxn pos:4] executing Sync
I180415 14:08:48.557125 2004 sql/conn_executor.go:1816  [n1,client=172.18.0.1:58664,user=kierenj] [NoTxn pos:5] executing PrepareStmt: SELECT EXISTS (SELECT 1 FROM 
pg_catalog.pg_class AS c JOIN pg_catalog.pg_namespace AS n ON n.oid = c.relnamespace WHERE c.relname = '__EFMigrationsHistory')
I180415 14:08:48.557494 2004 sql/conn_executor.go:1816  [n1,client=172.18.0.1:58664,user=kierenj] [NoTxn pos:6] executing BindStmt: ""->""
I180415 14:08:48.557556 2004 sql/conn_executor.go:1816  [n1,client=172.18.0.1:58664,user=kierenj] [NoTxn pos:7] executing Describe: ""
I180415 14:08:48.557632 2004 sql/conn_executor.go:1816  [n1,client=172.18.0.1:58664,user=kierenj] [NoTxn pos:8] executing ExecPortal name: ""

I180415 14:08:48.557679 2004 sql/conn_executor.go:928  [n1,client=172.18.0.1:58664,user=kierenj] portal resolved to: SELECT EXISTS (SELECT 1 FROM pg_catalog.pg_class 
AS c JOIN pg_catalog.pg_namespace AS n ON n.oid = c.relnamespace WHERE c.relname = '__EFMigrationsHistory')
I180415 14:08:48.557809 2004 sql/conn_executor.go:1816  [n1,client=172.18.0.1:58664,user=kierenj] [Open pos:8] executing ExecPortal name: ""
I180415 14:08:48.557868 2004 sql/conn_executor.go:928  [n1,client=172.18.0.1:58664,user=kierenj] portal resolved to: SELECT EXISTS (SELECT 1 FROM pg_catalog.pg_class 
AS c JOIN pg_catalog.pg_namespace AS n ON n.oid = c.relnamespace WHERE c.relname = '__EFMigrationsHistory')
I180415 14:08:48.559414 2004 sql/conn_executor.go:1816  [n1,client=172.18.0.1:58664,user=kierenj] [NoTxn pos:9] executing Sync
I180415 14:08:49.244433 2004 sql/conn_executor.go:1816  [n1,client=172.18.0.1:58664,user=kierenj] finishing connExecutor
I180415 14:08:56.188517 93 server/status/runtime.go:219  [n1] runtime stats: 150 MiB RSS, 116 goroutines, 116 MiB/47 MiB/175 MiB GO alloc/idle/total, 14 MiB/20 MiB 
CGO alloc/total, 96.30cgo/sec, 0.01/0.01 %(u/s)time, 0.00 %gc (0x)

I180415 14:09:06.188416 93 server/status/runtime.go:219  [n1] runtime stats: 152 MiB RSS, 116 goroutines, 125 MiB/38 MiB/175 MiB GO alloc/idle/total, 14 MiB/20 MiB 
CGO alloc/total, 71.30cgo/sec, 0.01/0.00 %(u/s)time, 0.00 %gc (0x)
I180415 14:09:16.188788 93 server/status/runtime.go:219  [n1] runtime stats: 155 MiB RSS, 116 goroutines, 135 MiB/28 MiB/175 MiB GO alloc/idle/total, 14 MiB/20 MiB 
CGO alloc/total, 69.60cgo/sec, 0.01/0.00 %(u/s)time, 0.00 %gc (0x)
I180415 14:09:26.188749 93 server/status/runtime.go:219  [n1] runtime stats: 157 MiB RSS, 116 goroutines, 80 MiB/80 MiB/175 MiB GO alloc/idle/total, 14 MiB/20 MiB CGO 
alloc/total, 70.80cgo/sec, 0.01/0.01 %(u/s)time, 0.00 %gc (1x)
I180415 14:09:36.185636 91 gossip/gossip.go:487  [n1] gossip status (ok, 2 nodes)
gossip client (1/3 cur/max conns)
  2: repro2:26257 (1m25s: infos 107/18 sent/received, bytes 25274B/5290B sent/received)

gossip server (0/3 cur/max conns, infos 113/50 sent/received, bytes 26454B/9412B sent/received)
I180415 14:09:36.188532 93 server/status/runtime.go:219  [n1] runtime stats: 158 MiB RSS, 116 goroutines, 90 MiB/71 MiB/175 MiB GO alloc/idle/total, 14 MiB/20 MiB CGO 
alloc/total, 71.40cgo/sec, 0.01/0.01 %(u/s)time, 0.00 %gc (0x)
I180415 14:09:46.188776 93 server/status/runtime.go:219  [n1] runtime stats: 158 MiB RSS, 116 goroutines, 99 MiB/62 MiB/175 MiB GO alloc/idle/total, 14 MiB/20 MiB CGO 
alloc/total, 77.70cgo/sec, 0.01/0.00 %(u/s)time, 0.00 %gc (0x)

I180415 14:09:56.188800 93 server/status/runtime.go:219  [n1] runtime stats: 159 MiB RSS, 116 goroutines, 109 MiB/53 MiB/175 MiB GO alloc/idle/total, 14 MiB/20 MiB 
CGO alloc/total, 64.50cgo/sec, 0.01/0.00 %(u/s)time, 0.00 %gc (0x)
I180415 14:10:06.188634 93 server/status/runtime.go:219  [n1] runtime stats: 159 MiB RSS, 116 goroutines, 118 MiB/44 MiB/175 MiB GO alloc/idle/total, 14 MiB/20 MiB 
CGO alloc/total, 68.30cgo/sec, 0.01/0.00 %(u/s)time, 0.00 %gc (0x)
I180415 14:10:16.188585 93 server/status/runtime.go:219  [n1] runtime stats: 159 MiB RSS, 116 goroutines, 128 MiB/35 MiB/175 MiB GO alloc/idle/total, 14 MiB/20 MiB 
CGO alloc/total, 74.10cgo/sec, 0.01/0.00 %(u/s)time, 0.00 %gc (0x)
jordanlewis commented 6 years ago

Yeah, I still don't get it. Running that query against CockroachDB and against Postgres produces identical results for the bool row.

I also doubt that it has to do with the wire protocol. Things like this are usually caused by subtle catalog entry differences, or faulty assumptions on the driver side. CockroachDB doesn't guarantee that all catalog OIDs are the same as Postgres for most of these virtual tables (pg_type is an exception), but we do keep the association between tables intact. So, if a driver assumed that other OIDs existed, it'd be an error with us. That doesn't seem like it's happening here, but it probably bears mentioning.

If I were more familiar with dot net, I'd probably pop open a debugger and figure out why the mapping isn't getting set up properly.

kierenj commented 6 years ago

Ah, thanks. I may try to do just that. Really keen on getting my all-kube megademo going ;)

roji commented 6 years ago

Note that Npgsql doesn't actually care what type OID is exposed, it looks for the type name (bool) and loads the OID from that.

kierenj commented 6 years ago

@roji the _byOID dictionary in ConnectorTypeMapper has no entries. There are a load of silent ArgumentExceptions prior to that - is that normal?

Exception has occurred: CLR/System.ArgumentException
Exception thrown: 'System.ArgumentException' in Npgsql.dll: 'A PostgreSQL type with the name hstore was not found in the database'
   at Npgsql.TypeMapping.ConnectorTypeMapper.DoBindType(NpgsqlTypeMapping mapping, NpgsqlConnector connector) in C:\git\kierenj\npgsql\src\Npgsql\TypeMapping\ConnectorTypeMapper.cs:line 328

...and the same for jsonb, record, text, xml, varchar, bpchar, name, json, refcursor, citext, unknown, int2vector, oidvector, date,interval, time, timestamp, timestamptz, timetz, varbit, bit, bool, bytea, char, uuid, void, float8, int2, int4, int8, money, numeric, float4, oid, xid, cid, regtype, cidr, inet, macaddr8, macaddr, tid, box, circle, line, lseg ... and more, it's getting a bit tedious to write these out ;).

In ConnectorTypeMapper.DoBindType, the DatabaseInfo.ByName and DatabaseInfo.ByFullName dictionaries are empty.

(Continuing to dig in..)

...because PostgresDatabaseInfo._types is empty ...because in LoadBackendTypes, the first await reader.ReadAsync() there returns false - i.e. no rows are read from that metadata query.

ExecuteDbDataReaderAsync calls NextResultAsync, which gets BackendMessageCode.EmptyQueryResponse - seems to cause this.

@jordanlewis any way of diagnosing responses to queries, not just logging them coming in?

roji commented 6 years ago

@kierenj this is definitely not normal and explains the errors you're seeing. I'd suggest a Wireshark session to look at exactly what's going back and forth between Npgsql and Cockroach. My guess is that some incompatibility between Cockroach and npgsql's type loading query is causing the empty results (probably one of the joins).

The best way to isolate this is simply to get the type loading query's SQL and execute it outside Npgsql, and then play around with it to find the source of the incompatibility.

As I wrote above, the next version of Npgsql (4.0, out in a couple months) allows plugins to define a custom type loading mechanism. If full PostgreSQL compatibility wrt pg_type and friends is too difficult this could be a good option moving forward.

kierenj commented 6 years ago

@roji cool. I ran the query by pasting it in and it seemed to give good results (via the text interface cockroachdb provides). The differences would be that 1) I'm using that text interface and 2) LoadBackendTypes issues 3 statements in a single batch.

I'm not sure if Wireshark will work to intercept traffic going into a docker container, but I'll see what I can do. ( @jordanlewis if there's more logging for responses I can turn on, or @anyone if you know an intercepting TCP logging proxy whose output would be in a good enough format to you guys in the know, let me know. Tyvm :) )

kierenj commented 6 years ago

Ok, I have a pcap file :) https://www.dropbox.com/s/rfqhyerdnzqbreq/issue-24173.pcap?dl=0

In Wireshark, packet number 48 seems to be the query heading in.

Hex dump of the TCP convo:

00000000  00 00 00 40 00 03 00 00  75 73 65 72 00 6b 69 65   ...@.... user.kie
00000010  72 65 6e 6a 00 63 6c 69  65 6e 74 5f 65 6e 63 6f   renj.cli ent_enco
00000020  64 69 6e 67 00 55 54 46  38 00 64 61 74 61 62 61   ding.UTF 8.databa
00000030  73 65 00 69 6e 66 72 61  73 65 72 76 65 72 00 00   se.infra server..
    00000000  52 00 00 00 08 00 00 00  00                        R....... .
    00000009  53 00 00 00 19 69 6e 74  65 67 65 72 5f 64 61 74   S....int eger_dat
    00000019  65 74 69 6d 65 73 00 6f  6e 00                     etimes.o n.
    00000023  53 00 00 00 19 73 65 72  76 65 72 5f 76 65 72 73   S....ser ver_vers
    00000033  69 6f 6e 00 39 2e 35 2e  30 00                     ion.9.5. 0.
    0000003D  53 00 00 00 23 73 74 61  6e 64 61 72 64 5f 63 6f   S...#sta ndard_co
    0000004D  6e 66 6f 72 6d 69 6e 67  5f 73 74 72 69 6e 67 73   nforming _strings
    0000005D  00 6f 6e 00                                        .on.
    00000061  53 00 00 00 66 63 72 64  62 5f 76 65 72 73 69 6f   S...fcrd b_versio
    00000071  6e 00 43 6f 63 6b 72 6f  61 63 68 44 42 20 43 43   n.Cockro achDB CC
    00000081  4c 20 76 32 2e 30 2e 30  20 28 78 38 36 5f 36 34   L v2.0.0  (x86_64
    00000091  2d 75 6e 6b 6e 6f 77 6e  2d 6c 69 6e 75 78 2d 67   -unknown -linux-g
    000000A1  6e 75 2c 20 62 75 69 6c  74 20 32 30 31 38 2f 30   nu, buil t 2018/0
    000000B1  34 2f 30 33 20 32 30 3a  35 36 3a 30 39 2c 20 67   4/03 20: 56:09, g
    000000C1  6f 31 2e 31 30 29 00                               o1.10).
    000000C8  53 00 00 00 19 63 6c 69  65 6e 74 5f 65 6e 63 6f   S....cli ent_enco
    000000D8  64 69 6e 67 00 55 54 46  38 00                     ding.UTF 8.
    000000E2  53 00 00 00 12 44 61 74  65 53 74 79 6c 65 00 49   S....Dat eStyle.I
    000000F2  53 4f 00                                           SO.
    000000F5  5a 00 00 00 05 49                                  Z....I
00000040  50 00 00 06 52 00 0d 0a  2f 2a 2a 2a 20 4c 6f 61   P...R... /*** Loa
00000050  64 20 61 6c 6c 20 73 75  70 70 6f 72 74 65 64 20   d all su pported 
00000060  74 79 70 65 73 20 2a 2a  2a 2f 0d 0a 53 45 4c 45   types ** */..SELE
00000070  43 54 20 6e 73 2e 6e 73  70 6e 61 6d 65 2c 20 61   CT ns.ns pname, a
00000080  2e 74 79 70 6e 61 6d 65  2c 20 61 2e 6f 69 64 2c   .typname , a.oid,
00000090  20 61 2e 74 79 70 72 65  6c 69 64 2c 20 61 2e 74    a.typre lid, a.t
000000A0  79 70 62 61 73 65 74 79  70 65 2c 0d 0a 43 41 53   ypbasety pe,..CAS
000000B0  45 20 57 48 45 4e 20 70  67 5f 70 72 6f 63 2e 70   E WHEN p g_proc.p
000000C0  72 6f 6e 61 6d 65 3d 27  61 72 72 61 79 5f 72 65   roname=' array_re
000000D0  63 76 27 20 54 48 45 4e  20 27 61 27 20 45 4c 53   cv' THEN  'a' ELS
000000E0  45 20 61 2e 74 79 70 74  79 70 65 20 45 4e 44 20   E a.typt ype END 
000000F0  41 53 20 74 79 70 65 2c  0d 0a 43 41 53 45 0d 0a   AS type, ..CASE..
00000100  20 20 57 48 45 4e 20 70  67 5f 70 72 6f 63 2e 70     WHEN p g_proc.p
00000110  72 6f 6e 61 6d 65 3d 27  61 72 72 61 79 5f 72 65   roname=' array_re
00000120  63 76 27 20 54 48 45 4e  20 61 2e 74 79 70 65 6c   cv' THEN  a.typel
00000130  65 6d 0d 0a 20 20 57 48  45 4e 20 61 2e 74 79 70   em..  WH EN a.typ
00000140  74 79 70 65 3d 27 72 27  20 54 48 45 4e 20 72 6e   type='r'  THEN rn
00000150  67 73 75 62 74 79 70 65  0d 0a 20 20 45 4c 53 45   gsubtype ..  ELSE
00000160  20 30 0d 0a 45 4e 44 20  41 53 20 65 6c 65 6d 6f    0..END  AS elemo
00000170  69 64 2c 0d 0a 43 41 53  45 0d 0a 20 20 57 48 45   id,..CAS E..  WHE
00000180  4e 20 70 67 5f 70 72 6f  63 2e 70 72 6f 6e 61 6d   N pg_pro c.pronam
00000190  65 20 49 4e 20 28 27 61  72 72 61 79 5f 72 65 63   e IN ('a rray_rec
000001A0  76 27 2c 27 6f 69 64 76  65 63 74 6f 72 72 65 63   v','oidv ectorrec
000001B0  76 27 29 20 54 48 45 4e  20 33 20 20 20 20 2f 2a   v') THEN  3    /*
000001C0  20 41 72 72 61 79 73 20  6c 61 73 74 20 2a 2f 0d    Arrays  last */.
000001D0  0a 20 20 57 48 45 4e 20  61 2e 74 79 70 74 79 70   .  WHEN  a.typtyp
000001E0  65 3d 27 72 27 20 54 48  45 4e 20 32 20 20 20 20   e='r' TH EN 2    
000001F0  20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 20                    
00000200  20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 20                    
00000210  20 20 20 20 2f 2a 20 52  61 6e 67 65 73 20 62 65       /* R anges be
00000220  66 6f 72 65 20 2a 2f 0d  0a 20 20 57 48 45 4e 20   fore */. .  WHEN 
00000230  61 2e 74 79 70 74 79 70  65 3d 27 64 27 20 54 48   a.typtyp e='d' TH
00000240  45 4e 20 31 20 20 20 20  20 20 20 20 20 20 20 20   EN 1             
00000250  20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 20                    
00000260  20 20 20 20 20 20 20 20  20 20 20 20 2f 2a 20 44                /* D
00000270  6f 6d 61 69 6e 73 20 62  65 66 6f 72 65 20 2a 2f   omains b efore */
00000280  0d 0a 20 20 45 4c 53 45  20 30 20 20 20 20 20 20   ..  ELSE  0      
00000290  20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 20                    
000002A0  20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 20                    
000002B0  20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 20                    
000002C0  20 20 20 20 20 2f 2a 20  42 61 73 65 20 74 79 70        /*  Base typ
000002D0  65 73 20 66 69 72 73 74  20 2a 2f 0d 0a 45 4e 44   es first  */..END
000002E0  20 41 53 20 6f 72 64 0d  0a 46 52 4f 4d 20 70 67    AS ord. .FROM pg
000002F0  5f 74 79 70 65 20 41 53  20 61 0d 0a 4a 4f 49 4e   _type AS  a..JOIN
00000300  20 70 67 5f 6e 61 6d 65  73 70 61 63 65 20 41 53    pg_name space AS
00000310  20 6e 73 20 4f 4e 20 28  6e 73 2e 6f 69 64 20 3d    ns ON ( ns.oid =
00000320  20 61 2e 74 79 70 6e 61  6d 65 73 70 61 63 65 29    a.typna mespace)
00000330  0d 0a 4a 4f 49 4e 20 70  67 5f 70 72 6f 63 20 4f   ..JOIN p g_proc O
00000340  4e 20 70 67 5f 70 72 6f  63 2e 6f 69 64 20 3d 20   N pg_pro c.oid = 
00000350  61 2e 74 79 70 72 65 63  65 69 76 65 0d 0a 4c 45   a.typrec eive..LE
00000360  46 54 20 4f 55 54 45 52  20 4a 4f 49 4e 20 70 67   FT OUTER  JOIN pg
00000370  5f 63 6c 61 73 73 20 41  53 20 63 6c 73 20 4f 4e   _class A S cls ON
00000380  20 28 63 6c 73 2e 6f 69  64 20 3d 20 61 2e 74 79    (cls.oi d = a.ty
00000390  70 72 65 6c 69 64 29 0d  0a 4c 45 46 54 20 4f 55   prelid). .LEFT OU
000003A0  54 45 52 20 4a 4f 49 4e  20 70 67 5f 74 79 70 65   TER JOIN  pg_type
000003B0  20 41 53 20 62 20 4f 4e  20 28 62 2e 6f 69 64 20    AS b ON  (b.oid 
000003C0  3d 20 61 2e 74 79 70 65  6c 65 6d 29 0d 0a 4c 45   = a.type lem)..LE
000003D0  46 54 20 4f 55 54 45 52  20 4a 4f 49 4e 20 70 67   FT OUTER  JOIN pg
000003E0  5f 63 6c 61 73 73 20 41  53 20 65 6c 65 6d 63 6c   _class A S elemcl
000003F0  73 20 4f 4e 20 28 65 6c  65 6d 63 6c 73 2e 6f 69   s ON (el emcls.oi
00000400  64 20 3d 20 62 2e 74 79  70 72 65 6c 69 64 29 0d   d = b.ty prelid).
00000410  0a 4c 45 46 54 20 4f 55  54 45 52 20 4a 4f 49 4e   .LEFT OU TER JOIN
00000420  20 70 67 5f 72 61 6e 67  65 20 4f 4e 20 28 70 67    pg_rang e ON (pg
00000430  5f 72 61 6e 67 65 2e 72  6e 67 74 79 70 69 64 20   _range.r ngtypid 
00000440  3d 20 61 2e 6f 69 64 29  20 0d 0a 57 48 45 52 45   = a.oid)  ..WHERE
00000450  0d 0a 20 20 61 2e 74 79  70 74 79 70 65 20 49 4e   ..  a.ty ptype IN
00000460  20 28 27 62 27 2c 20 27  72 27 2c 20 27 65 27 2c    ('b', ' r', 'e',
00000470  20 27 64 27 29 20 4f 52  20 20 20 20 20 20 20 20    'd') OR         
00000480  20 2f 2a 20 42 61 73 65  2c 20 72 61 6e 67 65 2c    /* Base , range,
00000490  20 65 6e 75 6d 2c 20 64  6f 6d 61 69 6e 20 2a 2f    enum, d omain */
000004A0  0d 0a 20 20 28 61 2e 74  79 70 74 79 70 65 20 3d   ..  (a.t yptype =
000004B0  20 27 63 27 20 41 4e 44  20 63 6c 73 2e 72 65 6c    'c' AND  cls.rel
000004C0  6b 69 6e 64 3d 27 63 27  29 20 4f 52 20 2f 2a 20   kind='c' ) OR /* 
000004D0  55 73 65 72 2d 64 65 66  69 6e 65 64 20 66 72 65   User-def ined fre
000004E0  65 2d 73 74 61 6e 64 69  6e 67 20 63 6f 6d 70 6f   e-standi ng compo
000004F0  73 69 74 65 73 20 28 6e  6f 74 20 74 61 62 6c 65   sites (n ot table
00000500  20 63 6f 6d 70 6f 73 69  74 65 73 29 20 62 79 20    composi tes) by 
00000510  64 65 66 61 75 6c 74 20  2a 2f 0d 0a 20 20 28 70   default  */..  (p
00000520  67 5f 70 72 6f 63 2e 70  72 6f 6e 61 6d 65 3d 27   g_proc.p roname='
00000530  61 72 72 61 79 5f 72 65  63 76 27 20 41 4e 44 20   array_re cv' AND 
00000540  28 0d 0a 20 20 20 20 62  2e 74 79 70 74 79 70 65   (..    b .typtype
00000550  20 49 4e 20 28 27 62 27  2c 20 27 72 27 2c 20 27    IN ('b' , 'r', '
00000560  65 27 2c 20 27 64 27 29  20 4f 52 20 20 20 20 20   e', 'd')  OR     
00000570  20 20 2f 2a 20 41 72 72  61 79 20 6f 66 20 62 61     /* Arr ay of ba
00000580  73 65 2c 20 72 61 6e 67  65 2c 20 65 6e 75 6d 20   se, rang e, enum 
00000590  64 6f 6d 61 69 6e 20 2a  2f 0d 0a 20 20 20 20 28   domain * /..    (
000005A0  62 2e 74 79 70 74 79 70  65 20 3d 20 27 63 27 20   b.typtyp e = 'c' 
000005B0  41 4e 44 20 65 6c 65 6d  63 6c 73 2e 72 65 6c 6b   AND elem cls.relk
000005C0  69 6e 64 3d 27 63 27 29  20 20 2f 2a 20 41 72 72   ind='c')   /* Arr
000005D0  61 79 20 6f 66 20 75 73  65 72 2d 64 65 66 69 6e   ay of us er-defin
000005E0  65 64 20 66 72 65 65 2d  73 74 61 6e 64 69 6e 67   ed free- standing
000005F0  20 63 6f 6d 70 6f 73 69  74 65 73 20 28 6e 6f 74    composi tes (not
00000600  20 74 61 62 6c 65 20 63  6f 6d 70 6f 73 69 74 65    table c omposite
00000610  73 29 20 2a 2f 0d 0a 20  20 29 29 20 4f 52 0d 0a   s) */..   )) OR..
00000620  20 20 28 61 2e 74 79 70  74 79 70 65 20 3d 20 27     (a.typ type = '
00000630  70 27 20 41 4e 44 20 61  2e 74 79 70 6e 61 6d 65   p' AND a .typname
00000640  20 49 4e 20 28 27 72 65  63 6f 72 64 27 2c 20 27    IN ('re cord', '
00000650  76 6f 69 64 27 29 29 20  20 2f 2a 20 53 6f 6d 65   void'))   /* Some
00000660  20 73 70 65 63 69 61 6c  20 73 75 70 70 6f 72 74    special  support
00000670  65 64 20 70 73 65 75 64  6f 2d 74 79 70 65 73 20   ed pseud o-types 
00000680  2a 2f 0d 0a 4f 52 44 45  52 20 42 59 20 6f 72 64   */..ORDE R BY ord
00000690  00 00 00 42 00 00 00 0e  00 00 00 00 00 00 00 01   ...B.... ........
000006A0  00 00 44 00 00 00 06 50  00 45 00 00 00 09 00 00   ..D....P .E......
000006B0  00 00 00 50 00 00 01 d6  00 2f 2a 2a 2a 20 4c 6f   ...P.... ./*** Lo
000006C0  61 64 20 66 69 65 6c 64  20 64 65 66 69 6e 69 74   ad field  definit
000006D0  69 6f 6e 73 20 66 6f 72  20 28 66 72 65 65 2d 73   ions for  (free-s
000006E0  74 61 6e 64 69 6e 67 29  20 63 6f 6d 70 6f 73 69   tanding)  composi
000006F0  74 65 20 74 79 70 65 73  20 2a 2a 2a 2f 0d 0a 53   te types  ***/..S
00000700  45 4c 45 43 54 20 74 79  70 2e 6f 69 64 2c 20 61   ELECT ty p.oid, a
00000710  74 74 2e 61 74 74 6e 61  6d 65 2c 20 61 74 74 2e   tt.attna me, att.
00000720  61 74 74 74 79 70 69 64  0d 0a 46 52 4f 4d 20 70   atttypid ..FROM p
00000730  67 5f 74 79 70 65 20 41  53 20 74 79 70 0d 0a 4a   g_type A S typ..J
00000740  4f 49 4e 20 70 67 5f 6e  61 6d 65 73 70 61 63 65   OIN pg_n amespace
00000750  20 41 53 20 6e 73 20 4f  4e 20 28 6e 73 2e 6f 69    AS ns O N (ns.oi
00000760  64 20 3d 20 74 79 70 2e  74 79 70 6e 61 6d 65 73   d = typ. typnames
00000770  70 61 63 65 29 0d 0a 4a  4f 49 4e 20 70 67 5f 63   pace)..J OIN pg_c
00000780  6c 61 73 73 20 41 53 20  63 6c 73 20 4f 4e 20 28   lass AS  cls ON (
00000790  63 6c 73 2e 6f 69 64 20  3d 20 74 79 70 2e 74 79   cls.oid  = typ.ty
000007A0  70 72 65 6c 69 64 29 0d  0a 4a 4f 49 4e 20 70 67   prelid). .JOIN pg
000007B0  5f 61 74 74 72 69 62 75  74 65 20 41 53 20 61 74   _attribu te AS at
000007C0  74 20 4f 4e 20 28 61 74  74 2e 61 74 74 72 65 6c   t ON (at t.attrel
000007D0  69 64 20 3d 20 74 79 70  2e 74 79 70 72 65 6c 69   id = typ .typreli
000007E0  64 29 0d 0a 57 48 45 52  45 0d 0a 20 20 28 74 79   d)..WHER E..  (ty
000007F0  70 2e 74 79 70 74 79 70  65 20 3d 20 27 63 27 20   p.typtyp e = 'c' 
00000800  41 4e 44 20 63 6c 73 2e  72 65 6c 6b 69 6e 64 3d   AND cls. relkind=
00000810  27 63 27 29 20 41 4e 44  0d 0a 20 20 61 74 74 6e   'c') AND ..  attn
00000820  75 6d 20 3e 20 30 20 41  4e 44 20 20 20 20 20 2f   um > 0 A ND     /
00000830  2a 20 44 6f 6e 27 74 20  6c 6f 61 64 20 73 79 73   * Don't  load sys
00000840  74 65 6d 20 61 74 74 72  69 62 75 74 65 73 20 2a   tem attr ibutes *
00000850  2f 0d 0a 20 20 4e 4f 54  20 61 74 74 69 73 64 72   /..  NOT  attisdr
00000860  6f 70 70 65 64 0d 0a 4f  52 44 45 52 20 42 59 20   opped..O RDER BY 
00000870  74 79 70 2e 74 79 70 6e  61 6d 65 2c 20 61 74 74   typ.typn ame, att
00000880  2e 61 74 74 6e 75 6d 00  00 00 42 00 00 00 0e 00   .attnum. ..B.....
00000890  00 00 00 00 00 00 01 00  00 44 00 00 00 06 50 00   ........ .D....P.
000008A0  45 00 00 00 09 00 00 00  00 00 50 00 00 00 93 00   E....... ..P.....
000008B0  2f 2a 2a 2a 20 4c 6f 61  64 20 65 6e 75 6d 20 66   /*** Loa d enum f
000008C0  69 65 6c 64 73 20 2a 2a  2a 2f 0d 0a 53 45 4c 45   ields ** */..SELE
000008D0  43 54 20 70 67 5f 74 79  70 65 2e 6f 69 64 2c 20   CT pg_ty pe.oid, 
000008E0  65 6e 75 6d 6c 61 62 65  6c 0d 0a 46 52 4f 4d 20   enumlabe l..FROM 
000008F0  70 67 5f 65 6e 75 6d 0d  0a 4a 4f 49 4e 20 70 67   pg_enum. .JOIN pg
00000900  5f 74 79 70 65 20 4f 4e  20 70 67 5f 74 79 70 65   _type ON  pg_type
00000910  2e 6f 69 64 3d 65 6e 75  6d 74 79 70 69 64 0d 0a   .oid=enu mtypid..
00000920  4f 52 44 45 52 20 42 59  20 6f 69 64 2c 20 65 6e   ORDER BY  oid, en
00000930  75 6d 73 6f 72 74 6f 72  64 65 72 00 00 00 42 00   umsortor der...B.
00000940  00 00 0e 00 00 00 00 00  00 00 01 00 00 44 00 00   ........ .....D..
00000950  00 06 50 00 45 00 00 00  09 00 00 00 00 00 53 00   ..P.E... ......S.
00000960  00 00 04                                           ...
    000000FB  31 00 00 00 04 32 00 00  00 04 54 00 00 00 d0 00   1....2.. ..T.....
    0000010B  08 6e 73 70 6e 61 6d 65  00 00 00 00 00 00 00 00   .nspname ........
    0000011B  00 00 13 ff ff ff ff ff  ff 00 00 74 79 70 6e 61   ........ ...typna
    0000012B  6d 65 00 00 00 00 00 00  00 00 00 00 13 ff ff ff   me...... ........
    0000013B  ff ff ff 00 00 6f 69 64  00 00 00 00 00 00 00 00   .....oid ........
    0000014B  00 00 1a 00 08 ff ff ff  ff 00 00 74 79 70 72 65   ........ ...typre
    0000015B  6c 69 64 00 00 00 00 00  00 00 00 00 00 1a 00 08   lid..... ........
    0000016B  ff ff ff ff 00 00 74 79  70 62 61 73 65 74 79 70   ......ty pbasetyp
    0000017B  65 00 00 00 00 00 00 00  00 00 00 1a 00 08 ff ff   e....... ........
    0000018B  ff ff 00 00 74 79 70 65  00 00 00 00 00 00 00 00   ....type ........
    0000019B  00 00 19 ff ff ff ff ff  ff 00 00 65 6c 65 6d 6f   ........ ...elemo
    000001AB  69 64 00 00 00 00 00 00  00 00 00 00 1a 00 08 ff   id...... ........
    000001BB  ff ff ff 00 00 6f 72 64  00 00 00 00 00 00 00 00   .....ord ........
    000001CB  00 00 14 00 08 ff ff ff  ff 00 00 43 00 00 00 0d   ........ ...C....
    000001DB  53 45 4c 45 43 54 20 30  00 31 00 00 00 04 32 00   SELECT 0 .1....2.
    000001EB  00 00 04 54 00 00 00 51  00 03 6f 69 64 00 00 00   ...T...Q ..oid...
    000001FB  00 00 00 00 00 00 00 1a  00 08 ff ff ff ff 00 00   ........ ........
    0000020B  61 74 74 6e 61 6d 65 00  00 00 00 00 00 00 00 00   attname. ........
    0000021B  00 13 ff ff ff ff ff ff  00 00 61 74 74 74 79 70   ........ ..atttyp
    0000022B  69 64 00 00 00 00 00 00  00 00 00 00 1a 00 08 ff   id...... ........
    0000023B  ff ff ff 00 00 43 00 00  00 0d 53 45 4c 45 43 54   .....C.. ..SELECT
    0000024B  20 30 00 31 00 00 00 04  32 00 00 00 04 54 00 00    0.1.... 2....T..
    0000025B  00 38 00 02 6f 69 64 00  00 00 00 00 00 00 00 00   .8..oid. ........
    0000026B  00 1a 00 08 ff ff ff ff  00 00 65 6e 75 6d 6c 61   ........ ..enumla
    0000027B  62 65 6c 00 00 00 00 00  00 00 00 00 00 19 ff ff   bel..... ........
    0000028B  ff ff ff ff 00 00 43 00  00 00 0d 53 45 4c 45 43   ......C. ...SELEC
    0000029B  54 20 30 00 5a 00 00 00  05 49                     T 0.Z... .I
00000963  58 00 00 00 04                                     X....
jordanlewis commented 6 years ago

Thanks! It sure looks like nothing's coming back from the server. Is there any way you could make a small proof of concept app that demonstrates this behavior that I could play around with myself?

Someone made an attempt here https://github.com/cockroachdb/cockroach/issues/24172, but I couldn't observe the issue using that sample.

kierenj commented 6 years ago

Ok - this does it for me. I made it as minimal as possible (1 code file with 2 lines of code, 1 project file with 1 dependency) :)

https://github.com/kierenj/cockroach-24173-repro

Is there anything else I can provide?

Oh I guess in terms of the cockroach environment: as I mentioned it's the 2.0.0 docker image, using (PowerShell syntax under Windows):

docker run -d `
 --name=repro1 `
 --hostname=repro1 `
 --net=roachnet `
 -p 26257:26257 -p 8080:8080 `
 -v "//c/Users/kiere/cockroach-data/roach1:/cockroach/cockroach-data" `
 cockroachdb/cockroach:v2.0.0 start --insecure --vmodule=conn_executor=2 --logtostderr

Then:

 docker run -d `
 --name=repro2 `
 --hostname=repro2 `
 --net=roachnet `
 -v "//c/Users/kiere/cockroach-data/roach2:/cockroach/cockroach-data" `
 cockroachdb/cockroach:v2.0.0 start --insecure --join=repro1 --vmodule=conn_executor=2 --logtostderr

Setup consists of CREATE USER kierenj; CREATE DATABASE postgres; CREATE DATABASE infraserver;.

kierenj commented 6 years ago

I'm not pushing, but just curious - the 2.1 milestone, is there a rough timeline on the roadmap for it?

knz commented 5 years ago

We have made significant strides on this area of the code in CockroachDB 2.1. We would appreciate if you could attempt to reproduce the issue with the latest 2.1 version and report your results.

Thanks

kierenj commented 5 years ago

@knz I don't see any 2.1 docker image, so I can't repro in the same way I did before. But the git repo I supplied is very straightforward to run - if you have a 2.1 running, it might be easier that way? Else, is there somewhere you can point me for a 2.1 image please?

knz commented 5 years ago

@kierenj we don't currently have an environment ready for windows testing -- this will come later in the release cycle.

However you can find a docker image for cockroachdb 2.1 here: https://hub.docker.com/r/cockroachdb/cockroach-unstable/

Cheers

kierenj commented 5 years ago

Its not windows-specific?

knz commented 5 years ago

.NET testing falls under the windows testing umbrella for us

kierenj commented 5 years ago

Ill give it a go

kierenj commented 5 years ago

Unfortunately I get a very similar exception still:

The field 'exists' has a type currently unknown to Npgsql (OID 16). You can retrieve it as a string by marking it as unknown, please see the FAQ.

This looks to be the same case as above, so hopefully the packet capture above is still handy :)

jordanlewis commented 5 years ago

Hey there @kierenj,

Thanks again for the repro. This issue is solved in 19.1, and your repro case seems to execute successfully. I downloaded cockroach-24173-repro and ran:

$ dotnet build && dotnet run
Microsoft (R) Build Engine version 15.9.20+g88f5fadfbe for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  Restore completed in 64.65 ms for /Users/jordan/ext/cockroach-24173-repro/Repro.csproj.
  Repro -> /Users/jordan/ext/cockroach-24173-repro/bin/Debug/netcoreapp2.0/Repro.dll

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:01.35
$

So, going to close this out. Please re-open if you have further issues!