ydb-platform / ydb

YDB is an open source Distributed SQL Database that combines high availability and scalability with strong consistency and ACID transactions
https://ydb.tech
Apache License 2.0
3.8k stars 522 forks source link

Sometimes queries over pg_class are not working from SuperSet #7616

Closed SloNN closed 2 weeks ago

SloNN commented 1 month ago

While receiving table List from YDB Database over PG protocol, SuperSet makes the query below. This query commonly fails, but sometimes this query works. It seems that it depends on session parameters

2024-08-08T20:19:05.727659Z :PGWIRE DEBUG: (#108,[2a02:6b8:c12:3b25:0:5a59:4602:0]:37684) -> [1] 'i' "Initial" Size(4) SSL handshake
2024-08-08T20:19:05.727666Z :PGWIRE DEBUG: (#108,[2a02:6b8:c12:3b25:0:5a59:4602:0]:37684) <- 'N' "Decline SSL"
2024-08-08T20:19:05.730530Z :PGWIRE DEBUG: (#108,[2a02:6b8:c12:3b25:0:5a59:4602:0]:37684) -> [1] 'i' "Initial" Size(70) protocol(0x00000300) user=test database=olap-testing-vla-common2/kikimr/apkobzev/test
2024-08-08T20:19:05.730562Z :LOCAL_PGWIRE DEBUG: TEvAuth protocol(0x00000300) user=test database=olap-testing-vla-common2/kikimr/apkobzev/test  cookie 1
2024-08-08T20:19:05.733619Z :PGWIRE DEBUG: (#108,[2a02:6b8:c12:3b25:0:5a59:4602:0]:37684) <- [1] 'R' "Auth" Size(4) ClearText
2024-08-08T20:19:05.736564Z :PGWIRE DEBUG: (#108,[2a02:6b8:c12:3b25:0:5a59:4602:0]:37684) -> [2] 'p' "PasswordMessage" Size(5)
2024-08-08T20:19:05.736586Z :LOCAL_PGWIRE DEBUG: TEvAuth protocol(0x00000300) user=test database=olap-testing-vla-common2/kikimr/apkobzev/test  cookie 2
2024-08-08T20:19:05.741611Z :PGWIRE DEBUG: (#108,[2a02:6b8:c12:3b25:0:5a59:4602:0]:37684) <- [2] 'R' "Auth" Size(4) OK
2024-08-08T20:19:05.741666Z :LOCAL_PGWIRE DEBUG: TEvConnectionOpened [8:7400865791948396964:5569] cookie 3
2024-08-08T20:19:05.741676Z :LOCAL_PGWIRE DEBUG: Created ydb connection [8:7400865791948396975:4245] num 43
2024-08-08T20:19:05.741689Z :LOCAL_PGWIRE DEBUG: Sent CreateSessionRequest to kqpProxy Request { Database: "/olap-testing-vla-common2/kikimr/apkobzev/test" } PgWire: true UserName: "test"
2024-08-08T20:19:05.742001Z :LOCAL_PGWIRE DEBUG: Received TEvCreateSessionResponse Error: "" Response { SessionId: "ydb://session/3?node_id=8&id=ZDBkZTY0MGMtOTk5OTg2N2YtYWM1ZjlhODAtYmJlZjc5Yg==" NodeId: 8 } YdbStatus: SUCCESS ResourceExhausted: false
2024-08-08T20:19:05.742005Z :LOCAL_PGWIRE DEBUG: Session id is ydb://session/3?node_id=8&id=ZDBkZTY0MGMtOTk5OTg2N2YtYWM1ZjlhODAtYmJlZjc5Yg==
2024-08-08T20:19:05.742014Z :PGWIRE DEBUG: (#108,[2a02:6b8:c12:3b25:0:5a59:4602:0]:37684) <- [3] 'K' "BackendKeyData" Size(8) cancellation PID 1437226402 KEY 1437226369
2024-08-08T20:19:05.742021Z :PGWIRE DEBUG: (#108,[2a02:6b8:c12:3b25:0:5a59:4602:0]:37684) <- [3] 'S' "ParameterStatus" Size(38) server_version=14.5 (ydb stable-23-4)
2024-08-08T20:19:05.742024Z :PGWIRE DEBUG: (#108,[2a02:6b8:c12:3b25:0:5a59:4602:0]:37684) <- [3] 'S' "ParameterStatus" Size(23) IntervalStyle=postgres
2024-08-08T20:19:05.742026Z :PGWIRE DEBUG: (#108,[2a02:6b8:c12:3b25:0:5a59:4602:0]:37684) <- [3] 'S' "ParameterStatus" Size(14) DateStyle=ISO
2024-08-08T20:19:05.742032Z :PGWIRE DEBUG: (#108,[2a02:6b8:c12:3b25:0:5a59:4602:0]:37684) <- [3] 'S' "ParameterStatus" Size(21) client_encoding=UTF8
2024-08-08T20:19:05.742034Z :PGWIRE DEBUG: (#108,[2a02:6b8:c12:3b25:0:5a59:4602:0]:37684) <- [3] 'S' "ParameterStatus" Size(21) server_encoding=UTF8
2024-08-08T20:19:05.742037Z :PGWIRE DEBUG: (#108,[2a02:6b8:c12:3b25:0:5a59:4602:0]:37684) <- [3] 'S' "ParameterStatus" Size(21) integer_datetimes=on
2024-08-08T20:19:05.742040Z :PGWIRE DEBUG: (#108,[2a02:6b8:c12:3b25:0:5a59:4602:0]:37684) <- [3] 'Z' "ReadyForQuery" Size(1) Status: I
2024-08-08T20:19:05.748465Z :PGWIRE DEBUG: (#108,[2a02:6b8:c12:3b25:0:5a59:4602:0]:37684) -> [4] 'Q' "Query" Size(6)  Query: "BEGIN"
2024-08-08T20:19:05.748495Z :LOCAL_PGWIRE DEBUG: TEvQuery [8:7400865791948396964:5569]
2024-08-08T20:19:05.748510Z :LOCAL_PGWIRE DEBUG: TEvSingleQuery [8:7400865791948396964:5569]
2024-08-08T20:19:05.748524Z :LOCAL_PGWIRE DEBUG: Created pgwireKqpProxyQuery: [8:7400865791948396983:4245]
2024-08-08T20:19:05.748592Z :LOCAL_PGWIRE DEBUG: Sent event to kqpProxy Request { SessionId: "ydb://session/3?node_id=8&id=ZDBkZTY0MGMtOTk5OTg2N2YtYWM1ZjlhODAtYmJlZjc5Yg==" KeepSession: true Action: QUERY_ACTION_BEGIN_TX TxControl { begin_tx { serializable_read_write { } } } Database: "/olap-testing-vla-common2/kikimr/apkobzev/test" } UserToken: "\n\004test\022\030\022\026\n\024all-users@well-known\032\352\003eyJhbGciOiJQUzI1NiIsImtpZCI6IjE2In0.eyJhdWQiOiJcL29sYXAtdGVzdGluZy12bGEtY29tbW9uMiIsImV4cCI6MTcyMzE5MTU0NSwiaWF0IjoxNzIzMTQ4MzQ1LCJzdWIiOiJ0ZXN0In0.RqlXyaYEkO7vxhZY8sq8-YvGro7GmhcqP49cmbAYAYHOpqnUr05oBpRNOtc1QllNxZbD4Imo8xG32gC21dmEYcWvCM1EQJUVFARxMu2sspGRzcLzQDwAZXDxhS5xS1OqSXecJDVNeMnLWaceRmRrt8e8toG1r8ePDpPKB5tUMuzgihsqWSPxKAIEqzQcZr6PTj5PzyneqUcGxdVi6Nl9F0sHQ7q-HvNMsMFcGv4HM2587oef6t--bGsvbTScWGsgvtvqgIj_ZVPhhH70kb5jhN9HYIijW3MXHv1hA_ehVWlUR0A3hyzbl8WNwk0WPvYNjLVp3uFNoeztG9Om7uAmkg\"\005Login" RequestActorId { RawX1: 7400865791948396983 RawX2: 4503633987113109 }
2024-08-08T20:19:05.748720Z :LOCAL_PGWIRE DEBUG: Handling TEvKqp::TEvQueryResponse Response { SessionId: "ydb://session/3?node_id=8&id=ZDBkZTY0MGMtOTk5OTg2N2YtYWM1ZjlhODAtYmJlZjc5Yg==" TxMeta { id: "01j4sta8cm9c56r8k2w0thfg62" } QueryDiagnostics: "" ExtraInfo { PgInfo { CommandTag: "BEGIN" } } } YdbStatus: SUCCESS
2024-08-08T20:19:05.748726Z :LOCAL_PGWIRE DEBUG: Finally replying to [8:7400865791948396964:5569] cookie 4
2024-08-08T20:19:05.748730Z :LOCAL_PGWIRE DEBUG: Received TEvProxyCompleted
2024-08-08T20:19:05.748731Z :LOCAL_PGWIRE DEBUG: Updating transaction state to T
2024-08-08T20:19:05.748732Z :LOCAL_PGWIRE DEBUG: Transaction id is 01j4sta8cm9c56r8k2w0thfg62
2024-08-08T20:19:05.748733Z :LOCAL_PGWIRE DEBUG: Session id is ydb://session/3?node_id=8&id=ZDBkZTY0MGMtOTk5OTg2N2YtYWM1ZjlhODAtYmJlZjc5Yg==
2024-08-08T20:19:05.748740Z :PGWIRE DEBUG: (#108,[2a02:6b8:c12:3b25:0:5a59:4602:0]:37684) <- [4] 'C' "CommandComplete" Size(6)  Tag: "BEGIN"
2024-08-08T20:19:05.748746Z :PGWIRE DEBUG: (#108,[2a02:6b8:c12:3b25:0:5a59:4602:0]:37684) <- [4] 'Z' "ReadyForQuery" Size(1) Status: T
2024-08-08T20:19:05.751663Z :PGWIRE DEBUG: (#108,[2a02:6b8:c12:3b25:0:5a59:4602:0]:37684) -> [5] 'Q' "Query" Size(134)  Query: "SELECT c.relname FROM pg_class c JOIN pg_namespace n ON n.oid = c.relnamespace WHERE n.nspname = 'public' AND c.relkind in ('r', 'p')"
2024-08-08T20:19:05.751677Z :LOCAL_PGWIRE DEBUG: TEvQuery [8:7400865791948396964:5569]
2024-08-08T20:19:05.751683Z :LOCAL_PGWIRE DEBUG: TEvSingleQuery [8:7400865791948396964:5569]
2024-08-08T20:19:05.751687Z :LOCAL_PGWIRE DEBUG: Created pgwireKqpProxyQuery: [8:7400865791948396985:4245]
2024-08-08T20:19:05.751721Z :LOCAL_PGWIRE DEBUG: Sent event to kqpProxy Request { SessionId: "ydb://session/3?node_id=8&id=ZDBkZTY0MGMtOTk5OTg2N2YtYWM1ZjlhODAtYmJlZjc5Yg==" Query: "SELECT c.relname FROM pg_class c JOIN pg_namespace n ON n.oid = c.relnamespace WHERE n.nspname = \'public\' AND c.relkind in (\'r\', \'p\')\n" Type: QUERY_TYPE_SQL_GENERIC_QUERY KeepSession: true Action: QUERY_ACTION_EXECUTE TxControl { tx_id: "01j4sta8cm9c56r8k2w0thfg62" } Database: "/olap-testing-vla-common2/kikimr/apkobzev/test" QueryCachePolicy { keep_in_cache: true } UsePublicResponseDataFormat: true Syntax: SYNTAX_PG } UserToken: "\n\004test\022\030\022\026\n\024all-users@well-known\032\352\003eyJhbGciOiJQUzI1NiIsImtpZCI6IjE2In0.eyJhdWQiOiJcL29sYXAtdGVzdGluZy12bGEtY29tbW9uMiIsImV4cCI6MTcyMzE5MTU0NSwiaWF0IjoxNzIzMTQ4MzQ1LCJzdWIiOiJ0ZXN0In0.RqlXyaYEkO7vxhZY8sq8-YvGro7GmhcqP49cmbAYAYHOpqnUr05oBpRNOtc1QllNxZbD4Imo8xG32gC21dmEYcWvCM1EQJUVFARxMu2sspGRzcLzQDwAZXDxhS5xS1OqSXecJDVNeMnLWaceRmRrt8e8toG1r8ePDpPKB5tUMuzgihsqWSPxKAIEqzQcZr6PTj5PzyneqUcGxdVi6Nl9F0sHQ7q-HvNMsMFcGv4HM2587oef6t--bGsvbTScWGsgvtvqgIj_ZVPhhH70kb5jhN9HYIijW3MXHv1hA_ehVWlUR0A3hyzbl8WNwk0WPvYNjLVp3uFNoeztG9Om7uAmkg\"\005Login" RequestActorId { RawX1: 7400865791948396985 RawX2: 4503633987113109 }
2024-08-08T20:19:05.856814Z :LOCAL_PGWIRE DEBUG: Handling TEvKqp::TEvQueryResponse Response { SessionId: "ydb://session/3?node_id=8&id=ZDBkZTY0MGMtOTk5OTg2N2YtYWM1ZjlhODAtYmJlZjc5Yg==" QueryIssues { message: "Check failed: path: \'/olap-testing-vla-common2\', error: path hasn\'t been resolved, nearest resolved path: \'\' (id: <Invalid>)" severity: 1 } QueryIssues { message: "Query invalidated on scheme/internal error during Data execution" issue_code: 2019 severity: 1 } TxMeta { id: "01j4sta8cm9c56r8k2w0thfg62" } } YdbStatus: ABORTED
2024-08-08T20:19:05.856824Z :LOCAL_PGWIRE DEBUG: Finally replying to [8:7400865791948396964:5569] cookie 5
SloNN commented 3 weeks ago

I noticed that queries to regular YDB tables work fine, queries to table pg_namespace work fine.

All problems with access occur only to the pg_class table.

/olap-testing-vla-common2/kikimr/apkobzev/test=> select * from test_row;
 a
---
 1
(1 row)

/olap-testing-vla-common2/kikimr/apkobzev/test=> SELECT * FROM pg_namespace ;
 nspacl |      nspname       | nspowner | oid
--------+--------------------+----------+------
        | information_schema |        1 |    1
        | public             |        1 | 2200
        | pg_catalog         |        1 |   11
(3 rows)

/olap-testing-vla-common2/kikimr/apkobzev/test=> SELECT * FROM pg_class ;
Status: ABORTED
Issues:
<main>: Error: Check failed: path: '/olap-testing-vla-common2', error: path hasn't been resolved, nearest resolved path: '' (id: <Invalid>)
<main>: Error: Query invalidated on scheme/internal error during Data execution, code: 2019

/olap-testing-vla-common2/kikimr/apkobzev/test=>