search

yugabyte / yugabyte-db

YugabyteDB - the cloud native distributed SQL database for mission-critical applications.
https://www.yugabyte.com
Other
8.99k stars 1.07k forks source link

Dropped tables still appearing in YugaByte Web UI #1653

Open mrpanda opened 5 years ago

mrpanda commented 5 years ago

Jira Link: DB-1634 Hi,

I was running a Python script that was dynamically building table names and creating them in quick succession. There was a mistake in my script and I was able to create the same table twice.

I dropped the tables and the pg catalog seems clean - even allowing me to create the table again and drop it again. But the web UI still reports these 2 tables. I will attach screenshots.

yb-webui-dup-tables yb-yql-n8tbl1
mrpanda commented 5 years ago

I have also copied the Web UI Logs in case that can assist you. yb-webui-logs-n8tbl1.log

mbautin commented 5 years ago

@mrpanda Thank you for reporting this issue! There are still some known issues related to the fact that the PostgreSQL catalog is currently not updated atomically together with YugaByte DB's internal system tables. We will be hopefully addressing these kinds of transactional DDL issues in the next couple of weeks.

mrpanda commented 5 years ago

Hi, rather than copy to catalog tables can the catalog tables be a special view of your system tables so that there is no need for synchronisation ?

mbautin commented 5 years ago

@mrpanda: that is a good suggestion, and this is in fact how we do it for YCQL. However, for YSQL, the format of the internal system tables and PostgreSQL system catalog is substantially different, and the PostgreSQL system catalog has a lot of specific metadata that is missing in YugaByte DB's internal metadata tables. So our plan for addressing this is to implement transactional DDL at YSQL (PostgreSQL system catalog) level, and then do the necessary asynchronous cleanup of the internal tables, which should not block even creation of another table with the same name.

mrpanda commented 5 years ago

Hi Martin,

can I clarify your last statement please. Do you mean that it will be possible to create tables of the same name and that is going to be OK ?

mbautin commented 5 years ago

No, what I am saying is after a PostgreSQL table is deleted, the underlying DocDB table (identified only by its internal UUID) can be cleaned up asynchronously by the master and a new YSQL (PostgreSQL) table with the same name as the old one could be created concurrently with that clean-up process. Of course, from the user's point of view, only one YSQL table identified by a (database, schema, table_name) combination could exist at any given time.

mrpanda commented 5 years ago

Thanks for the clarification. When I get the new version I will re-test trying to create the same table multiple times in short periods of time and see how it behaves.

mbautin commented 5 years ago

I would like to clarify that transactional DDL has not been implemented yet, but is very high on the priority list.

mrpanda commented 5 years ago

Ok, is there an issue or something I can subscribe to so that I can re-test at that point in time ?

mbautin commented 5 years ago

@mrpanda: it is tracked under https://github.com/YugaByte/yugabyte-db/issues/1404 -- will make sure to let you know when it is fixed!