[YSQL][Upgrade] Direct DMLs on system tables should update system cache

[frozenspider]

Jira Link: DB-3096

Description

When doing direct INSERT/UPDATE/DELETE on system tables, related PG system caches are only updated on INSERT path.

[frozenspider]

Turns out cache update didn't work when oid was filtered from HeapTuple by a junk filter.

CREATE OR REPLACE FUNCTION inc(i integer) RETURNS integer AS $$
  BEGIN RETURN i + 1; END;
$$ LANGUAGE plpgsql;

BEGIN;
UPDATE pg_proc SET proargnames = '{j}' WHERE proname = 'inc';
CREATE OR REPLACE VIEW inc_view AS SELECT inc(j => 10);
-- ERROR: function inc(j => integer) does not exist
--  Hint: No function matches the given name and argument types. You might need to add explicit type casts.

[fizaaluthra]

Previously the upgrade failures due to this issue were hard to reproduce locally (see migrations V43, V31), but now these can be reproduced consistently with ysql_enable_read_request_caching set to true.

[yifanguan]

I have an in-progress diff which partially solves this problem (no mixed DDL & DML in the same txn). However, for alex's example above (direct DML + cache usage within a same txn), I think cache invalidation works properly. We seem to fetch old data from DocDB while loading PG system cache. My hypothesis is CREATE VIEW starts its own YB DDL txn and cannot see uncommitted UPDATE change even if they are within the same "txn".