laurenz / oracle_fdw

PostgreSQL Foreign Data Wrapper for Oracle
http://laurenz.github.io/oracle_fdw/
Other
494 stars 157 forks source link

Upgrade From Postgresql 11 to Postgresql 15 was failed due to prefetch - oid null value. #711

Open Post-User2024 opened 5 days ago

Post-User2024 commented 5 days ago

We have updated below extension on PostgreSQL 11 before proceeding the PostgreSQL 15 upgrade.

On PostgreSQL 11

Oracle FDW - 1.1 To 1.2 orafce - 3.7 To 4.9

While updating the orafce for one B-database getting below error. For other A-database it was updated successfully.

A - Database:

orafce - 4.9

B-database


orafce - 3.7 <<<<<<<< Unable to update the extension.

alter extension orafce update; ERROR: type "oracle.date" does not exist

drop extension orafce; ERROR: cache lookup failed for type XXXXXXX

drop extension orafce restrict; ERROR: cache lookup failed for type XXXXXXX

drop extension orafce cascade; ERROR: cache lookup failed for type XXXXXX

Since there was no update on the extension we took once to initialize the upgrade with dry run it went fine there was no issue with dry run.

When we start the actual upgrade it got failed with below error.

pg_restore: error: could not execute query: ERROR: invalid value for option "prefetch" HINT: Valid values in this context are integers between 1 and 1000.

CREATE FOREIGN TABLE XXXXXX()server "server_name" OPTIONS ( "prefetch" '10000',

_When we checked "prefetch" value form pgtype it was null

select oid,typname from pg_type where typname='prefetch'; oid | typname -----+--------- (0 rows)

_Can you please advise since prefetch value was null our upgrade was failed (or) orafce version 3.7 incompatible to the PG15 which is on B-database creating this problem?

Can we try as per the HINT to update the binary upgrade set values as below?_

-- For binary upgrade, must preserve pg_type oid SELECT pg_catalog.binary_upgrade_set_next_pg_type_oid('14535618'::pg_catalog.oid);

-- For binary upgrade, must preserve pg_type array oid SELECT pg_catalog.binary_upgrade_set_next_array_pg_type_oid('14535617'::pg_catalog.oid);

-- For binary upgrade, must preserve pg_class oids and relfilenodes SELECT pg_catalog.binary_upgrade_set_next_heap_pg_class_oid('14535616'::pg_catalog.oid); SELECT pg_catalog.binary_upgrade_set_next_heap_relfilenode('14535616'::pg_catalog.oid);

laurenz commented 5 days ago

Your problem report is very confused. You complain about orafce (which is unrelated to oracle_fdw), and you don't even tell me how you tried to upgrade.

Please describe what exactly you did, which version of oracle_fdw you have before and after the upgrade (use oracle_diag() to find the version; the extension version is not significant), and please show the complete, unmodified error message.

Yes, the limit for the prefetch option was reduced to 1000, but I undid that change again in commit 13ac9fb4e5ca5a515062b2c441d53079b0ec814d because of popular demand. So you could use the latest development version of oracle_fdw, and you should avoid that error.

Still, my recommendation is to lower prefetch to a more reasonable value before you upgrade. 10000 is crazy high.

Post-User2024 commented 5 days ago

Hi Laurenz,

Sorry for the confusion let me summarize the step by step what we did.

last week we tried upgrade as below.

1) Pg version 11 - Oracle FDW was 1.1 and orafce 3.7 2) Pg version 15 was installed after that orafce & oracle fdw latest version was installed using the make and make install.

then we started dry run it was success.

/usr/pgsql-15/bin/pg_upgrade --old-bindir /usr/pgsql-11/bin --new-bindir /usr/pgsql-15/bin --old-datadir /pgDATA/data --new-datadir /pgDATA/15/data --link --check Performing Consistency Checks

Checking cluster versions ok Checking database user is the install user ok Checking database connection settings ok Checking for prepared transactions ok Checking for system-defined composite types in user tables ok Checking for reg* data types in user tables ok Checking for contrib/isn with bigint-passing mismatch ok Checking for removed "abstime" data type in user tables ok Checking for removed "reltime" data type in user tables ok Checking for removed "tinterval" data type in user tables ok Checking for user-defined encoding conversions ok Checking for user-defined postfix operators ok Checking for incompatible polymorphic functions ok Checking for tables WITH OIDS ok Checking for invalid "sql_identifier" user columns ok Checking for presence of required libraries ok Checking database user is the install user ok Checking for prepared transactions ok Checking for new cluster tablespace directories ok

Clusters are compatible

Then started actual upgrade it failed with below error.

/usr/pgsql-15/bin/pg_upgrade --old-bindir /usr/pgsql-11/bin --new-bindir /usr/pgsql-15/bin --old-datadir /pgDATA/data --new-datadir /pgDATA/15/data --link

pg_restore: creating FUNCTION "public.nanvl(numeric, character varying)" pg_restore: creating FUNCTION "public.normal_rand(integer, double precision, double precision)" pg_restore: creating FUNCTION "public.nvarchar2_transform("internal")" pg_restore: while PROCESSING TOC: pg_restore: from TOC entry 10787; 1255 541860000 FUNCTION nvarchar2_transform("internal") abc pg_restore: error: could not execute query: ERROR: there is no built-in function named "varchar_transform" Command was: CREATE FUNC.

This week again we tested the upgrade as below

To fix above nvarchar2_transform issue we updated oracle fdw (From 1.1. to 1.2) and orafce (3.7 To 4.9) on PG version 11 then started upgrade again it failed and reporting the below error

Actually after install the Postgresql version 15 we have not done oracle fdw and orafce extension make && make install after upgrade will plan for the extension make && make install then update the extension on specific database.

pg_restore: while PROCESSING TOC: pg_restore: from TOC entry 1674; 1259 14535616 FOREIGN TABLE product_full abc pg_restore: error: could not execute query: ERROR: invalid value for option "prefetch" HINT: Valid values in this context are integers between 1 and 1000. Command was: -- For binary upgrade, must preserve pg_type oid SELECT pg_catalog.binary_upgrade_set_next_pg_type_oid('14535618'::pg_catalog.oid);

-- For binary upgrade, must preserve pg_type array oid SELECT pg_catalog.binary_upgrade_set_next_array_pg_type_oid('14535617'::pg_catalog.oid);

-- For binary upgrade, must preserve pg_class oids and relfilenodes SELECT pg_catalog.binary_upgrade_set_next_heap_pg_class_oid('14535616'::pg_catalog.oid); SELECT pg_catalog.binary_upgrade_set_next_heap_relfilenode('14535616'::pg_catalog.oid);

Post-User2024 commented 5 days ago

We used latest development version of oracle_fdw,

Post-User2024 commented 5 days ago

@laurenz

Could you please share inputs on this issue?

Post-User2024 commented 5 days ago

We used latest development version of oracle_fdw - 2.7.0

Post-User2024 commented 5 days ago

select oracle_diag(); oracle_diag

oracle_fdw 2.7.0, PostgreSQL 11.21, Oracle client 18.3.0.0.0,

laurenz commented 5 days ago

I understand the problem now, thanks.

The nvarchar_transform error must be from orafce. It plays no role here.

The error about prefetch above 1000 actually is an oracle_fdw problem. I didn't think of the dump/restore case.

The solution is to use the latest development version. That is not 2.7.0, but this.

Post-User2024 commented 5 days ago

Thanks for the quick reply.

alter extension orafce update; --- failed to update on PostgreSQL V11 ERROR: type "oracle.date" does not exist

can we ignore above ERROR and proceed for the upgrade-15 please advise.

laurenz commented 5 days ago

I don't know, and I won't investigate an orafce error. I have nothing to do with orafce.

Post-User2024 commented 4 days ago

Could you please advise who can help us on the orafce update issue?

laurenz commented 4 days ago

Sure, that project is here. You can open an issue there.

Post-User2024 commented 4 days ago

Thank you so much. Opened an issue.

laurenz commented 4 days ago

Please let me know when we can close this issue.

Post-User2024 commented 4 days ago

Could you please give me some time till this weekend?

laurenz commented 4 days ago

Sure, no problem at all.