failed to create extension

[zhaoyuan2022]

will get error "ERROR: attempt to redefine parameter "hdfs_fdw.classpath" (guc.c:8879)" when run `create extension hdfs_fdw', I have set the required guc "hdfs_fdw.classpath" and "hdfs_fdw.jvmpath". I can see the code from hdfs_fdw.c the fourth parameter 'flag' is set to 0. " DefineCustomStringVariable("hdfs_fdw.classpath", "Specify the path to HiveJdbcClient-X.X.jar, hadoop-common-X.X.X.jar and hive-jdbc-X.X.X-standalone.jar", NULL, &g_classpath, "", PGC_SUSET, 0, NULL, NULL, NULL); ", and the code in guc.c will check the custom flag: " if (((res)->flags & GUC_CUSTOM_PLACEHOLDER) == 0) ereport(ERROR, (errcode(ERRCODE_INTERNAL_ERROR), errmsg("attempt to redefine parameter \"%s\"", name))); " the value of GUC_CUSTOM_PLACEHOLDER is 0x0080, so the result of ' ((res)->flags & GUC_CUSTOM_PLACEHOLDER) == 0' will always be 0, this means the error will always be triggered. Do I need extra modification for my code?

[zhaoyuan2022]

I have postgres 12 running on ubuntu 20.04.1

[sravanvelag]

Hi, hdfs_fdw works with postgres. I tried to reproduce the error, the only way I could reproduce the error is in the following way:

prompt=#select from emp limit 2; ERROR: class not found HINT: Add path of HiveJdbcClient-X.X.jar to hdfs_fdw.classpath. prompt=#select from emp limit 2; ERROR: attempt to redefine parameter "hdfs_fdw.classpath"

However, if you login to a session again and start by setting the GUC variables hdfs_fdw.classpath and hdfs_fdw.jvmpath, or if you set the these variables in postgresql.conf and restart the server, the extension should work.

Is there any other way you are able to reproduce the error?

[zhaoyuan2022]

Hi, sravanvelag, I can reproduce this error when I was trying to create extension hdfs_fdw. Maybe I am I am using this FDW in a wrong way, but I compile and install the fdw follow the installation document. The steps I reproduce the error:

postgres@dev-50:~$ psql demo psql (12beta2, server 12.11) Type "help" for help.

demo=# SET hdfs_fdw.classpath='/usr/local/pg/lib/postgresql/HiveJdbcClient-1.0.jar:/home/postgres/workspace/hadoop_fdw/hive-jdbc-3.1.2-standalone.jar:/home/postgres/workspace/hadoop_fdw/hadoop-3.2.3/share/hadoop/common/hadoop-common-3.2.3.jar'; SET demo=# ALTER SYSTEM SET hdfs_fdw.classpath='/usr/local/pg/lib/postgresql/HiveJdbcClient-1.0.jar:/home/postgres/workspace/hadoop_fdw/hive-jdbc-3.1.2-standalone.jar:/home/postgres/workspace/hadoop_fdw/hadoop-3.2.3/share/hadoop/common/hadoop-common-3.2.3.jar'; ALTER SYSTEM demo=# SET hdfs_fdw.jvmpath='/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server'; SET demo=#ALTER SYSTEM SET hdfs_fdw.jvmpath='/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server'; ALTER SYSTEM demo=# create extension hdfs_fdw; ERROR: class not found HINT: Add path of HiveJdbcClient-X.X.jar to hdfs_fdw.classpath. demo=# SET hdfs_fdw.classpath='/usr/local/pg/lib/postgresql/HiveJdbcClient-1.0.jar:/home/postgres/workspace/hadoop_fdw/hive-jdbc-3.1.2-standalone.jar:/home/postgres/workspace/hadoop_fdw/hadoop-3.2.3/share/hadoop/common/hadoop-common-3.2.3.jar'; SET demo=# create extension hdfs_fdw; ERROR: attempt to redefine parameter "hdfs_fdw.classpath" I use the alter system command to settle down the guc, then the values are stored into postgresql.auto.conf

Then I append the guc into postgresql.conf and restart the server, I continue to try to create extention, I got the following error:

postgres@dev-50:~$ psql demo psql (12beta2, server 12.11) Type "help" for help.

demo=# create extension hdfs_fdw; 2022-07-13 10:40:05.090 HKT [583993] ERROR: class not found 2022-07-13 10:40:05.090 HKT [583993] HINT: Add path of HiveJdbcClient-X.X.jar to hdfs_fdw.classpath. 2022-07-13 10:40:05.090 HKT [583993] STATEMENT: create extension hdfs_fdw; ERROR: class not found HINT: Add path of HiveJdbcClient-X.X.jar to hdfs_fdw.classpath. demo=# ALTER SYSTEM SET hdfs_fdw.classpath='/usr/local/pg/lib/postgresql/HiveJdbcClient-1.0.jar:/home/postgres/workspace/hadoop_fdw/hive-jdbc-3.1.2-standalone.jar:/home/postgres/workspace/hadoop_fdw/hadoop-3.2.3/share/hadoop/common/hadoop-common-3.2.3.jar'; ALTER SYSTEM demo=# create extension hdfs_fdw; 2022-07-13 10:40:11.053 HKT [583993] ERROR: attempt to redefine parameter "hdfs_fdw.classpath" 2022-07-13 10:40:11.053 HKT [583993] STATEMENT: create extension hdfs_fdw; ERROR: attempt to redefine parameter "hdfs_fdw.classpath" demo=# SET hdfs_fdw.classpath='/usr/local/pg/lib/postgresql/HiveJdbcClient-1.0.jar:/home/postgres/workspace/hadoop_fdw/hive-jdbc-3.1.2-standalone.jar:/home/postgres/workspace/hadoop_fdw/hadoop-3.2.3/share/hadoop/common/hadoop-common-3.2.3.jar'; SET demo=# create extension hdfs_fdw; 2022-07-13 10:40:22.010 HKT [583993] ERROR: attempt to redefine parameter "hdfs_fdw.classpath" 2022-07-13 10:40:22.010 HKT [583993] STATEMENT: create extension hdfs_fdw; ERROR: attempt to redefine parameter "hdfs_fdw.classpath"

[sravanvelag]

Most probably HiveJdbcClient-1.0.jar is not copied to this location: /usr/local/pg/lib/postgresql/HiveJdbcClient-1.0.jar. Can you please verify?

[zhaoyuan2022]

Most probably HiveJdbcClient-1.0.jar is not copied to this location: /usr/local/pg/lib/postgresql/HiveJdbcClient-1.0.jar. Can you please verify?

It is already in that location.

I followed the steps in the install page, so the error confusing me a lot.

[sravanvelag]

Can you please extract the contents of the jar file and share the details here? jar xf HiveJdbcClient-1.0.jar ls -lt *.class

[zhaoyuan2022]

ls -lt *.class

postgres@dev-50:/usr/local/pg/lib/postgresql$ jar xf HiveJdbcClient-1.0.jar postgres@dev-50:/usr/local/pg/lib/postgresql$ ls -lt *.class -rw-rw-r-- 1 postgres postgres 11923 Jul 13 09:25 HiveJdbcClient.class -rw-rw-r-- 1 postgres postgres 793 Jul 13 09:25 MsgBuf.class -rw-rw-r-- 1 postgres postgres 2793 Jun 30 16:00 JDBCType.class

Is there any class lost?

[zhaoyuan2022]

Please ignore the issue, found the root cause is that the inconsistent of java version.

[sravanvelag]

Hi, can you please share the details of what the issue was and how it was resolved?