Problem Statement:
The HDFS_FDW was inferring the authentication type
from the options of the CREATE USER MAPPING command
If it found that the username is empty, it concluded NOSASL
otherwise it concluded LDAP.
In case of NOSASL since the username was empty while
connecting to the hive server the FDW was specifying an
arbitrary username. While this technique worked for the rest
of the cases, it failed in case of ANALYZE.
ANALYZE needed an OS user that had write access to the
folder /tmp/hadoop-yarn
Solution:
Add another option in CREATE SERVER command, called the
auth_type. This option, which can be omitted, can have the
following values: NOSASL & LDAP
When specified a non empty username must be specified while
creating the user mapping.
When omitted the FDW tries to infer the auth_type from the options
of the user mapping, like it used to do.
Hence the FDW is backwards compatible and existing test cases
do not need any modification.
If auth_type is specified and a valid username is specified
that has write access to the folder /tmp/hadoop-yarn, then
ANALYZE works fine.
For Example:
postgres=# CREATE EXTENSION hdfs_fdw;
CREATE EXTENSION
postgres=# CREATE SERVER hdfs_svr FOREIGN DATA WRAPPER hdfs_fdw OPTIONS (host '127.0.0.1',port '10000',client_type 'hiveserver2', auth_type 'NOSASL');
CREATE SERVER
postgres=# CREATE USER MAPPING FOR abbasbutt server hdfs_svr OPTIONS (username 'abbasbutt', password '');
CREATE USER MAPPING
postgres=# CREATE FOREIGN TABLE fnt( a int, name varchar(255)) SERVER hdfs_svr OPTIONS (dbname 'testdb', table_name 'names_tab');
CREATE FOREIGN TABLE
postgres=# SELECT * FROM fnt;
a
Problem Statement: The HDFS_FDW was inferring the authentication type from the options of the CREATE USER MAPPING command If it found that the username is empty, it concluded NOSASL otherwise it concluded LDAP. In case of NOSASL since the username was empty while connecting to the hive server the FDW was specifying an arbitrary username. While this technique worked for the rest of the cases, it failed in case of ANALYZE. ANALYZE needed an OS user that had write access to the folder /tmp/hadoop-yarn
Solution: Add another option in CREATE SERVER command, called the auth_type. This option, which can be omitted, can have the following values: NOSASL & LDAP When specified a non empty username must be specified while creating the user mapping. When omitted the FDW tries to infer the auth_type from the options of the user mapping, like it used to do. Hence the FDW is backwards compatible and existing test cases do not need any modification.
If auth_type is specified and a valid username is specified that has write access to the folder /tmp/hadoop-yarn, then ANALYZE works fine.
For Example:
(12 rows)
postgres=# ANALYZE fnt; ANALYZE