Redshift driver throws NPE when IAM auth is use with profile and credential_source

jxerome commented 1 year ago

Driver version

2.1.0.12 with AWS SDK 1.12.408

Redshift version

Client Operating System

Linux

JAVA/JVM version

Java 11

Table schema

Not relevant

Problem description

Configuration

Program is on a EC2 instance with a role allowed to assume another role with access permission. A profile is defined in .aws/credentials

[redshift_by_iam]
region = eu-west-3
credential_source = Ec2InstanceMetadata
role_arn = arn:aws:iam::123456789012:role/role_with_redshift_access

The connection URL contains a reference to the profile:

jdbc:redshift:iam://cluster.abcd123efgh456.eu-west-3.redshift.amazonaws.com:5439/dev?profile=redshift_by_iam".

Expected behavior:

I would expect the connection to be established.
At least, because redshift driver SDK usage is out of date, I would expect a clean exception message the explains that "source_profile" is missing.

Actual behavior:

The driver throws an NullPointerException.

com.amazon.redshift.Driver.connect: java.lang.NullPointerException
    at java.base/java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936)
    at com.amazon.redshift.core.PluginProfilesConfigFile.getCredentials(PluginProfilesConfigFile.java:50)
    at com.amazon.redshift.core.PluginProfilesConfigFile.getCredentials(PluginProfilesConfigFile.java:89)
    at com.amazon.redshift.core.PluginProfilesConfigFile.getCredentials(PluginProfilesConfigFile.java:31)
    at com.amazonaws.auth.profile.ProfileCredentialsProvider.getCredentials(ProfileCredentialsProvider.java:161)
    at com.amazon.redshift.core.IamHelper.setIAMCredentials(IamHelper.java:418)
    at com.amazon.redshift.core.IamHelper.setIAMProperties(IamHelper.java:280)
    at com.amazon.redshift.jdbc.RedshiftConnectionImpl.<init>(RedshiftConnectionImpl.java:263)
    at com.amazon.redshift.Driver.makeConnection(Driver.java:474)
    at com.amazon.redshift.Driver.connect(Driver.java:295)
    at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:677)
    at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:189)
    at Main.main(Main.java:9)

JDBC trace logs

Mar 27 15:40:05.767 DEBUG  [1 main] com.amazon.redshift.Driver.connect: ===================================
Mar 27 15:40:05.767 FUNCTION  [1 main] com.amazon.redshift.Driver.connect:  Enter (jdbc:redshift:iam://cluster.abcd123efgh456.eu-west-3.redshift.amazonaws.com:5439/dev?profile=redshift_by_iam&DSILogLevel=5&LogPath=/home/ec2-user/application/logs,{user=redshift_user}) 
Mar 27 15:40:05.768 DEBUG  [1 main] com.amazon.redshift.Driver.connect: Connecting with URL: jdbc:redshift:iam://cluster.abcd123efgh456.eu-west-3.redshift.amazonaws.com:5439/dev?profile=redshift_by_iam&DSILogLevel=5&LogPath=/home/ec2-user/application/logs
Mar 27 15:40:05.769 DEBUG  [1 main] com.amazon.redshift.Driver.connect: Caller stack[main]: application.Main.main(Main.java)
Mar 27 15:40:05.771 DEBUG  [1 main] com.amazon.redshift.jdbc.RedshiftConnectionImpl.RedshiftConnectionImpl: Redshift JDBC Driver 2.1.0.12
Mar 27 15:40:05.774 DEBUG  [1 main] com.amazon.redshift.jdbc.RedshiftConnectionImpl.RedshiftConnectionImpl: Start IAM authentication
Mar 27 15:40:05.796 DEBUG  [1 main] com.amazon.redshift.core.IamHelper.setIAMCredentials: IDP Credential Provider com.amazonaws.auth.profile.ProfileCredentialsProvider@1e226bcd:null
Mar 27 15:40:05.796 DEBUG  [1 main] com.amazon.redshift.core.IamHelper.setIAMCredentials: Calling provider.getCredentials()
Mar 27 15:40:05.797 INFO  [1 main] com.amazon.redshift.core.PluginProfilesConfigFile.getCredentials: profiles:[redshift_by_iam]
Mar 27 15:40:05.798 ERROR  [1 main] com.amazon.redshift.Driver.connect: java.lang.NullPointerException
    at java.base/java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936)
    at com.amazon.redshift.core.PluginProfilesConfigFile.getCredentials(PluginProfilesConfigFile.java:50)
    at com.amazon.redshift.core.PluginProfilesConfigFile.getCredentials(PluginProfilesConfigFile.java:89)
    at com.amazon.redshift.core.PluginProfilesConfigFile.getCredentials(PluginProfilesConfigFile.java:31)
    at com.amazonaws.auth.profile.ProfileCredentialsProvider.getCredentials(ProfileCredentialsProvider.java:161)
    at com.amazon.redshift.core.IamHelper.setIAMCredentials(IamHelper.java:418)
    at com.amazon.redshift.core.IamHelper.setIAMProperties(IamHelper.java:280)
    at com.amazon.redshift.jdbc.RedshiftConnectionImpl.<init>(RedshiftConnectionImpl.java:263)
    at com.amazon.redshift.Driver.makeConnection(Driver.java:474)
    at com.amazon.redshift.Driver.connect(Driver.java:295)
    at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:677)
    at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:189)
    at Main.main(Main.java:9)

Mar 27 15:40:05.850 FUNCTION  [3 Finalizer] com.amazon.redshift.jdbc.RedshiftConnectionImpl.close:  Enter () 
Mar 27 15:40:05.851 FUNCTION  [3 Finalizer] com.amazon.redshift.jdbc.RedshiftConnectionImpl.close:  Return

First analysis

In PluginProfilesConfigFile.java on line 88, the source profile is read from the current profile and passed to the method getCredentials(String) on line 89 without null check.

        if (profile.isRoleBasedProfile())
        {
            String srcProfile = profile.getRoleSourceProfile();
            CredentialsHolder srcCred = getCredentials(srcProfile);
            AWSCredentialsProvider provider = new AWSStaticCredentialsProvider(srcCred);

Reproduction code

import java.sql.DriverManager;
import java.util.Properties;

public class Main {
    public static void main(String[] args) {
        var info = new Properties();
        info.setProperty("user", "redshift_user");
        var url = "jdbc:redshift:iam://cluster.abcd123efgh456.eu-west-3.redshift.amazonaws.com:5439/dev?profile=redshift_by_iam&DSILogLevel=5&LogPath=/home/ec2-user/logs";
        try (var conn = DriverManager.getConnection(url, info);
            var stmt = conn.createStatement();
            var rs = stmt.executeQuery("select version()")) {
                while (rs.next()) {
                    System.out.println("version: " + rs.getString(1));
                }
        } catch (Exception e) {
            System.err.println("ERROR: " + e.getMessage());
            e.printStackTrace();
        }
    }
}

bhvkshah commented 1 year ago

Thanks for submitting this issue @jxerome and the corresponding PR! I'll get back to you with an update after taking a look.

bhvkshah commented 1 year ago

We'll have the fix for this in our next release, version 2.1.0.14. Thanks for contributing to the Redshift JDBC Driver!

aws / amazon-redshift-jdbc-driver