Open ShahNewazKhan opened 5 years ago
Hi @ShahNewazKhan,
My best guess is to look in the Google Cloud audit log to see if there are failing events. It looks like the service client is being created with whatever the default credentials are in your environment (the KeyManagementServiceClient.create
portion) rather than using the same credential logic that the plugin has https://github.com/logstash-plugins/logstash-input-google_pubsub/blob/master/lib/logstash/inputs/google_pubsub.rb#L224-L227.
You might want to consider creating a logstash-filter plugin rather than modifying this one, that gives a few benefits:
Let me know if that helps!
Hi @josephlewis42,
Thanks for the input! I will try and resolve this as an extension for this plugin with the java jar
for a quick poc win. If not I will pursue the the logstash-filter
plugin method you have suggested.
Regarding the auth method, I tried passing in the fixedCredentialsProvider
object used by the Pubsub client
and it still hangs at the same point. Unfortunately the audit logs does not log any of the kms client create
actions, only the subsequent get, list, decrypt, encrypt
actions.
public byte[] decryptWrappedKey(String wrappedKey, String cid, FixedCredentialsProvider creds)
throws IOException {
if (this.testMode) { logger.log(Level.INFO, "Decrypting wrapped key");}
byte[] wrappedKeyBytes = Base64.decodeBase64(wrappedKey);
// Create the KeyManagementServiceClient using try-with-resources to manage client cleanup.
if (this.testMode) { logger.log(Level.INFO, "Creating KMS Client");}
try {
if (this.testMode) { logger.log(Level.INFO, "Creating KMS settings");}
KeyManagementServiceSettings keyManagementServiceSettings =
KeyManagementServiceSettings.newBuilder()
.setCredentialsProvider(creds)
.build();
if (this.testMode) { logger.log(Level.INFO, "Creating KMS settings");}
KeyManagementServiceClient client =
KeyManagementServiceClient.create(keyManagementServiceSettings);
logger.log(Level.INFO, "ENTERED KEY CREATION");
String keyResourceName = CryptoKeyName.format(
this.projectId,
this.locationId,
this.keyRingId,
cid);
// Decrypt the ciphertext with Cloud KMS.
if (this.testMode) { logger.log(Level.INFO, "Decrypting KMS response"); }
DecryptResponse response = client.decrypt(keyResourceName, ByteString.copyFrom(wrappedKeyBytes));
if (this.testMode) { logger.log(Level.INFO, "Returning decrypted wraped key"); }
client.close();
logger.log(Level.INFO, "**************CLOSING KMS CLIENT ***********");
return response.getPlaintext().toByteArray();
}catch (Exception e) {
throw new RuntimeException(e);
}
}
I will try to take the decrypt
logic out of the .java
file and re-write it as a jruby
function and see if I get any more verbose log messages.
@josephlewis42 an update:
EDIT
I was able to find the relevant by instantiating the KeyManagementServiceClient
as an instance variable. It seems the 'com.google.cloud', 'google-cloud-kms', '0.81.0-beta'
jar I included using jar-dependencies
is missing the com/google/cloud/kms/v1/ListKeyRingsRequest
class.
java.lang.NoClassDefFoundError: com/google/cloud/kms/v1/ListKeyRingsRequest
at com.google.cloud.kms.v1.stub.GrpcKeyManagementServiceStub.<clinit>(com/google/cloud/kms/v1/stub/GrpcKeyManagementServiceStub.java:88)
at com.google.cloud.kms.v1.stub.KeyManagementServiceStubSettings.createStub(com/google/cloud/kms/v1/stub/KeyManagementServiceStubSetting
s.java:292)
at com.google.cloud.kms.v1.KeyManagementServiceClient.<init>(com/google/cloud/kms/v1/KeyManagementServiceClient.java:154)
at com.google.cloud.kms.v1.KeyManagementServiceClient.create(com/google/cloud/kms/v1/KeyManagementServiceClient.java:135)
at com.google.cloud.kms.v1.KeyManagementServiceClient.create(com/google/cloud/kms/v1/KeyManagementServiceClient.java:126)
I have checked $CLASSPATH
and found com/google/cloud/kms
jar is listed, however that jar does not contain a com/google/cloud/kms/v1/ListKeyRingsRequest
class file.
Hello,
I am extending this plugin to include a GCP KMS decryption procedure inside the
pubsub subscribe handler
by creating adecrypt.java
jar file that stalls when I call thedecrpytWrappedKey
method as described below:The unit tests for this method runs fine, however when I jar it up and call it from within the
subscribe handler
in this plugin, it hangs at the client create portion.I can see the
ENTERED KEY CREATION
log and then it just hangs, any ideas how I can go about debugging this?