Glue with MSK

[ali-raza-rizvi]

Hi,

We are trying to use Glue schema registry with MSK. MSK and Glue are deployed in same AWS account , however the services which are consuming it are in other AWS account. We managed to access MSK through other account by creating assume role but still facing an issue to access Schema registry from cross account since aws-glue-schema-registry library accepts only registryName , Is there any way to pass registryARN instead of registryname so we could access it from other AWS accounts ?

Our use-case is similar to this : https://aws.amazon.com/blogs/big-data/validate-streaming-data-over-amazon-msk-using-schemas-in-cross-account-aws-glue-schema-registry/

But we do not want to implement it this way , is there any better work around ?

[allkliu]

You should be able to configure your serializer and deserializer to use the credentials of your assumed role. The serializer/deserializer takes a credential object. I've attached a code reference.

https://github.com/awslabs/aws-glue-schema-registry/blob/master/serializer-deserializer/src/main/java/com/amazonaws/services/schemaregistry/serializers/GlueSchemaRegistryKafkaSerializer.java#L49

[shaarif-khan]

hi @allkliu, we also have the exact similar use-case, and we are stuck on that, for accessing cross-account msk we used msk-iam-auth library and created this config props.put("sasl.mechanism", "AWS_MSK_IAM"); props.put("security.protocol", "SASL_SSL"); props.put("sasl.jaas.config", "software.amazon.msk.auth.iam.IAMLoginModule required awsRoleArn=\"arn:aws:iam::{accountId}:role/{assume-role-name}\";"); and passed these configuration to KafkaProducer class. that did the job for us, but for accessing glue registry, don't we have any such configuration? are we suppose to generate temporary credentials using sts assumeRole api? could you guide us with some code snippet on how to achieve it?

[blacktooth]

You can avoid calling AssumeRole API by leveraging StsAssumeRoleCredentialsProvider. As mentioned above, GlueSchemaRegistryKafkaSerializer accepts a credential provider instance during initialization. You can pass in an instance of StsAssumeRoleCredentialsProvider to it.

If you want this to be a configurable option like in msk-iam-auth, we need to take this up as a feature request.