itsvikramagr
commented
4 years ago can you check if you have permission to describe Stream? Also, can you look at driver logs to find out the cause of the error and maybe fix that.
Closed manishkaushik29 closed 4 years ago
itsvikramagr
commented
4 years ago can you check if you have permission to describe Stream? Also, can you look at driver logs to find out the cause of the error and maybe fix that.
Hi ,
I am creating a glue job with following details : Source Kinesis
Target Aurora Mysql
Glue Version1 Python3, Spark2.4
Used following jars as dependencies : spark-sql-kinesis_2.11-1.1.3-spark_2.4.jar, spark-tags_2.11-2.4.0.jar, unused-1.0.0.jar, scala-library-2.11.12.jar, spark-sql_2.11-2.4.0.jar
I am getting following error while running the glue job which throws this error while connecting to Kinesis stream: WARNING:root:StreamingQueryException caught. Retry number 10 ERROR:root:Exceeded maximuim number of retries in streaming interval, exception thrown Parse yarn logs get error message: StreamingQueryException: 'Error while Describe Streams\n=== Streaming Query ===\nIdentifier: [id = 60ed9d34-b7bb-4969-a033-5e7ebda85824, runId = 23bb23bb-1e28-4f40-aa87-976a73b26747]\nCurrent Committed Offsets: {} \nCurrent Available Offsets: {} \n\nCurrent State: ACTIVE\nThread State: RUNNABLE\n\nLogical Plan:\nProject [cast(data#5 as string) AS $json$da