Packet for query is too large (5,584,499 > 4,194,304) in glue 4.0

awslabs / aws-glue-libs

AWS Glue Libraries are additions and enhancements to Spark for ETL operations.

Other

636 stars 300 forks source link

Packet for query is too large (5,584,499 > 4,194,304) in glue 4.0 #98

Closed andcau closed 1 year ago

andcau commented 2 years ago

Hi I'm getting the following error in glue 3.0 and 4.0 while writing on s3: "An error occurred while calling o441.count. Packet for query is too large (5,584,499 > 4,194,304). You can change this value on the server by setting the 'max_allowed_packet' variable."

Everything works in 2.0 version

### Tasks