search

apache / beam

Apache Beam is a unified programming model for Batch and Streaming data processing.
https://beam.apache.org/
Apache License 2.0
7.81k stars 4.23k forks source link

Mongo DB Upgrade to 5.0 having compatibitlity issue with Apache Beam included mongo-java-driver. #21646

Open damccorm opened 2 years ago

damccorm commented 2 years ago

I am from the Jaguar Landrover (JLR UK) team and we are using Apach beam Dataflow Pipeline to ingest JLR vehicle data into Mongo DB from more than a Year. On production we have upgraded our Mongo DB from 4.4 to 5.0. And We have Apach Beam Pipeline running with version 2.34 on GCP.  And after upgrading we are experinceing connection issue.

And as per Mongo DB Documentation says this Apache Beam dependecy mongo-java-driver is not compatible with Mongo DB 5.0.

https://www.mongodb.com/docs/drivers/java/sync/current/compatibility/#compatibility

!image-2022-05-05-17-16-23-236.png!

Atelast we should have Mongo Java Driver 4.4 or higher with Mongo DB 5.0 

And inside the Apach Beam version i can see Mongo Java Driver version is till 3.12.x so Please provide your valudable help in this issue. Our Application is on JLR Production and it's causing critical issues.

 

Please have a look at this on urgent basis as it's blocking on production.

 

Imported from Jira BEAM-14420. Original Jira may contain additional context. Reported by: dneema.

Abacn commented 2 years ago

Beam 2.38 has bumped driver version to 3.12.10 which should support MongoDB 5.0 server: https://github.com/mongodb/mongo-java-driver/blob/d5e75f4a72976c70a2371cad82c5769abbe6e124/driver-core/src/main/com/mongodb/connection/ServerDescription.java#L64