search

trinodb / trino

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
https://trino.io
Apache License 2.0
10.51k stars 3.03k forks source link

Trino MongoDB connector with Atlas on version 460 only works with Dedicated Server Cluster versions #23739

Open hgoycoolea opened 1 month ago

hgoycoolea commented 1 month ago

After carefully working with the slack community we had an issue with the connector for MongoDB, the connector works as expected on :

Mongo regular deployments. (all*) Mongo Atlas ----> works only on Dedicated servers

It won't work on: Mongo Atlas Serverless Mongo Atlas Shared

Trino will be able write from trino to mongo Serverless and Shared but Introspection of Database and collections won't work. Meaning you can still fetch data and write data but without introspection (mapping the tables or auto discovery with _schema). The issue is on the LoadBalancer as we checked, hope this helps and the issue is on the slack channel if you need more details.

trino> SHOW TABLES FROM atlas.main_db --debug -> ; Query 20241008_102838_00034_ppn3j failed: line 1:1: Schema 'main_db' does not exist

trino> SHOW TABLES FROM atlas.main_db; Query 20241008_103011_00035_ppn3j failed: line 1:1: Schema 'main_db' does not exist io.trino.spi.TrinoException: line 1:1: Schema 'main_db' does not exist at io.trino.sql.analyzer.SemanticExceptions.semanticException(SemanticExceptions.java:58) at io.trino.sql.analyzer.SemanticExceptions.semanticException(SemanticExceptions.java:52) at io.trino.sql.rewrite.ShowQueriesRewrite$Visitor.visitShowTables(ShowQueriesRewrite.java:255) at io.trino.sql.rewrite.ShowQueriesRewrite$Visitor.visitShowTables(ShowQueriesRewrite.java:219) at io.trino.sql.tree.ShowTables.accept(ShowTables.java:62) at io.trino.sql.tree.AstVisitor.process(AstVisitor.java:27) at io.trino.sql.rewrite.ShowQueriesRewrite.rewrite(ShowQueriesRewrite.java:216) at io.trino.sql.rewrite.StatementRewrite.rewrite(StatementRewrite.java:54) at io.trino.sql.analyzer.Analyzer.analyze(Analyzer.java:93) at io.trino.sql.analyzer.Analyzer.analyze(Analyzer.java:87) at io.trino.execution.SqlQueryExecution.analyze(SqlQueryExecution.java:289) at io.trino.execution.SqlQueryExecution.(SqlQueryExecution.java:222) at io.trino.execution.SqlQueryExecution$SqlQueryExecutionFactory.createQueryExecution(SqlQueryExecution.java:892) at io.trino.dispatcher.LocalDispatchQueryFactory.lambda$createDispatchQuery$0(LocalDispatchQueryFactory.java:153) at io.trino.$gen.Trino_460____20241008_092332_2.call(Unknown Source) at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131) at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:76) at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) at java.base/java.lang.Thread.run(Thread.java:1575)

but as mentioned, it does fetch and writes data.

image

Peace! thanks to the community and special thanks to Ashar Hasan, you rock dude !!

hashhar commented 1 week ago

cc: @mosabua Can you please help this get documented. People might otherwise needlessly waste time when using the Mongo connector.

The TL;DR is:

Trino Mongo connector has limitations when used with the following, specifically the schema introspection does NOT work:

@ebyhr / @mayankvadariya do you by any chance know if the statement is accurate?

mayankvadariya commented 1 week ago

I don't remember testing connecting to Atlas Serverless & shared.

mosabua commented 5 days ago

Well.. we don't document all the things that are not tested .. looking at https://trino.io/docs/current/connector/mongodb.html#requirements - Atlas it not tested or supported at all.. in any version.

We could add a note about Atlas but it would have to say that nothing is supported since we don't test with Atlas.. ideally we only add this if we have actual tests with it.

Also that would go against all other connectors where we also dont document what doesn't work.. In addition if we document it not working for Atlas .. what about CosmosDB and the various other "compatible" systems.