AWS DocumentDB support: Yes? No? Soon?

[max-rocket-internet]

Heya 👋 Does the Chaos Control Plane support AWS DocumentDB? I can't find any mention of it in the docs or in the issues here.

[smitthakkar96]

When we tried with DocumentDB all queries here with $lookup started failing with the below error 😢 causing API endpoints for the control plane to throw 5xx

aggregation stage not supported: '$lookup on multiple join conditions and uncorrelated subquery

From DocumentDB docs: Amazon DocumentDB supports the ability to do equality matches (for example, left outer join) but does not support uncorrelated subqueries.

As per MongoDB docs there is only 34% between DocumentDB and MongoDB 😓.

It would be nice to document this somewhere, we spent a week trying to experiment with DocumentDB when we first adopted Litmus.

[namkyu1999]

Hey @SarthakJain26 @Saranya-jena, Currently, litmus's graphql-server is not compatible with documentdb. If we modify the graphql-server, we can make them compatible with documentdb. However, we cannot assure that documentdb will remain fully compatible with mongodb in the future. I would like to know your opinion - Do you think it is necessary for litmus to be compatible with documentdb?

fyi: supported mongodb apis in documentdb https://docs.aws.amazon.com/documentdb/latest/developerguide/mongo-apis.html

[max-rocket-internet]

Hey @SarthakJain26 @Saranya-jena,

These users are not involved in this issue? never mind, I see they are a contributor

Do you think it is necessary for litmus to be compatible with documentdb?

Being compatible with AWS DocumentDB would be one solution that would enable running Litmus natively in the (AWS) cloud. The problem we face is that we want to run Litmus on EKS in AWS and the only remotely compatible data storage service is DocumentDB, which does not work. So we are faced with 2 less-than-ideal options:

1. Run MongoDB ourselves which brings up all the normal topics of running something yourself:
   • How do we scale it?
   • How do we back it up?
   • How do we monitor it?
2. We use a third party to provide MongoDB as a service, e.g. MongoDB Atlas:
   • It's expensive
   • We have create an account with yet another provider
   • We have to deal with the connectivity challenges of connecting to this third party (VPC peering etc)

An even better solution to this would be to just support PostgreSQL but I'm not sure how much effort would be required for this.

[SarthakJain26]

We need to check on compatibility of DB queries in litmus portal with DocumentDB. @smitthakkar96, I would like to know more findings from your try to use DocumentDB. Were only the queries involving lookup stages failing or there were other issues as well?

[SarthakJain26]

@smitthakkar96 would you please share some more insights from your research on DocumentDB with Litmus. We can then try to plan items accordingly

[smitthakkar96]

Lookups were only thing that was failing unfortunately due to so many errors I couldn't test each and every flow 😢. Sorry I don't have better insights for you.

[SarthakJain26]

Thanks @smitthakkar96 🙌 , @max-rocket-internet please do share your findings in the process, we can plan things accordingly

[jongwooo]

Hi @namkyu1999 I want to take this issue. Can you please assign it to me?

[namkyu1999]

go ahead, @jongwooo