search

couchbaselabs / Linq2Couchbase

A Language Integrated Query (LINQ) provider for the Couchbase .NET SDK
Apache License 2.0
94 stars 48 forks source link

Query fails due to absence of index #346

Closed mail2hjain closed 3 years ago

mail2hjain commented 3 years ago

I must admit being a newbie for Couchbase. While trying to use where predicates with query APIs, an exception is thrown indicating no Index found for the KeySpace... In this regard, I have below questions -

  1. Is it required to have a index created for each attribute that is required for filtering ? Or just having a Primary Index sufficient?
  2. How does it work to know (and thereby fixing) the model schema in a NoSql world? Aren't document / model supposed to be generic and not known upfront? What are the best practices here?
brantburnett commented 3 years ago

@mail2hjain

First of all, I'm going to assume that you mean Couchbase, not CouchDB. They are two completely different products. That said, let me see if I can answer your questions.

  1. In production environments, yes you typically want an index create to cover each query. However, good index design can cover multiple queries with a single index. Traditional SQL can support querying small tables via a table scan, which isn't a great practice but does make things easier in RDBMS. Couchbase does have a similar concept, the PRIMARY INDEX, however it is strongly recommended not to use a primary index in production. It creates a situation where you're scanning the entire database for matches, which will usually cause timeouts at production scale.
  2. Some people refer to NoSQL document databases like Couchbase as "schemaless". However, I believe that this is a misnomer. In fact, they usually must have a well-known schema. The difference is where the schema is enforced. With RDBMS, it is enforced at that database. With Couchbase, you enforce the schema in your application via the POCOs you define for serialization. This is much more flexible and makes versioning over time much easier. The downside is you must be more strict with your development practices. You must be careful to limit the points in your code which touch a specific document type so that all data access flows through the same code. If you don't do this, you'll tend to introduce bugs as your application evolves over time.
mail2hjain commented 3 years ago

Thanks @brantburnett for the explanation. Yes I meant Couchbase itself. I will update the description. I get your point about having a well known schema. This works good with RDBMS as the schema is known at database level which is not the case with NoSQL where schema is known only by Application. This would mean that any change in the schema or introducing a new business entity would force a change in the persistence layer as well. This is where I am looking to find some best practices to handle such scenarios. TIA.

brantburnett commented 3 years ago

I'd be happy to discuss it further, but it's a bit tricky because your application architecture is a big factor in how you approach it. For example, a monolith vs microservices may have different approaches.

The best place for this discussion is probably on the couchbase forums at https://forums.couchbase.com/. So I'm going to close this issue, and if you want to start a discussion there in the .NET forum I'll watch out for it.