Feature; is it possible to have collection level above document?

[mohshraim]

Iam with SQL background and new to couchDB...

The feature is if its possible to have extra level between the DB and Document.

i want to transfer my current inventory system, FOREX trading also to couchDB. When i have many DBs each one contains 3-4 mill up to 10 mill records, original come from 40 entity. its not practical to add around 30 - 40 views to generate my reports. that each view have to read the on average 5 mill record(document) to generate related index. and its not reliable to open separate 40 DB for each client.. So main reason of this feature request to make my DB organized and to be able to generate the view on collection level and saving machine resources.

**Current scenario -- of 5 mill record and 40 index -->

5 mill * 40 = 200 mill read operation

**Collection Scenario --of 5 mill record DEVIDED EQUALLY ON 40 COLLECTION and 80 index as 2 per collection -

-> 5 mill / 40 collection * 80 = 8 mill read operation with doubled index for more control..

FOREX application DB have more than 100 mill records.. one table with 10 mill records, and end user request this data (sub records) every time he use the application. so building indexes on this DB will take forever..

if someone ask why to have many indexes... the answer that most of application entity is transactional type and system doing automated operations according to these transaction plus advance dashboard..

I didnt find any issue or feature request talking about collection. so just open this and hope someone help me with any suggestion

Thanks

[wohali]

This is what sharding is in CouchDB 2.x. With a large enough cluster and appropriate settings for q, you can separate view building across multiple CPUs and processes.

http://docs.couchdb.org/en/latest/cluster/index.html

Also, look at Mango.

http://docs.couchdb.org/en/latest/api/database/find.html

[mohshraim]

@wohali Thanks for your reply and links.. I have already read the whole documentation a hundred times last month, everything is under control expect the sharding, that i already have a huge dedicated server, and another small one for real time backup only...

But please let me share what i though even its wrong, you are the CouchDB masters... So Even that my previous request was good or bad but i didnt recognize and estimate the changes and efforts for something like that so let me start another try targeting more performance of CouchDB.

---

I will make it simple for your estimation.. GENERAL: can we build a View on exist View data (Sub View )??

Little DETAILS:

• add a new option on design document called (parent_view or view_source or .... what ever)
• CouchDB check this option and if exist will build new View MapReduce according to selected parent view Documents, so will go through Small Indexed Documents.
• On add,edit,delete of Documents, CouchDB will start with Views that doesn't contain the new option (parent_view).

Is Building like this feature is applicable and does it required dramatic changes? Thanks

[wohali]

This has been requested in the past and is not possible to implement the way you are requesting.

A better option for you is to use filtered (continuous) replication to a second database, then create a new view in that second database. Using this you can create a view in the 2nd database.

Be sure to look at Mango for improved performance over traditional JavaScript views.