CouchDB Write Only, Disable Update

[roshanr10]

I'd like for a design doc. to make our CouchDB data read/write only and disable update for data-integrity reasons. The following documentation features a method called on every update.

CouchDB Update Function

The design doc. need only reject all updates.

[shashjar]

function(doc, req) {
  return[null, 'Cannot Overwrite Data']
}

[roshanr10]

Tried using this, and found that this also blocks writes.. need to fix this.

[shashjar]

function(newDoc, oldDoc, userCtx, secObj) {
      if(newDoc._deleted === true) {
            if ((userCtx.roles.indexOf('_admin') !== -1) ||
            (userCtx.name == oldDoc.name)) {
            return;
      } else {
            function(doc, req) {
                  return[null, 'Cannot Overwrite Data']
                  }
            }

[roshanr10]

Your parenthesis don't match up, also shouldn't be a function within itself.

[shashjar]

function(newDoc, oldDoc, userCtx, secObj) {
      if(newDoc._deleted === true) {
            if ((userCtx.roles.indexOf('_admin') !== -1) ||
            (userCtx.name == oldDoc.name)) {
            return;
      } else {
            return[null, 'Cannot Overwrite Data']
         }
     }
}

Haven't tested this yet...

[roshanr10]

Right now it's checking for admin privs, that' shouldn't matter for our use case...

Due to changes in the way we store data, I'd now like an if statement that runs if there is an existing document. Inside of this condition, we will then take the stats, average the two documents and re-enter to the database. This will allow for redundant data collection. I believe that's doable since this func is called for write and update.

However can we try to implement roles so only the proper teams can insert the data? Say 4909 is at marea, can we use that as a role, and block the team from inserting data for negsd? If that fails, throw an error.

[jaspercoughlin]

How would we average what's in the JSON document if it's all just text? Where would the numerical data be?

[roshanr10]

Hypothetical DB Entry:

{ "event_match_key": "MAREA_SF2M2", "points": 2 }

Averages: Assume a majority of metrics are just numbers like the points metric here. If there is an existing document, then average metrics between the two (you can hardcode just points for now), and update the DB.

Auth: Split event_match_key by the _ and compare that to the user's role for authentication.

[shashjar]

function(newDoc, oldDoc, userCtx, secObj) {
    if ((userCtx.roles.indexOf('_marea') !== -1) ||
        (userCtx.name == oldDoc.name)) {
            return;
            emit(null, 'points')
    } else {
                return[null, 'Cannot Overwrite Data']
       }
}
function(keys, values, rereduce) {
    if (!rereduce) {
        var length = values.length
        return [sum(values) / length, length]
    } else {
        var length = sum(values.map(function(v){return v[1]}))
        var avg = sum(values.map(function(v) {
            return v[0] * (v[1] / length)
            }))
        return [avg, length]
    }
}        

[roshanr10]

@ShashJar could you document the above code a bit more? I'm having trouble following how things interconnect. links to documentation about the method signatures would be nice as well.

[shashjar]

function(newDoc, oldDoc, userCtx, secObj) {
    if ((userCtx.roles.indexOf('_marea') !== -1) ||  //tests for marea role of user to ensure correct data input
        (userCtx.name == oldDoc.name)) {
            return;
            emit(null, 'points')  //emits key value pair to show which value is being averaged, points
    } else {
        return[null, 'Cannot Overwrite Data']  //if user does not have marea role, data cannot be input
       }
}
function(keys, values, rereduce) {  //function to take average
    if (!rereduce) {
        var length = values.length  //length variable is how many values are averaged
        return [sum(values) / length, length]  //returns sum divided by length, or average of points values
    } else {
        var length = sum(values.map(function(v){return v[1]}))
        var avg = sum(values.map(function(v) {
            return v[0] * (v[1] / length)
            }))
        return [avg, length]  //returns two values: the average and how many values were averaged
    }
}        

Taking Average of JSON Field Values: http://tobyho.com/2009/10/07/taking-an-average-in-couchdb/

Validating Doc Update Function: http://docs.couchdb.org/en/2.1.1/ddocs/ddocs.html#validate-document-update-functions

[roshanr10]

The latter method is just a reduce function, for map/reduce analysis, not for update. The core logic should be built in to the first update method. Good stuff though, we'll use that logic later for some other stats.

Few Things I See:

• return gets called before the emit if the role exists causing emit to never be called
• the role should be checked against the doc they tried to insert
• where does the new doc actually get inserted? I can't find it

[shashjar]

function(newDoc, oldDoc, userCtx, secObj) {
    if (newDoc.role !== ("marea")) {   //tests for marea role of doc to ensure correct data input
            emit(null, 'points')  //emits key value pair to show which value is being averaged, points
            return;
    } else {
        return[null, 'Cannot Overwrite Data']  //if doc does not have marea role, data cannot be input
       }
}

function(keys, values, rereduce) {  //function to take average
    if (!rereduce) {
        var length = values.length  //length variable is how many values are averaged
        return [sum(values) / length, length]  //returns sum divided by length, or average of points values
    } else {
        var length = sum(values.map(function(v){return v[1]}))
        var avg = sum(values.map(function(v) {
            return v[0] * (v[1] / length)
            }))
        return [avg, length]  //returns two values: the average and how many values were averaged
    }
}        

function(doc, req)  {  //function to insert averaged data back into database
    if(!doc)  {  //if there is a doc in the database available
        if('id' in req && req['id'])  {
            return[{'_id': req['id']}, 'New Doc']  //create a new document
        }
        return[null, 'Empty Database.']  //if no doc is available for averaging, return this statement
    }
    doc['New Doc'] = avg;  //enter the averaged value into the new doc
    doc['edited_by'] = req['userCtx']['name']  //return who edited/inputted the data
    return[doc, 'Edited Data.']  //return a confirmation that data has successfully been input
}

[shashjar]

function(newDoc, oldDoc, userCtx, secObj) {

    if ((userCtx.roles.indexOf('2018marea') !== -1) || (userCtx.name == oldDoc.name)) {   //tests for marea role of doc to ensure correct data input

        if (!doc) {   //if there is a doc in the database available

            if ('id' in req && req['id']) {

                return [{'_id': req['id']}, 'New Doc']   //create a new document
                emit(null, 'points')
                var pointsArray = ['points'], thisTotal = 0, thisAverage = 0;
                for(var i = 0;i < pointsArray.length; i++)  {    //for function to find the total number of "points" fields being averaged

                  thisTotal+ = pointsArray[i];

                }

                thisAverage = (thisTotal/pointsArray.length);    //calculates the average

            }

            return [null, 'Empty Database.']    //if no doc is available for averaging, return this statement

        }

        doc['New Doc'] = thisAverage;    //enter the averaged value into the new doc
        doc['edited_by'] = req['userCtx']['name']    //return who edited/inputted the data
        return [doc, 'Edited Data.']    //return a confirmation that data has successfully been input

    } else {

        return [null, 'Cannot Overwrite Data']  //if doc does not have marea role, data cannot be input

    }
}

[shashjar]

  function(newDoc, oldDoc, userCtx, secObj) {

    if ((userCtx.roles.indexOf("2018marea") !== -1) || (userCtx.name == oldDoc.name)) { 

        if (!doc) {

            if ("id" in req && req["id"]) {

                return [{"_id": req["id"]}, "New Doc"]  
                emit(null, "points")
                var pointsArray = ["points"], thisTotal = 0, thisAverage = 0;
                for(var i = 0;i < pointsArray.length; i++)  {

                  thisTotal+ = pointsArray[i];

                }

                thisAverage = (thisTotal/pointsArray.length); 

            }

            return [null, "Empty Database."]

        }

        doc["New Doc"] = thisAverage;
        doc["edited_by"] = req["userCtx"]["name"] 
        return [doc, "Edited Data."]  

    } else {

        return [null, "Cannot Overwrite Data"]

    }
  } 

Same code but with double quotes (for CouchDB syntax)

[shashjar]

{
  "_id": "_design/marea",
  "language": "javascript",
  "validate_doc_update": "function(newDoc, oldDoc, userCtx, secObj) {\r\n\r\n    if ((userCtx.roles.indexOf(\"2018marea\") !== -1) || (userCtx.name == oldDoc.name)) { \r\n    \r\n        if (!doc) {\r\n        \r\n            if (\"id\" in req && req[\"id\"]) {\r\n            \r\n                return [{\"_id\": req[\"id\"]}, \"New Doc\"]  \r\n                emit(null, \"points\")\r\n                var pointsArray = [\"points\"], thisTotal = 0, thisAverage = 0;\r\n                for(var i = 0;i < pointsArray.length; i++)  {\r\n                \r\n                  thisTotal+ = pointsArray[i];\r\n                \r\n                }\r\n                \r\n                thisAverage = (thisTotal/pointsArray.length); \r\n                \r\n            }\r\n            \r\n            return [null, \"Empty Database.\"]\r\n            \r\n        }\r\n        \r\n        doc[\"New Doc\"] = thisAverage;\r\n        doc[\"edited_by\"] = req[\"userCtx\"][\"name\"] \r\n        return [doc, \"Edited Data.\"]  \r\n\r\n    } else {\r\n    \r\n        return [null, \"Cannot Overwrite Data\"]\r\n   \r\n    }\r\n  } "
}

CouchDB Design Document Format, able to save, so there are no syntax errors, but still need to test this code.

Useful links: https://stackoverflow.com/questions/36517173/how-to-store-a-javascript-function-in-json, https://www.freeformatter.com/javascript-escape.html#ad-output, http://guide.couchdb.org/draft/design.html, https://pouchdb.com/guides/documents.html

[shashjar]

{
    "_id": "_design/marea",
    "language": "javascript",
    "validate_doc_update":  "function(newDoc, oldDoc, userCtx, secObj) {\r\n\r\n    if ((userCtx.roles.indexOf(\"2018marea\") !== -1) || (userCtx.name == oldDoc.name)) { \r\n       return;\r\n    }\r\n}"
}

This code works....if the user trying to create a doc has the role "2018marea", they are able to save the doc successfully. If not, the database will block them from saving.

[shashjar]

function(newDoc, oldDoc, userCtx, secObj) {
    if ((userCtx.roles.indexOf(newDoc.eventkey) !== -1) || (userCtx.name == oldDoc.name)) {
        if (!doc) {
            if ("id" in req && req["id"]) {
                return [{"_id": req["id"]}, "New Doc"]; 
                emit(null, "points");
                var pointsArray = ["points"], thisTotal = 0, thisAverage = 0;
                for(var i = 0;i < pointsArray.length; i++)  {
                  thisTotal += pointsArray[i];
                }
                thisAverage = (thisTotal/pointsArray.length); 
            }
        }
        doc["New Doc"] = thisAverage;
        doc["edited_by"] = req["userCtx"]["name"] 
        return [doc, "Edited Data."]  
    } 
}

Code same as 3 comments ago but without else statements to get rid of redundant code JSON version of this code:

{
    "_id": "_design/marea",
    "language": "javascript",
    "validate_doc_update": "function(newDoc, oldDoc, userCtx, secObj) {\r\n    if ((userCtx.roles.indexOf(\"2018marea\") !== -1) || (userCtx.name == oldDoc.name)) {\r\n        if (!doc) {\r\n            if (\"id\" in req && req[\"id\"]) {\r\n                return [{\"_id\": req[\"id\"]}, \"New Doc\"]; \r\n                emit(null, \"points\");\r\n                var pointsArray = [\"points\"], thisTotal = 0, thisAverage = 0;\r\n                for(var i = 0;i < pointsArray.length; i++)  {\r\n                  thisTotal += pointsArray[i];\r\n                }\r\n                thisAverage = (thisTotal\/pointsArray.length); \r\n            }\r\n        }\r\n        doc[\"New Doc\"] = thisAverage;\r\n        doc[\"edited_by\"] = req[\"userCtx\"][\"name\"] \r\n        return [doc, \"Edited Data.\"]  \r\n    } \r\n}"
}

[roshanr10]

@ShashJar does the updating segment work as well?

[shashjar]

When I include the update code, I get the following error message when I try to create a doc as the user, even when the user has the correct role.

Save failed: {[{<<"stack">>, <<"([object Object],null,[object Object],[object Object])@validate_doc_update:2\n(function (newDoc, oldDoc, userCtx, secObj) {if (userCtx.roles.indexOf(\"2018marea\") !== -1 || userCtx.name == oldDoc.name) {if (!doc) {if (\"id\" in req && req.id) {return [{_id: req.id}, \"New Doc\"];emit(null, \"points\");var pointsArray = [\"points\"], thisTotal = 0, thisAverage = 0;for (var i = 0; i < pointsArray.length; i++) {thisTotal += pointsArray[i];}thisAverage = thisTotal / pointsArray.length;}}doc['New Doc'] = thisAverage;doc.edited_by = req.userCtx.name;return [doc, \"Edited Data.\"];}},[object Object],[object Array])@./share/server/main.js:1291\n(\"_design/marea\",[object Array],[object Array])@./share/server/main.js:1537\n()@./share/server/main.js:1582\n()@./share/server/main.js:1603\n@./share/server/main.js:1\n">>}, {<<"message">>,<<"oldDoc is null">>}, {<<"fileName">>,<<"validate_doc_update">>}, {<<"lineNumber">>,2}]}

[shashjar]

The following code is to check the value of the event-key field in the document a user is trying to create. If the user a role equivalent to that value, then they are able to create it. If not, then the document is not able to be created. The code is tested, and the following error message comes up: Save failed: Expression does not eval to a function. ( function(newDoc, oldDoc, userCtx, secObj) { var newDoc.eventKey = ["event-key"]; if ((userCtx.roles.indexOf(newDoc.eventKey) !== -1) || (userCtx.name == oldDoc.name)) { return; } })

{
  "_id": "_design/marea",
  "_rev": "7-cca521656e1daea89bb6e8923af0521c",
  "language": "javascript",
  "validate_doc_update": "   function(newDoc, oldDoc, userCtx, secObj) {\r\n     var newDoc.eventKey = [\"event-key\"];\r\n     if ((userCtx.roles.indexOf(newDoc.eventKey) !== -1) || (userCtx.name == oldDoc.name)) { \r\n       return;\r\n    }\r\n  }"
}

Not sure how to fix this, the problem may be in declaring the variable for the event key. It might not be registering to call on the value from the new document.

[roshanr10]

Why are you redefining newDoc.eventKey? Does CouchDB not pass that value to the function already in some manner?

Could you please elaborate upon what the issue is? Does it not allow you to insert any data or does it not like the design document itself?

[shashjar]

{
  "_id": "_design/marea",
  "language": "javascript",
  "validate_doc_update": "   function(newDoc, oldDoc, userCtx, secObj) {\r\n  if ((userCtx.roles.indexOf(newDoc.eventkey) !== -1) || (userCtx.name == oldDoc.name)) { \r\n       return;\r\n    }\r\n  }"
}

This code works for checking the role of the user. Note: the JSON field in the document must be spelled "eventkey", without a dash. If the user has a role that matches the value of the "eventkey" field, then they are able to create the document. If not, an error is brought up.

[shashjar]

{
  "_id": "_design/marea",
  "language": "javascript",
  "validate_doc_update": " function(newDoc, oldDoc, userCtx, secObj) {\r\n  if ((userCtx.roles.indexOf(newDoc.eventkey) !== -1) || (userCtx.name == oldDoc.name)) { \r\n       return;\r\n       var pointsArray = [\"points\"];\r\n       thisTotal = 0;\r\n       thisAverage = 0;\r\n       for(var i = 0; i < pointsArray.length; i++) {\r\n     thisTotal += pointsArray[i];\r\n}\r\n      thisAverage = (thisTotal/pointsArray.length);\r\n      newDoc.points = thisAverage;\r\n    }\r\n  }\r\n"
}

This code is code is able to save as the design doc...the same error message of the expression not evaluating to a function doesn't come up. The averaging system still doesn't work as needed, still working on it.

[jasonbishop11]

JSON Doc for Creating a New User dbreader - substitute in new user's name, in lines 1 and 2. Add any roles needed for the user to create a new doc. Roles should be added in quotes. If there are multiple for a single user, separate the quoted roles with commas. Put the new user's password in quotes for line 5.

{
    "_id": "org.couchdb.user:dbreader",
    "name": "dbreader",
    "type": "user",
    "roles": [],
    "password": "plaintext_password"
}