SoftInstigate / restheart

Rapid API Development with MongoDB
https://restheart.org
GNU Affero General Public License v3.0
807 stars 171 forks source link

in Sharding Server PATCH method how to use the shard key exclude _id #98

Closed caarto closed 8 years ago

caarto commented 8 years ago

hi there: in sharding server enviroment ,we use the "hphm" field for the shard key how we add the shard key in patch method when we update document.

ciao

mkjsix commented 8 years ago

Actually I don't have much experience with MongoDB' sharding, so I have more questions than answers:

Is that a custom shard key you created? Do you read it back when getting a collection or single document? If yes, then you should be able to patch it as you'd do with any other JSON property: see PATCH document on documentation API reference

Please also always specify RESTHeart and MongoDB versions in use, it usually helps framing the context.

caarto commented 8 years ago

enviroment:mongodb:3.2.1,restheart:1.1.4 yes,we create a custom shard key,and we can get result by GET method. in single mongodb server,PATCH method work. in sharding server,PATCH method got the 500 status. below is the restheart response,I took the important picture here

{
    "_links": {
        "self": {
            "href": "/ivms/CLTX/20160127110302020691"
        }
    },
    "http status code": 500,
    "http status description": "Internal Server Error",
    "message": "Error handling the request",
    "_embedded": {
        "rh:exception": [
            {
                "exception": "com.mongodb.MongoCommandException",
                "exception message": "Command failed with error 61: 'query for sharded findAndModify must have shardkey' on server 127.0.0.1:27017. The full response is { \"ok\" : 0.0, \"errmsg\" : \"query for sharded findAndModify must have shardkey\", \"code\" : 61 }",
                "stack trace": [
                    "com.mongodb.MongoCommandException: Command failed with error 61: 'query for sharded findAndModify must have shardkey' on server 127.0.0.1:27017. The full response is { \"ok\" : 0.0, \"errmsg\" : \"query for sharded findAndModify must have shardkey\", \"code\" : 61 }\r",
                    "  at com.mongodb.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:86)\r",
                    "  at com.mongodb.connection.CommandProtocol.execute(CommandProtocol.java:119)\r",
                    "  at com.mongodb.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:159)\r",
                    "  at com.mongodb.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:286)\r",
                    "  at com.mongodb.connection.DefaultServerConnection.command(DefaultServerConnection.java:173)\r",
                    "  at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:215)\r",
                    "  at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:152)\r",
                    "  at com.mongodb.operation.FindAndUpdateOperation$1.call(FindAndUpdateOperation.java:305)\r",
                    "  at com.mongodb.operation.OperationHelper.withConnectionSource(OperationHelper.java:230)\r",
                    "  at com.mongodb.operation.OperationHelper.withConnection(OperationHelper.java:221)\r",
                    "  at com.mongodb.operation.FindAndUpdateOperation.execute(FindAndUpdateOperation.java:302)\r",
                    "  at com.mongodb.Mongo.execute(Mongo.java:782)\r",
                    "  at com.mongodb.Mongo$2.execute(Mongo.java:765)\r",
                    "  at com.mongodb.DBCollection.findAndModifyImpl(DBCollection.java:1818)\r",
                    "  at com.mongodb.DBCollection.findAndModify(DBCollection.java:1681)\r",
                    "  at com.mongodb.DBCollection.findAndModify(DBCollection.java:1628)\r",
                    "  at org.restheart.db.DocumentDAO.upsertDocument(DocumentDAO.java:72)\r",
                    "  at org.restheart.handlers.document.PatchDocumentHandler.handleRequest(PatchDocumentHandler.java:89)\r",
                    "  at org.restheart.handlers.metadata.AbstractTransformerHandler.handleRequest(AbstractTransformerHandler.java:64)\r",
                    "  at org.restheart.handlers.metadata.CheckMetadataHandler.handleRequest(CheckMetadataHandler.java:69)\r",

for this exception we make a mongdb api test case, here is the code:

DBObject query = SearchBuilder.start("_id").eq("20160127110302020641").get();
DBObject update = new BasicDBObject();
update.put("_id", "20160127110302020641");
update.put("jllx", "3");
update.put("hphm", "粤U01812");
update.put("etag", new ObjectId());
log.info("query object is - " + query.toString());
log.info("update object is - " + update.toString());
collection.findAndModify(query, update);

this test case got the same exception as the rh. for fix the bug ,we modify this

DBObject query = SearchBuilder.start("_id").eq("20160127110302020641").get();

to

DBObject query = SearchBuilder.start("_id").eq("20160127110302020641").and("hphm").eq("粤U01812").get();

we add the shard key in query condition ,it works.

So,the rh PATCH method may have some parameters for the sharding with a custom shard key.

ps. we found that PUT verb got the same problem.does it use the same method?

mkjsix commented 8 years ago

Thanks @caarto for your detailed description. We've just opened a JIRA issue for tracking progress on this. To answer you last question: yes, I think the PUT verb has exactly the same problem.

Please contribute to https://softinstigate.atlassian.net/browse/RH-163 in order to help testing any future progress.

caarto commented 8 years ago

Thanks for your reply and thanks for your formatting,it is terrible before for reading.

ujibang commented 8 years ago

hi @caarto

I just pushed an improvement to master branch to support shard keys.

I would appreciate if you can test it in your use case. You have to build restheart 1.2-SNAPSHOT and use restheart.jar found in target directory:

$ git clone https://github.com/SoftInstigate/restheart.git
$ mvn package

basically now you can add the shardkey query parameter to any request: you can pass all shared keys in it as json.

example, to create a document with shard key type="C"

http POST 127.0.0.1:8080/sdb/coll/?shardkey='{"type":"C"}' a:=1 _id=doc

to update it:

http PATCH 127.0.0.1:8080/sdb/coll/doc?shardkey='{"type":"C"}' a:=2
mkjsix commented 8 years ago

Snapshots are available here: https://oss.sonatype.org/content/repositories/snapshots/org/restheart/restheart/ Get the most recent build.

caarto commented 8 years ago

we change our RH to the snapshot ,and test for a while, it works. thanks for your work.