Allow an array of documents to be posted?

[libby2cm]

Not being much of a dev myself, would it be difficult to allow restheart to accept an array of json docs during a post? When I try now, it just fails and gives an error that you can't use an array.

I was looking to use restheart to receive some webhooks from a 3rd party cloud service, and depending how many events are queued up, several can be sent across at one time.

[ujibang]

do you mean allowing a single POST to create multiple documents?

[mkjsix]

It would also be useful to have a look at what kind of post is giving you errors, so that we can reproduce the same behaviour: perhaps it's just a matter of composing the json doc in a slightly different way.

[libby2cm]

That was quick :)

Yes, I mean allowing multiple documents to be submitted, as an array, in a single POST.

If I post a single document, it works, as expected & I get a 201 back, for example:

{
    "email": "john.doe@some.com",
    "event": "processed",
    "category":["category1","category2","category3"],
    "id": "001",
    "purchase":"PO1452297845",
    "uid": "123456"
}

If I post 2 CSV documents, only the first one gets inserted, but it doesn't give an error:

{
    "email": "john.doe@some.com",
    "event": "processed",
    "category":["category1","category2","category3"],
    "id": "001",
    "purchase":"PO1452297845",
    "uid": "123456"
},
{
    "email": "john.doe@some.com",
    "event": "processed",
    "category":["category1","category2","category3"],
    "id": "001",
    "purchase":"PO1452297845",
    "uid": "123456"
}

If I try to insert more than one record in a POST, as an array, it fails:

[ { "email": "john.doe@some.com", "event": "processed", "category":["category1","category2","category3"], "id": "001", "purchase":"PO1452297845", "uid": "123456" }, { "email": "john.doe@some.com", "event": "processed", "category":["category1","category2","category3"], "id": "001", "purchase":"PO1452297845", "uid": "123456" }]

With this error as result:

HTTP/1.1 406 Not Acceptable Connection: keep-alive Access-Control-Allow-Origin: * Access-Control-Allow-Credentials: true Access-Control-Expose-Headers: Location Content-Type: application/hal+json Content-Length: 258 Date: Thu, 08 Jan 2015 23:35:34 GMT

{ "http status code" : 406 , "http status description" : "Not Acceptable" , "message" : "data cannot be an array" , "_embedded" : { "rh:exception" : [ { "_links" : { "self" : { "href" : "#"}}}]} , "_links" : { "self" : { "href" : "/alerts/events"}}}

The array should work.. it's a standard format for a JSON array, other tools I have used work with it, the restheart code just needs to detect the array and loop thru it and insert each document.

Unfortunately the JSON coming from the vendor is static, so I kinda have to work around it, but it is well formed & valid.

This is easily replicated using just fiddler, so hopefully you can see what I am talking about.

Thanks for your help!

[libby2cm]

Hi, I see this was marked as a bug. If that's the case, I was just curious when some test code may be available for me to try out? No rush of course, but was looking to use this for a project, so if it may be awhile I might have to come up with some else temporarily.

Thanks!

[ujibang]

batch operations (creation, deletion and update of several resources at once) is on our roadmap and currently planned for version 0.9.9 (we are currently at 0.9.7, so you get it in 2 versions).

No commitment on release date, but 0.9.9 should be out in few weeks.

However we are still thinking about this feature and we foresee some issues:

• what should be returned in case one document gets created, one updated and one doc fails (e.g. because of a not matching etag)
• how to pass back the URI of the created resources (Location header only allows one URI to be specified)
• the HTTP 1.1 specifications always refers to POST as a verb that creates/updates ONE resources.

In other words I cannot guarantee now that you'll have exactly what you are asking for.

As a temporary solution you can easily extend RESTHeart to cope with this use case by adding some custom application logic to get the array and iterate its elements to create several documents.

[mkjsix]

Update: we have discussed the implementation details and we have a good solution in mind, however it will take some time to implement it properly.

As explained, the quickest workaround would be to repackage the request so that the JSON payload will start with a document name ("items" in the below example, but the name can be anything), so that it would be inserted as a single document containing an array of documents.

{"items": [
            {
                "email": "john.doe@some.com",
                "event": "processed",
                "category":["category1","category2","category3"],
                "id": "001",
                "purchase":"PO1452297845",
                "uid": "123456"
            },
            {
                "email": "john.doe@some.com",
                "event": "processed",
                "category":["category1","category2","category3"],
                "id": "001",
                "purchase":"PO1452297845",
                "uid": "123456"
            }]
}

Alternatively you could implement a temporary solution in terms of a custom handler, as Andrea wrote before, so that you can split the original JSON in Java and create single documents from the array, being in control of error recovering. This is, more or less, what we plan to add to the solution, but it will require more work to be general, RESTful and capable to handle all possible runtime errors properly. This is the general case of handling a batch import of documents and must be fully transactional in terms of the whole bulk operation.

[libby2cm]

Thanks, but I just needed something quick for a small project I was working on (receiving webhooks from SendGrid).. I ended up going with another solution I found that runs on python/bottle, and is working perfectly for me for what I needed.

Thanks anyway!