Support bulk updates

[remmeier]

A variety of implementations (most notably Ember) allows to ship a data section within relationship fields. This allows to not only update the root entity, but also related entities. This way all updates run in the same transaction, which is a hard requirement for many applications. For GET this is already specified with the include parameter. This ticket would extend that support to PATCH and POST.

• Maybe at some point that will also be specified.
• Clients that do not want to make use of the feature are not affected by teh change.
• an operation api would be the alternative, but seems much more complex, in particular if non-UUIDs are in use that cannot be generated by clients.

[Ramblurr]

Super excited to see this brought up. Our katharsis api is in desperate need of bulk updates.

There is a non-jsonapi-standard technique for supporting bulk updates without client side id generation. It is outlined in the article Saving Models and Their Relationships with JSON API.

In summary, it requires that the client side an attribute __id__ inside the attributes object, and that the server simply mirror that field back to the client if it is present.

Example:

{
  id: 1,
  type: "lines",
  attributes: {
    description: "Chocolate",
    price: 2.10,
    __id__: "1internal-model"
  }
}

Now, since this is non-standard, perhaps Katharsis shouldn't support it out of the box. But I'd definitely like to add this as an optional feature. Either has a core feature that is configurable, or via a Katharsis Module.

What do you all think?

@remmeier once you start the code for this, I will jump in and assist! Make sure and link the branch here.

[corrspt]

Just want to add, that this also interests me (I'm using Ember with Katharsis) and I also had to hack a way to deal with bulk updates.

[chb0github]

I had no idea this was so desired. Internally, Java isn't going to like thatid thing. However, we could do something with the value.

There is another issue: you presume that Katharsis as a framework knows what a good id generating function is.

Consider: Where will this be persisted? In a db? Is the table this Id a key for capable of accepting a uuid? Like, the column is defined as varchar(36)? Katharsis would need to have deep understanding of the underlying persistence store.

Next question: what would the response code be if for any reason some creates failed? Gather up a list of

{
    "type" : "foo",
    "id" : null,
    "status" : 500
}

This is totally non restful and makes the servers job way more complicated. But I know the jsonapi spec sorta allows this.

Here is how I would do bulk :

Support create on PUT and then client side make a massive parallelized set of calls.

On Tue, Nov 29, 2016, 2:04 AM Pedro Rio notifications@github.com wrote:

Just want to add, that this also interests me (I'm using Ember with Katharsis) and I also had to invent a way to deal with bulk updates.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/katharsis-project/katharsis-framework/issues/222#issuecomment-263528296, or mute the thread https://github.com/notifications/unsubscribe-auth/ABaI0JdzNV-fea1_mkfLo2wtMfe2uY9pks5rC_jLgaJpZM4K1kem .

[Ramblurr]

Ah, I see my Tl;DR version of this technique was perhaps too brief.

Where will this be persisted? In a db? Is the table this Id a key for capable of accepting a uuid?

It is not persisted anywhere on the server side. It doesn't involve a persistence store at all.

If a POST/PATCH/PUT comes in, and a jsonapi object has the __id__ field, then the server only must mirror it back for the same object when returning the result of the operation. That's it. It doesn't need to store it for longer than it takes to process the request.

Katharsis would need to have deep understanding of the underlying persistence store.

No, that isn't the case. Sorry my TL;DR version of this technique was not clear enough. Here's more info

1. Consider the scenario where a client is creating a set of objects that have relationships among each other.
2. For each new object the client generates some temporary id, this becomes the value of __id__. Internally on the client, these temporary ids are used to link the object relationships until such time as a canonical server side id is known.
3. At some point the user clicks "save", and all these new objects are POSTed in bulk to the server. The attributes section includes the __id__ key with the temporary id value. Notably the ACTUAL json id field is null.
4. Server side, Katharsis does everything as normal, except when it comes to serializing the response it mirrors the __id__.

The entities in a result from a successful POST will have:

1. a non-null json api id -> the canonical server side id
2. a __id__ attribute that is the temporary client id that was POSTed

What's the result? It allows the client to update its local store by assigning the entity with __id__ to the canonical server side `id.

This allows for the creation of complex relationships without real client side id generation, and importantly it ensures the request can happen in an atomic transaction.

[Ramblurr]

Additionally:

• AFAIK, This __id__ stuff doesn't need to be handled by Katharsis directly, we could implement that outside it
• Katharsis need only support bulk updates by supporting a full data block in a relationship section, by dispatching save() to the correct repositories.

Egh: sorry for the open/close spam, my finger slipped.

[remmeier]

• for a first version we could consider ignoring all the ID generation stuff and just support UUIDs and manual ids set by client (i.e. no db auto-increments). This makes life much simpler since the client can create those ids.
• it would be good to make some noise on the json api issue page. There are this two approaches, the one of Ember and the serivce operations api (similar to http://jsonpatch.com/). But no decision for a long time.
• but since Ember already has a "standard" on its own and json api is coming from this direction, I guess it is quite worthwhile to support it regardless of the lack of decision by the json api spec.
• my time is a bit limited at the moment but also very interested in supporting it (for many of our apps atomicity comes before 100%-restfullness, that's the way it is, no possiblity to change...). So if somebody wants to go ahead here, I can give the necessary pointers and help.

[Ramblurr]

@remmeier Yes I agree with your points. Perhaps you have time to lay out a set of [ ] that could be used as a roadmap to implement this? I'm happy to do the grunt work.

[remmeier]

no longer needed

[NetForce1]

@remmeier Could you give some extra information about why this is no longer needed? I was looking for something like this today, and came upon this issue. Is it implemented somehow already in Katharsis 3.0?