Adding a record when jsonapi server does not accept client generated ids

[julfla]

Hi,

I would like to use orbit with a rails backend using jsonapi-resources. In order to created a new record, the api expects a POST request which does not include an id.

As explained on JSON api documentation, here is what should happen.

If a POST request did not include a Client-Generated ID and the requested resource has been created successfully, the server MUST return a 201 Created status code. The response SHOULD include a Location header identifying the location of the newly created resource. The response MUST also include a document that contains the primary resource created.

However, when I call addRecord, orbitjs is automatically generating a uuid and includes it to the post data.

Is there a way to prevent orbit from generating uuid for new records ? I am getting confused with the remoteId, feature. Is it the way to go ?

Here is what I'm doing, its largely inspired by the Getting started in the documentation.

import Store from '@orbit/store';
import Orbit, { Schema } from '@orbit/data';
import Coordinator, { SyncStrategy, RequestStrategy } from '@orbit/coordinator';
import JSONAPISource from '@orbit/jsonapi';
import 'isomorphic-fetch';

Orbit.fetch = fetch;

const schema = new Schema({
  models: {
    customer: {
      attributes: {
        name: { type: 'string' },
      },
    },
  },
});

const store = new Store({ schema });

const remote = new JSONAPISource({
  schema,
  name: 'remote',
  host: 'http://localhost:3000/api/v1',
});

const coordinator = new Coordinator({
  sources: [store, remote],
});

// Query the remote server whenever the store is queried
coordinator.addStrategy(new RequestStrategy({
  source: 'store',
  on: 'beforeQuery',
  target: 'remote',
  action: 'pull',
  blocking: true,
}));

// Update the remote server whenever the store is updated
coordinator.addStrategy(new RequestStrategy({
  source: 'store',
  on: 'beforeUpdate',
  target: 'remote',
  action: 'push',
  blocking: true,
}));

// Sync all changes received from the remote server to the store
coordinator.addStrategy(new SyncStrategy({
  source: 'remote',
  target: 'store',
  blocking: true,
}));

test('store addRecord does not force remote id', async () => {
  const customer = {
    type: 'customer',
    attributes: {
      name: 'Bob',
    },
  };

  await coordinator.activate();

  // Add record make a post query to the api, with a id gererated by orbitjs.
  // My api expects no id to be present, the server will assign an id and
  // return the created object.
  const createdCustomer = await store.update(t => t.addRecord(customer));
  console.log(createdCustomer);
});

[dgeb]

Hi @julfla,

You'll want to use a key to represent the remote ID that's distinct from the locally generated ID. Conventionally, we use remoteId for this key. Keys are explained in the guides here: http://orbitjs.com/v0.15/guide/modeling-data.html#Keys

You'll need to declare this key in your schema for every model:

const schema = new Schema({
  models: {
    customer: {
      keys: { 
        remoteId: {} 
      },
      attributes: {
        name: { type: 'string' },
      },
    },
  },
});

And you'll need to tell your serializer to use it for every resource's id:

class CustomJSONAPISerializer extends JSONAPISerializer {
  resourceKey(type) {
    return 'remoteId';
  }
}

And you'll want to pass the custom serializer to your source:

const remote = new JSONAPISource({
  schema,
  name: 'remote',
  host: 'http://localhost:3000/api/v1',
  SerializerClass: CustomJSONAPISerializer
});

You'll also need a key map that you can share among sources so that they can map between keys and local ids, and you'll need to pass this same key map into every source:

import { KeyMap } from '@orbit/data';

let keyMap  = new KeyMap();

const remote = new JSONAPISource({
  schema,
  keyMap
  name: 'remote',
  host: 'http://localhost:3000/api/v1',
  SerializerClass: CustomJSONAPISerializer
});

const store = new Store({ schema, keyMap });

I obviously need to centralize these steps into some docs. Let me know if I've missed anything.

[julfla]

Hi @dgeb !

Thank you very much for your detailed reply. The example is now working :slightly_smiling_face:

One thing I found confusing is that it seems that the notion of Record Identity is unaware of the keys. As a result, and if I understand correctly, manipulation of a record knowing only its remoteId is bit tedious.

We need to manually lookup for the id in the keymap, and if missing generate an id and push it.

Here is a helper function I'm using:

function recordIdentityFromKeys({ type, id, keys }) {
  const recordIdentity = {
    type,
    keys,
    id: id || keyMap.idFromKeys(type, keys) || schema.generateId(type),
  };
  keyMap.pushRecord(recordIdentity);
  return recordIdentity;
}

[mspiderv]

Hi @julfla,

You'll want to use a key to represent the remote ID that's distinct from the locally generated ID. Conventionally, we use remoteId for this key. Keys are explained in the guides here: http://orbitjs.com/v0.15/guide/modeling-data.html#Keys

You'll need to declare this key in your schema for every model:

const schema = new Schema({
  models: {
    customer: {
      keys: { 
        remoteId: {} 
      },
      attributes: {
        name: { type: 'string' },
      },
    },
  },
});

And you'll need to tell your serializer to use it for every resource's id:

class CustomJSONAPISerializer extends JSONAPISerializer {
  resourceKey(type) {
    return 'remoteId';
  }
}

And you'll want to pass the custom serializer to your source:

const remote = new JSONAPISource({
  schema,
  name: 'remote',
  host: 'http://localhost:3000/api/v1',
  SerializerClass: CustomJSONAPISerializer
});

You'll also need a key map that you can share among sources so that they can map between keys and local ids, and you'll need to pass this same key map into every source:

import { KeyMap } from '@orbit/data';

let keyMap  = new KeyMap();

const remote = new JSONAPISource({
  schema,
  keyMap
  name: 'remote',
  host: 'http://localhost:3000/api/v1',
  SerializerClass: CustomJSONAPISerializer
});

const store = new Store({ schema, keyMap });

I obviously need to centralize these steps into some docs. Let me know if I've missed anything.

We would appreciate these information in docs :)

[maniodev]

Is there a way to simply not include the id parameter ? I suppose overriding serialize() and remove it could work but is there a simpler way ? i'm using a B/E which doesn't allow this parameter even if it's empty string.

[RichardsonWTR]

@dgeb and others, sorry to bother you guys... Do you have any reference for me to accomplish a simple POST request using 0.16.6? I tried to follow the steps above:

• added the keyMap in the MemorySource
• added the SerializerClass in the JSONAPISource
• added the keys: { remoteId: {} }, in the models.

But I couldn't get this to work, and now not even a simple get request is working. One error I'm receiving is this:

I'm feeling that I need to use julfla's method recordIdentityFromKeys, but I don't know where I would use it.

This piece of code to make a POST request doesn't work:

let res = await memory.update((t) => t.addRecord(newPerson));

As if that weren't enough, after calling the line above and receiving a failed result, following calls of this line are never resolves (the promise doesn't reject or succeeds, no errors thrown, the code apparently hangs). I need to reload the page to try again.

Related info

I found this issue because I was trying to get rid from the local ID from the post request body.
At that time when I accomplished a post request, I noticed that the all the property names where wrongly dasherized. The resource type was also pluralized, too (but I think this is subject for another post, and some of these behaviors/bugs will be changed in v0.17. (as stated by dgeb's comment on #714 - Handling of Dasherized Fields Not Documented )

Thanks for your work with this awesome library. Any help would be appreciated.

[dgeb]

@RichardsonWTR you'll want to share the same keyMap with all your sources. It's expecting keyMap to be present (idFromKeys is a method on the KeyMap class).

[RichardsonWTR]

Wow! Such a fast response!Thanks!
Sharing the same keyMap with all the sources fixed the GET request. Now it's working as expected.
Now I need to change the "dasherization" and the pluralization behavior to make a successful POST request.

Probably root of the pluralization problem

I've implemented my models.. OK. But when I run the app the dev console is full of messages like these:

Orbit's built-in naive singularization rules cannot singularize X. Pass singularize & pluralize functions to Schema to customize.

Where X are my models: e.g. pessoa, endereco, contato, etc. (translating from Portuguese: person, address, contact).

I've implemented the pluralize and the singularize methods to get rid of these warnings.

But I think that there's a problem with the pluralize and the singularize methods.
See the example below:

Implementation that I imagine to be the correct:

const inflectPluralize = {
  pessoa: "pessoas",
  contato: "contatos",
  endereco: "enderecos"
};

const inflectSingularize = {
  pessoas: "pessoa",
  contatos: "contato",
  enderecos: "endereco",
};
const schema = new Schema({
  pluralize: (word) => inflectPluralize[word],
  singularize: (word) => inflectSingularize[word],
.....

But this way when I run the app the following error is thrown:

I dug into the code and discovered that the pluralize and singularize are trying to access undefined properties.
To get rid of this error I need to implement both ways in pluralize and singularize:

// plural of 'pessoa': pessoas
// plural of 'contato': contatos
// plural of 'endereco': enderecos

const inflectPluralize = {
  pessoas: "pessoas",
  pessoa: "pessoas",
  contato: "contatos",
  contatos: "contatos",
  endereco: "enderecos",
  enderecos: "enderecos",
};

const inflectSingularize = {
  pessoa: "pessoa",
  pessoas: "pessoa",
  contato: "contato",
  contatos: "contato",
  endereco: "endereco",
  enderecos: "endereco",
};

const schema = new Schema({
  pluralize: (word) => inflectPluralize[word],
  singularize: (word) => inflectSingularize[word],

With this change errors are gone. But when I send the POST request, apart from all the fields being dasherized, the type is being pluralized, too: pessoa becomes pessoas.
The sent API endpoint is correct, POST /pessoas. But the entity is pessoa,not pessoas.

Do you have any thoughts on this, or I need to wait until v0.17? Have you heard/faced something like this?

[dgeb]

@RichardsonWTR for v0.16, you'll want to override the JSONAPISerializer as described here. Customize resourceType and recordType.

And yes, this will be even easier in v0.17 :)

[RichardsonWTR]

FTR, this is what my serializer looks like after the changes.

import { JSONAPISerializer } from "@orbit/jsonapi";
import { dasherize } from "@orbit/utils";

export default class CustomJSONAPISerializer extends JSONAPISerializer {
  resourceKey(type: string) {
    return "remoteId";
  }

  resourceType(type: string) {
    let caller = this.getCallerName();
    if (caller === "JSONAPIURLBuilder.resourcePath")
      return dasherize(this.schema.pluralize(type));
    return this.recordType(type);
  }

  resourceAttribute(type: string, attr: string): string {
    return attr;
  }

  getCallerName() {
    try {
      throw new Error();
    } catch (e) {
      try {
        return e.stack.split("at ")[3].split(" ")[0];
      } catch (e) {
        return "";
      }
    }
  }
}

At least in this version, when I call await memory.update((t) => t.addRecord(newPerson));, the resourceType is called twice:

• for serializing the resource path; and
  • for serializing the resource type

So when the caller is the resource path I change the string appropriately.
It's not beautiful but it worked: The resource path is pessoas and the resource type sent is pessoa. The attributes also are OK now.

[RichardsonWTR]

BUT (hope this is the last thing),

When the sent data is not the ideal (e.g., when the server return status 400) I'm having some problems.
When the first request is sent I can see the following Orbit log events (in order):

• [source-event] remote beforePush
• [source-event] remote pushFail
• [source-event] memory updateFail

When I click again in the send button nothing happens. No events, no errors. It just hangs.

About my Orbit Schema

I'll share a piece of my schema:

Orbit Schema

```typescript import { Schema } from "@orbit/data"; import MemorySource from "@orbit/memory"; import Coordinator, { RequestStrategy, SyncStrategy, EventLoggingStrategy, } from "@orbit/coordinator"; import JSONAPISource from "@orbit/jsonapi"; import { ApplicationPaths } from "../../components/api-authorization/ApiAuthorizationConstants"; import CustomJSONAPISerializer from "./Orbit/CustomJSONAPISerializer"; import { KeyMap } from "@orbit/data"; let keyMap = new KeyMap(); const inflectPluralize = { pessoas: "pessoas", pessoa: "pessoas", contato: "contatos", contatos: "contatos", endereco: "enderecos", enderecos: "enderecos", }; const inflectSingularize = { pessoa: "pessoa", pessoas: "pessoa", contato: "contato", contatos: "contato", endereco: "endereco", enderecos: "endereco", }; const schema = new Schema({ pluralize: (word) => inflectPluralize[word], singularize: (word) => inflectSingularize[word], models: { pessoa: { keys: { remoteId: {} }, attributes: { nome: { type: "string" }, cpf: { type: "string" }, dataDeNascimento: { type: "date" }, }, }, }, }); const remote = new JSONAPISource({ schema, name: "remote", host: 'https://whatever...', namespace: "api", SerializerClass: CustomJSONAPISerializer, keyMap, }); export const memory = new MemorySource({ schema, keyMap, }); export const coordinator = new Coordinator({ sources: [remote, memory], }); // Query the remote server whenever the store is queried coordinator.addStrategy( new RequestStrategy({ source: "memory", on: "beforeQuery", target: "remote", action: "pull", blocking: true, }) ); // Update the remote server whenever the store is updated coordinator.addStrategy( new RequestStrategy({ source: "memory", on: "beforeUpdate", target: "remote", action: "push", blocking: true, // catch(e) { // I tried to call requestQueue.skip() and the requestQueue.shift() from source and target, but I had no luck. // When I do this, the original exception (with the message sent from the server) is lost. // }, }) ); // Sync all changes received from the remote server to the store coordinator.addStrategy( new SyncStrategy({ source: "remote", target: "memory", blocking: true, }) ); coordinator.addStrategy( new EventLoggingStrategy({ interfaces: ["pullable", "pushable", "queryable", "syncable", "updatable"], }) ); ```

If you see the code above, you'll notice that I borrowerd the catch block from #599. The main difference is that my app is not offline first.
In summary, I need to be able to try again as much as I can, and I also need to get the data from the server, which contains a JSON API message with the error details...

Again, thank you for your support.
Any help would be appreciated.
Regards

[shiwanmenghuxiahuashan]

Orbit is undeniably great！But it is hard to hide the poor quality of its documentation。I had the same problem and finally found the answer not in the document, but in issues。Good documentation can help Orbit go further and further。Thank you