Problems when performing non-get requests with HAL-browser on resources with associations [DATAREST-721]

[spring-projects-issues]

Loïc Dewerchin opened DATAREST-721 and commented

Hello, we have been trying out the HAL-browser in our Spring Boot / SDR project by using the spring-data-rest-hal-browser dependency

Everything works fine when performing:

• GET requests on resources with and without associated resources
• non-GET requests on resources without associated resources

However when performing a non-GET request on a resource with another associated resource the Create/Update pop up does not display the input-fields(should be generated by json-editor) or a general JSON input field (fallback?) . The Make Request button also does not trigger any action.

When checking the browser dev-tools there is an error in CustomPostForm.js (line 112) when loading the JSON Schema-driven editor. Error:

TypeError: can't convert undefined to object

line of code

Object.keys(schema.properties).forEach(function (property) {

If I understand correctly first a call to the profile/ of the resource is made (OK) and that information is used to feed the JSON editor so it can construct input fields. I will post the profile/ information of the resource in the comment below as it may provide some valuable information for you.

Now , I also checked with a small sample SDR project, with a couple of simple entities that are linked : here the HAL browser does not choke on processing the profile/ but we observed another behavior : the linked resource are not shown as "input fields" in the Create/Update pop-up (on either side of the bidirectional linking... ) , making it impossible to create a valid new resource instance on one side of the linking

I am wondering if it wouldn't be convenient to just fall-back to the old input field were one can just enter some json (so drop the json-editor) when the call the the profile/ reveals that there is an associated resource?

---

Affects: 2.4.1 (Gosling SR1)

[spring-projects-issues]

Loïc Dewerchin commented

This is the response body of a call to api/profile/items , in regards to the first problem

{
  "alps": {
    "version": "1.0",
    "descriptors": [
      {
        "id": "toDo-representation",
        "href": "http://local.todo.ugent.be:8080/api/profile/items",
        "descriptors": [
          {
            "name": "id",
            "type": "SEMANTIC"
          },
          {
            "name": "name",
            "type": "SEMANTIC"
          },
          {
            "name": "createdByUser",
            "type": "SEMANTIC"
          },
          {
            "name": "checked",
            "type": "SEMANTIC"
          },
          {
            "name": "checkedByUser",
            "type": "SEMANTIC"
          },
          {
            "name": "toDoList",
            "type": "SAFE",
            "rt": "http://local.todo.ugent.be:8080/api/profile/lists#toDoList-representation"
          }
        ]
      },
      {
        "id": "get-items",
        "name": "items",
        "type": "SAFE",
        "rt": "#toDo-representation",
        "descriptors": [
          {
            "name": "page",
            "doc": {
              "value": "The page to return.",
              "format": "TEXT"
            },
            "type": "SEMANTIC"
          },
          {
            "name": "size",
            "doc": {
              "value": "The size of the page to return.",
              "format": "TEXT"
            },
            "type": "SEMANTIC"
          },
          {
            "name": "sort",
            "doc": {
              "value": "The sorting criteria to use to calculate the content of the page.",
              "format": "TEXT"
            },
            "type": "SEMANTIC"
          }
        ]
      },
      {
        "id": "create-items",
        "name": "items",
        "type": "UNSAFE",
        "rt": "#toDo-representation"
      },
      {
        "id": "get-toDo",
        "name": "toDo",
        "type": "SAFE",
        "rt": "#toDo-representation"
      },
      {
        "id": "patch-toDo",
        "name": "toDo",
        "type": "UNSAFE",
        "rt": "#toDo-representation"
      },
      {
        "id": "delete-toDo",
        "name": "toDo",
        "type": "IDEMPOTENT",
        "rt": "#toDo-representation"
      },
      {
        "id": "update-toDo",
        "name": "toDo",
        "type": "IDEMPOTENT",
        "rt": "#toDo-representation"
      },
      {
        "name": "findByToDoList",
        "type": "SAFE",
        "descriptors": [
          {
            "name": "toDoListId",
            "type": "SEMANTIC"
          }
        ]
      }
    ]
  }
}

[spring-projects-issues]

Oliver Drotbohm commented

Greg Turnquist - Looks like our custom JavaScript needs some tweaks. Care to have a look?

[spring-projects-issues]

Greg Turnquist commented

@loic.dewerchin You attached the ALPS metadata document. Can you query the entity's "profile" link with header "Accept:application/schema+json" instead, and post that document in this ticket?

[spring-projects-issues]

Loïc Dewerchin commented

Ok, thx for that tip, didn't know about that... this is the output

{
  "title" : "To do",
  "definitions" : { },
  "type" : "object",
  "$schema" : "http://json-schema.org/draft-04/schema#"
}

Doesn't look that good , the properties of todo should be here (http://docs.spring.io/spring-data/rest/docs/current/reference/html/#metadata.json-schema), checked our jackson-objectmapper config , and found the "problem" there : the access to the properties was configured to use public getter methods only.

@Override
    public void configureJacksonObjectMapper(ObjectMapper objectMapper) {
        super.configureJacksonObjectMapper(objectMapper);
        objectMapper.setVisibility(PropertyAccessor.ALL, JsonAutoDetect.Visibility.NONE);
        objectMapper.setVisibility(PropertyAccessor.GETTER, JsonAutoDetect.Visibility.PUBLIC_ONLY);
        objectMapper.setVisibility(PropertyAccessor.IS_GETTER, JsonAutoDetect.Visibility.PUBLIC_ONLY);*/

    }

Falling back to the default jackson object-mapper , gives the expected result

{
  "title" : "To do",
  "properties" : {
    "checkedByUser" : {
      "title" : "Checked by user",
      "readOnly" : false,
      "type" : "string"
    },
    "toDoList" : {
      "title" : "To do list",
      "readOnly" : false,
      "type" : "string",
      "format" : "uri"
    },
    "createdByUser" : {
      "title" : "Created by user",
      "readOnly" : false,
      "type" : "string"
    },
    "name" : {
      "title" : "Name",
      "readOnly" : false,
      "type" : "string"
    },
    "checked" : {
      "title" : "Checked",
      "readOnly" : false,
      "type" : "boolean"
    },
    "id" : {
      "title" : "Id",
      "readOnly" : false,
      "type" : "string"
    }
  },
  "definitions" : { },
  "type" : "object",
  "$schema" : "http://json-schema.org/draft-04/schema#"
}

And idd, the problem with the JSON editor disappears, and the input fields are visible... Now does this mean one cannot use a specific objectmapping config like ours, or is the visibility of the fields mandatory? Another note on this : we also tried out a object-mapping config with only fields for object mapping, but that messed up using projections.

Another problem remains though

the linked resource are not shown as "input fields" in the Create/Update pop-up (on either side of the bidirectional linking... ) , making it impossible to create a valid new resource instance on one side of the linking

So in this PoC, we have a ToDo list which can contain a number of ToDo items... each ToDo item has a link back to its list (bi-directional relationship) Creating a ToDo list with the HAL browser works fine : all the necessary fields are present , end up with a POST request body of

{"name" : "a hal-browser list" , "createdByUser" : "user1" , "type":  "work"}

However, to create a new ToDo item , one has to reference the list, which results in the following POST request body

{"name" : "do  stuff" , "createdByUser" : "user1" , "toDoList":  "http://localhost:8080/api/lists/442"}

but the input field for the associated resource (toDoList) is not present, making it impossible to create a new item resource ...

[spring-projects-issues]

Greg Turnquist commented

Whether or not your altered version of serving up getters should still render JSON Schema metadata sounds like a separate issue @olivergierke may need to evaluate.

Regarding handling links through the HAL Browser is a bit trickier. For now, they are simply disabled. That's because for a proof of concept, I couldn't justify spending a lot of time figuring out how to find a list of possible entities and putting them in a drop down. How easy is it to build a list? Would it have 10s, 100s, or 1000s of entries? Should paging be used? Since I wasn't sure how to customize a single field inside the JSON Editor to use a raw string for URI, nor was I sure that is the right solution anyways, I simply punted on supporting that feature for now

[spring-projects-issues]

Loïc Dewerchin commented

Hello,

thanks for the feedback :)

Now I understand there can be some issues like you described, but the problem remains : it is not possible to add entities with associated resources, making the HAL browser tool for developers not ideal. The fact that the field for the associated resource is not visible also means that the "documenting" part of the REST service(as in, an interactive documentation experience that the HAL browser gives) is a bit lacking, as they have to do a separate call to get the json schema and find out that the lack of the associated resource URL field is the reason for the failing request.

I might miss something but as the json schema specifies

    "toDoList" : {

      "title" : "To do list",

      "readOnly" : false,
    "toDoList" : {

      "title" : "To do list",

      "readOnly" : false,

      "type" : "string",

      "format" : "uri"

    }
      "type" : "string",

      "format" : "uri"

    }

is it not possible to add a plain String input field (with a mention that an URI to the associated resource is expected), with the responsibility for the developer who is experimenting with the REST service to use a correct URI. Drop-downs etc seem really nice but not really necessary. This way the format of the POST request is also close to what the developer will use when actually working with the REST service.

If not, maybe it's possible to provide the option of just constructing the POST body yourself? Like a text field as used here ( http://haltalk.herokuapp.com/explorer/browser.html#/ ))

best regards, Loïc

[spring-projects-issues]

Greg Turnquist commented

The trick in all this, is that we are attempting to reuse a few off-the-shelf OSS tools to minimize work on our end.

• HAL Browser - by default, EVERYTHING is raw JSON you hand type for entry
• JSON Editor - out-of-the-box JavaScript project to turn JSON Schema into an entry form

I coded a plugin to HAL Browser that fetches a selected entity's JSON Schema info, and instead of putting a plain HTML text entry, it swaps it out with the JSON Editor code to dynamically render a form. Since that thing doesn't really comprehend links, I simply filtered them out for a proof of concept.

Customizing the output of JSON Editor is out of the question. The point of using it is to quickly have a solution. Rehabbing all of HAL Browser is also not an effective use of resources.

So....perhaps the approach that dovetails with your idea of hand entering URIs for links, is to add an extra section below the JSON Editor section where link information can be supplied as a free form entry. Since I am removing those fields from the JSON Schema, I would know what needs to be added to this section

[spring-projects-issues]

Loïc Dewerchin commented

Hello,

yeah I think the solution with the free form as a good idea, as it is time-cost effective and certainly good enough for developers (IMO) In any case, thanks for the work you put into it, SDR is a great Spring component and the easy integration of the HAL browser is a very valuable addition! Easy testing/documenting of a REST service is a great asset :)

best regards, Loïc