Document JSON structure contains `map` keys when adding a List of POJOs

[fritzfranzke]

Describe the bug Creating a Document from a Java Pojo containing a List of another Pojo creates a weird JSON structure by introducing the key map.

Edit: For context, this extra nesting breaks the index (and subsequently the selector) I intended to use. Removing the nesting manually fixes this problem.

{
  "index": {
    "fields": [
      "pages.[].content"
    ]
  },
  "type": "json"
}

To Reproduce

1. Create a Pojo, for example Book, containing a List of another Pojo, for example Page, eg:

   
   public class Book {
   private String _id;
   private String _rev;
   private String author;
   private List<Page> pages;
   
   public Document toDocument() {
       var document = new Document();
       document.put("_id", _id);
       document.put("_rev", _rev);
       document.put("author", author);    
       document.put("pages", pages);
       return document;
   }
   
   // constructor + getter/setter
   }

public class Page { private Integer number; private String content;

// constructor + getter/setter

}

2. Insert a `Book` using the `toDocument()` method
```java
var pages = List.of(
    new Page(1, "LoremIpsum"),
    new Page(2, "LoremIpsum")
);

var book = new Book("my-author", pages)
    .toDocument();

var documentOptions = new PostDocumentOptions.Builder()
    .db(dbName)
    .document(book)
    .build();

try {
    var response = dbClient.postDocument(documentOptions).execute().getResult();

    // handle response
} catch (ConflictException | BadRequestException e) {
    // handle error
}

3. The JSON stored in the CouchDB has the following structure:

 {
    "_id": "my-id",
    "_rev": "my-ref",
    "author": "my-author",
    "pages": [
      {
        "map": {
          "number": "1",
          "content": "Lorem Ipsum"
        }
      },
      {
        "map": {
          "number": "2",
          "content": "Lorem Ipsum"
        }
      }
      // more pages
    ]
  }

Expected behavior I expect the document to look like this:

 {
    "_id": "my-id",
    "_rev": "my-ref",
    "author": "my-author",
    "pages": [
      {
          "number": "1",
          "content": "Lorem Ipsum"
      },
      {
          "number": "2",
          "content": "Lorem Ipsum"
      }
      // more pages
    ]
  }

Must gather (please complete the following information):

• SDK Version: 0.4.3
• Java Version: openjdk version "17.0.6" 2023-01-17
• Trying to insert document using the SDK with CouchDB Docker container v. 3.3.1

Additional context

• Is this a bug in the document.put() method?
• If the behaviour is intended, does that mean I manually need to convert a Result back to the correct JSON structure?

I considered adding a toDocument() method to Page as well, but to me this doesn't make much sense, since a Page isn't a standalone Document.

Hoping I didn't miss something in the docs and kindly asking for your support. Best, Fritz

[mojito317]

Hi Fritz! Thanks for the thorough issue description!

I wanted to repro your code with openjdk 17, but I was not sure if you intentionally left out the return type from your toDocument method. I have used the com.ibm.cloud.cloudant.v1.model.Document return type there. With that I got the following document in the database:

{
  "_id": "8310e043f95bf8a8845b681d8c0109ce",
  "_rev": "1-8859dde37e892bd6951b455130523c70",
  "pages": [
    {
      "number": 1,
      "content": "LoremIpsum"
    },
    {
      "number": 2,
      "content": "LoremIpsum"
    }
  ],
  "author": "my-author"
}

Did not you use some serialization in between that could change your Page object to a map?

[fritzfranzke]

Hey, thanks for your quick reply! You're right, the return type was indeed meant to be Document. I edited my initial comment accordingly.

So after all your suspicion was correct: I am using your library in a Vert.X application in conjunction with the Service Proxy. The way this works is that Pojos are serialized to JSON, sent over the EventBus, and deserialized back into the original Pojo. At least that's the way I understood it.

Changing the constructor from

public Book(JsonObject jsonObject) {
    this._id = jsonObject.getString("_id");
    this._rev = jsonObject.getString("_rev");
    this.author = jsonObject.getString("author");
    this.pages = (List<Page>) jsonObject.getJsonArray("pages").getList();
}

to this

public Book(JsonObject jsonObject) {
    this._id = jsonObject.getString("_id");
    this._rev = jsonObject.getString("_rev");
    this.author = jsonObject.getString("author");

    List<Page> pages = new ArrayList<>();
    jsonObject.getJsonArray("pages").stream()
        .map(page -> (JsonObject) page)
        .forEach(page -> pages.add(new Page(page)));

    this.pages = pages;
}

fixes the problem, i.e. generates the correct JSON structure without the extra map key.

But here is what I don't get: In my application the toDocument() method from my initial comment doesn't use the JsonObject but works directly on the Pojo. So how can this intermediate step produce different results? The Pojo should always be the same, no?

Anyway, thanks for your help, much appreciated!

[ricellis]

@fritzfranzke this appears to be a vertx specific behaviour for being able to use either raw or JSON values. Their docs for JsonArray#getList() method say:

Get the underlying List as is. This list may contain values that are not the types returned by the JsonArray and with an unpredictable representation of the value, e.g you might get a JSON object as a JsonObject or as a Map.

JsonArray#stream() on the other hand says:

Get a Stream over the entries in the JSON array. The values in the stream will follow the same rules as defined in getValue(int), respecting the JSON requirements. To stream the raw values, use the storage object stream instead: jsonArray.getList().stream()

[ricellis]

Given that I suspect that this.pages = (List<Page>) jsonObject.getJsonArray("pages").stream().collect(Collectors.toList()); might work as well.

[fritzfranzke]

Hi, thanks for the insights! Both your suggestion as well as the following snippet generated a JSON structure including the map key.

this.pages = (List<Page>) jsonObject.getJsonArray("pages").getList().stream().toList();

Still wondering how a Java pojo can have different internal representations, and also what other side effects this may have in other scenarios. Never occurred to me that this was possible.