Wrong results from AllDocsRequest interface getResponse().getIdsAndRevs()

[Kenzku]

Please include the following information in your ticket.

• CloudantClient (java-cloudant) version(s) that are affected by this issue. I tried 2.9 and 2.11
• Java version (including vendor and platform). java version "1.8.0_144"
• If you're using the optional okhttp dependency. 'com.squareup.okhttp3:okhttp:3.9.0'
• A small code sample that demonstrates the issue.

Database db = client.database(database, false);
    AllDocsRequest allDocsRequest = db.getAllDocsRequestBuilder()
                                      .keys(ids)
                                      .build();

    try {
      return allDocsRequest.getResponse().getIdsAndRevs();
    }  catch (Exception e) {

    }

create 4 docs in a database, with ids e.g. k_1, k_2, k_3, k_4, now remove k_3 and k_4, request _all_docs interface from a browser or postman chrome app, I got a full list of 4 items, with k_3 and k_4 marked as deleted with Cloudant-client java lib, I got 3 items back

[ricellis]

I am not able to reproduce this.

The content/construction of your String[] ids that you are passing to the keys method is not shown in your example. Please can you validate that the keys passed correctly includes "ken_4_"?

I can also see that the _rev IDs are mostly different in the second screenshot. Are all of these from the same server at localhost:5984 and what type and version of server is it (CouchDB, Cloudant Local)?

[Kenzku]

Hej,

I passed the right keys, but example above I used another keys. sorry the screenshots were caught before I wrote the issue. The second screenshots were caught after many re-tries.

I can get the wrong results every time with different data. I tested on local CouchDB, Dockerised CouchDB, ICP Dockerised CouchDB, and Bluemix Cloudant, with and without throttle of low writing and reading limits.

[Kenzku]

@ricellis

Please try with these block of code:

1. create a database in couchdb/cloudant called ken_cloudant_test_2

2. use the following code

3. run testBulkSave() to get your data in database

4. delete ken_3_, run testGetAllDocsRequestBuilder()

5. repeat 3

6. delete ken_3_ and ken_4_, run testGetAllDocsRequestBuilder()

7. you see the output result are different from what it has in the database:

   

import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
import java.util.stream.Stream;

class CouchDbUtilsDevTest {

  private CouchDbUtilsDevTest() {}

  private List<JsonObject> _getTestJsonArray() {
    List<JsonObject> jsonArray = new ArrayList<>();

    JsonObject jsonObject_1 = new JsonObject();
    jsonObject_1.addProperty("_id", "ken_1_");
    jsonObject_1.addProperty("name", "ken 1");
    jsonObject_1.addProperty("wealth", "580 billion euro");
    jsonArray.add(jsonObject_1);

    JsonObject jsonObject_2 = new JsonObject();
    jsonObject_2.addProperty("_id", "ken_2_");
    jsonObject_2.addProperty("name", "ken 2");
    jsonObject_2.addProperty("wealth", "600 billion euro 1 ");
    jsonArray.add(jsonObject_2);

    JsonObject jsonObject_3 = new JsonObject();
    jsonObject_3.addProperty("_id", "ken_3_");
    jsonObject_3.addProperty("name", "ken 3");
    jsonObject_3.addProperty("wealth", "700 billion euro 1 ");
    jsonArray.add(jsonObject_3);

    JsonObject jsonObject_4 = new JsonObject();
    jsonObject_4.addProperty("_id", "ken_4_");
    jsonObject_4.addProperty("name", "ken 4");
    jsonObject_4.addProperty("wealth", "80 billion euro new");
    jsonArray.add(jsonObject_4);

    return jsonArray;
  }

 private void testBulkSave() {
   /*GET YOUR client here*/
    Database db = client.database("ken_cloudant_test_2", false);
    List<JsonObject> oldJsonArray = this._getTestJsonArray();
    db.bulk(oldJsonArray);
  }

  private void testGetAllDocsRequestBuilder() {
    /*GET YOUR client here*/
    Database db = client.database("ken_cloudant_test_2", false);
    String[] ids = new String[]{"ken_1_", "ken_2_", "ken_3_", "ken_4_"};
    AllDocsRequest allDocsRequest = db.getAllDocsRequestBuilder()
        .keys(ids)
        .build();

    try {
      Map<String, String> allIdsAndRevs = allDocsRequest.getResponse().getIdsAndRevs();
      System.out.println(allIdsAndRevs);
    } catch (Exception ignored) {

    }
  }

  public static void main(String[] args) {
    CouchDbUtilsDevTest aCouchDBUtilsDevTest = new CouchDbUtilsDevTest();
//    aCouchDBUtilsDevTest.testBulkSave();
//    aCouchDBUtilsDevTest.testGetAllDocsRequestBuilder();
  }
}

[ricellis]

Thanks for the update, I was able to reproduce the issue now.

With the specified keys the server returns 4 rows, but gives the total_rows as 2 (the undeleted items). We use common code for handling the results from "view like" requests and the issue in this case is that common code path uses total_rows as the rows per page (for un-paginated cases). With total_rows < rows per page it falls into the pagination code path when it shouldn't, which causes it to drop the last row form the result (which is part of the pagination recipe).