Partitioned _all_docs endpoint with non-partitioned docId answers with not-properly-formatted HTTP response

vmatyus commented 4 years ago

Description

This problem occurs only on CouchDB. Can be reproduced with PostMan and ibm-cloudant js and java package. With POST {db}/_partition/{partition}/_all_docs endpoint if the body contains a non partitioned key e.g: "invalidKey" then the server (CouchDB) responds intermittently 400 Bad Request.

Steps to Reproduce

Create a partitioned database

Send the following request:

POST {db}/_partition/{partition}/_all_docs
{
"keys": [
 "doc12"
]
}

The PostMan will respond with no result code and the following error message:

Could not get any response
There was an error connecting to {couchdb_url}/{db}/_partition/{partition}/_all_docs.
Why this might have happened:
The server couldn't send a response:
Ensure that the backend is working properly
Self-signed SSL certificates are being blocked:
Fix this by turning off 'SSL certificate verification' in Settings > General
Proxy configured incorrectly
Ensure that proxy is configured correctly in Settings > Proxy
Request timeout:
Change request timeout in Settings > General

The JS package will response with:

400 "HPE_INVALID_CONSTANT"
body: "Response not received - no connection was made to the service."
message: "Parse Error: Expected HTTP/"
stack: "Error: Parse Error: Expected HTTP/
    at RequestWrapper.formatError (../node_modules/ibm-cloud-sdk-core/lib/request-wrapper.js:208:21)
    at ../node_modules/ibm-cloud-sdk-core/lib/request-wrapper.js:196:25
    at processTicksAndRejections (internal/process/task_queues.js:85:5)"

The Java responds with:

Caused by: java.net.ProtocolException: Unexpected status line: 2c
    at okhttp3.internal.http.StatusLine.parse(StatusLine.java:69)

If you make this command with curl command with verbose option:

curl --request POST {couchdb_url}/{db}/_partition/{partition}/_all_docs' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic {token}' \
--data-raw '{"keys": ["invalidKey"]}' -v

Then the curl response sometimes includes the following line: * Excess found in a non pipelined read: excess = 58, size = 73, maxdownload = 73, bytecount = 0

Expected Behaviour

The PostMan, JS and Java package should receive a well-formatted HTTP response.

Your Environment

CouchDB version used: "3.0.0-6d432a9"
Browser name and version: PostMan 7.15.0
Operating system and version: MacOS Catalina 10.15.2

Additional Context

ricellis commented 4 years ago

This is reproducible for me (intermittently, but more often than not). Steps:

curl -X PUT -u 'u:p' 'http://localhost:5984/test2480?partitioned=true'
curl -u 'u:p' -H 'Content-Type: application/json' -d '{"keys":["invalid"]}' -v 'http://localhost:5984/test2480/_partition/a/_all_docs'

Repeat step 2 until the * Excess found in a non pipelined read message appears e.g.

*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 5984 (#0)
* Server auth using Basic with user '****'
> POST /test2480/_partition/a/_all_docs HTTP/1.1
> Host: localhost:5984
> Authorization: Basic ****==
> User-Agent: curl/7.64.1
> Accept: */*
> Content-Type: application/json
> Content-Length: 20
> 
* upload completely sent off: 20 out of 20 bytes
< HTTP/1.1 400 Bad Request
< Cache-Control: must-revalidate
< Content-Length: 73
< Content-Type: application/json
< Date: Wed, 26 Feb 2020 10:33:41 GMT
< Server: CouchDB/3.0.0 (Erlang OTP/20)
< X-Couch-Request-ID: 9c85a12afc
< X-CouchDB-Body-Time: 0
< 
* Excess found in a non pipelined read: excess = 63, size = 73, maxdownload = 73, bytecount = 0
{"error":"illegal_docid","reason":"Doc id must be of form partition:id"}
* Connection #0 to host localhost left intact
* Closing connection 0

For me the excess is 63 bytes and if I capture them with wireshark they are:

2c
{"total_rows":0,"offset":null,"rows":[

]}
3
);

0

So it looks like sometimes the view-like "wrapper" gets mangled into the error response.

wohali commented 4 years ago

[edited] Cannot reproduce on Windows. CAN reproduce on Linux. Interestingly, I can confirm this behaviour on both SM 1.8.5 and 60.

$ curl -u 'u:p' -H 'Content-Type: application/json' -d '{"keys":["invalid"]}' -v 'http://localhost:15984/test2480/_partition/a/_all_docs'
* Expire in 0 ms for 6 (transfer 0x56228db12e80)
* Expire in 1 ms for 1 (transfer 0x56228db12e80)
* Expire in 0 ms for 1 (transfer 0x56228db12e80)
* Expire in 1 ms for 1 (transfer 0x56228db12e80)
* Expire in 0 ms for 1 (transfer 0x56228db12e80)
* Expire in 0 ms for 1 (transfer 0x56228db12e80)
* Expire in 0 ms for 1 (transfer 0x56228db12e80)
*   Trying ::1...
* TCP_NODELAY set
* Expire in 149999 ms for 3 (transfer 0x56228db12e80)
* Expire in 200 ms for 4 (transfer 0x56228db12e80)
* connect to ::1 port 15984 failed: Connection refused
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Expire in 149999 ms for 3 (transfer 0x56228db12e80)
* Connected to localhost (127.0.0.1) port 15984 (#0)
* Server auth using Basic with user 'u'
> POST /test2480/_partition/a/_all_docs HTTP/1.1
> Host: localhost:15984
> Authorization: Basic dTpw
> User-Agent: curl/7.64.0
> Accept: */*
> Content-Type: application/json
> Content-Length: 20
>
* upload completely sent off: 20 out of 20 bytes
< HTTP/1.1 400 Bad Request
< Cache-Control: must-revalidate
< Content-Length: 73
< Content-Type: application/json
< Date: Wed, 26 Feb 2020 18:52:27 GMT
< Server: CouchDB/3.0.0-03a77db (Erlang OTP/20)
< X-Couch-Request-ID: d842650a3e
< X-CouchDB-Body-Time: 0
<
* Excess found in a non pipelined read: excess = 63, size = 73, maxdownload = 73, bytecount = 0
{"error":"illegal_docid","reason":"Doc id must be of form partition:id"}

rnewson commented 4 years ago

fyi: it's intermittent because it depends on whether the fabric:get_doc_count response arrives before the ?MODULE, open_doc response.

vmatyus commented 4 years ago

In my local development environment it is quite frequent. 1 of 10 times succeeds returning the body content, but it is indeed not deterministic, sometimes the 8th try succeeds.

apache / couchdb