Investigate _all_docs throwing a 500 error

CouchDb Logs:

[error] 2024-08-23T14:02:29.272852Z couchdb@127.0.0.1 <0.10861.1744> 582fa4c03d req_err(1041696419) unknown_error : function_clause
    [<<"couch_mrview_http:view_cb/2 L338">>,<<"chttpd_db:all_docs_view/4 L769">>,<<"chttpd:handle_req_after_auth/2 L320">>,<<"chttpd:process_request/1 L303">>,<<"chttpd:handle_request_int/1 L243">>,<<"mochiweb_http:headers/6 L128">>,<<"proc_lib:init_p_do_apply/3 L247">>]
[notice] 2024-08-23T14:02:29.273144Z couchdb@127.0.0.1 <0.10861.1744> 582fa4c03d haproxy.echis-kisumu.svc.cluster.local:5984 10.42.41.15 medic POST /medic/_all_docs?include_docs=true 500 ok 10002
[error] 2024-08-23T14:02:30.126800Z couchdb@127.0.0.1 <0.14771.1780> 3aa63855a4 req_err(1041696419) unknown_error : function_clause
    [<<"couch_mrview_http:view_cb/2 L338">>,<<"chttpd_db:all_docs_view/4 L769">>,<<"chttpd:handle_req_after_auth/2 L320">>,<<"chttpd:process_request/1 L303">>,<<"chttpd:handle_request_int/1 L243">>,<<"mochiweb_http:headers/6 L128">>,<<"proc_lib:init_p_do_apply/3 L247">>]
[notice] 2024-08-23T14:02:30.127147Z couchdb@127.0.0.1 <0.14771.1780> 3aa63855a4 haproxy.echis-kisumu.svc.cluster.local:5984 10.42.41.15 medic POST /medic/_all_docs?include_docs=true 500 ok 10002

API logs:

2024-08-23 14:05:26 ERROR: Server error: {
  error: 'unknown_error',
  reason: 'function_clause',
  ref: 1041696419,
  status: 500,
  name: 'unknown_error',
  message: 'function_clause',
  stack: 'Error\n' +
    '    at Object.generateErrorFromResponse (/api/node_modules/pouchdb-errors/lib/index.js:104:18)\n' +
    '    at /api/node_modules/pouchdb-adapter-http/lib/index.js:254:33\n' +
    '    at runMicrotasks (<anonymous>)\n' +
    '    at processTicksAndRejections (node:internal/process/task_queues:96:5)'
} 
RES 8d4139b0-da1c-4617-8a60-6e34f02b02bc 10.42.41.0 - GET /medic/_changes?timeout=600000&style=all_docs&heartbeat=10000&since=14686981-g1AAAAO1eJyV0rFKBDEQBuDgCTY2ivbWFrJZs5u9SgtfQO_yADvZhGVZtbrawkcQwUZEBb1LZWfhyeFD3APYXuUDeBfHjOUhhBQ_JOQjM5OWMbZedyq2oc8Huq7gkKdyL8HFWzxaKRlsK9Vv6g6wzfHVKe6tCZMVWubL7vwj_ToKdp0bEfYxCBg3QnQzE4uh4-DA-wVhb1sB01IWuSljMXQ84Ot6hL20ATPAdSogFkOnD41zQ8Luv6hMa2Hf6lgMnRFceD8n7P0zYGlp0spGl4nO4myVnSh1TdzDUeCsMdLKLJJDqYcSxjPOgsAbG8AEINdlEQ8OUcL49n5C4O0jjUIXhUxkPDhHCeNYqSmBr2MCuRYCYjtIzZtiPDk3I_BuJ4DdLK8k2HgQmzfDwB_o_4ZySTPOK254suxq8wMaIQdg&limit=100 HTTP/1.1 500 - 60085.845 ms
2024-08-23 14:07:35 ERROR: Server error: FetchError: Invalid response body while trying to fetch http://haproxy.echis-kisumu.svc.cluster.local:5984/medic/_all_docs?include_docs=true: Premature close
    at PassThrough.<anonymous> (/api/node_modules/node-fetch/lib/index.js:400:12)
    at PassThrough.emit (node:events:525:35)
    at emitErrorNT (node:internal/streams/destroy:157:8)
    at emitErrorCloseNT (node:internal/streams/destroy:122:3)
    at processTicksAndRejections (node:internal/process/task_queues:83:21) {
  message: 'Invalid response body while trying to fetch http://haproxy.echis-kisumu.svc.cluster.local:5984/medic/_all_docs?include_docs=true: Premature close',
  type: 'system',
  errno: 'ERR_STREAM_PREMATURE_CLOSE',
  code: 'ERR_STREAM_PREMATURE_CLOSE',
  [stack]: 'FetchError: Invalid response body while trying to fetch http://haproxy.echis-kisumu.svc.cluster.local:5984/medic/_all_docs?include_docs=true: Premature close\n' +
    '    at PassThrough.<anonymous> (/api/node_modules/node-fetch/lib/index.js:400:12)\n' +
    '    at PassThrough.emit (node:events:525:35)\n' +
    '    at emitErrorNT (node:internal/streams/destroy:157:8)\n' +
    '    at emitErrorCloseNT (node:internal/streams/destroy:122:3)\n' +
    '    at processTicksAndRejections (node:internal/process/task_queues:83:21)',
  name: 'FetchError'
} 
RES df6c3abe-0243-4e35-a9b6-7e072a60fa88 10.42.41.0 - POST /medic/_bulk_docs HTTP/1.1 500 - 13805.508 ms

medic / cht-sync

Investigate _all_docs throwing a 500 error #141