medic / couchdb-migration

0 stars 0 forks source link

Continue pre-indexing other views when a view was not found #19

Open 1yuv opened 3 months ago

1yuv commented 3 months ago

It has been seen that couchdb pre-indexing views fails occasionally due to missing or deleted views.

An unexpected error occurred HTTPResponseError: HTTP Error Response: 504 Gateway Time-out
    at Object.request (/app/src/utils.js:110:11)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async indexView (/app/src/pre-index-views.js:91:14)
    at async Promise.all (index 6)
    at async indexStagedDdocs (/app/src/pre-index-views.js:116:3)
    at async preIndexViews (/app/src/pre-index-views.js:125:3)
    at async /app/bin/pre-index-views.js:13:5 {
  response: Response {
    size: 0,
    timeout: 0,
    [Symbol(Body internals)]: { body: [PassThrough], disturbed: true, error: null },
    [Symbol(Response internals)]: {
      url: 'http://medic/:@haproxy:5984/medic/_design/:staged:medic/_view/messages_by_state?limit=0',
      status: 504,
      statusText: 'Gateway Time-out',
      headers: [Headers],
      counter: 0
    }
  },
  status: 504
}

The manul process on this case at the moment is to restart preindex views and it will resume from previous point. For larger migrations, it takes many hours to complete preindex views. It's not convinient that you figure out preindex views failed due to missing view and you have to restart.

Proposed solution:

  1. Continue indexing other views if a particular view is not found
  2. Log at the end of the process which views were not preindexed.
dianabarsan commented 3 months ago

A gateway timeout is probably not an error that we should just ignore. This can mean that the service we're connected to has crashed, and the state has changed.