medic / cht-core

The CHT Core Framework makes it faster to build responsive, offline-first digital health apps that equip health workers to provide better care in their communities. It is a central resource of the Community Health Toolkit.
https://communityhealthtoolkit.org
GNU Affero General Public License v3.0
438 stars 209 forks source link

Install button doesn't appear after staging an upgrade sometimes #8573

Open dianabarsan opened 1 year ago

dianabarsan commented 1 year ago

Describe the bug When staging an upgrade, the process should pause after view indexing and the admin upgrade page should show a button labeled "install" that serves to complete the upgrade. This button sometimes does not appear.

To Reproduce Steps to reproduce the behavior:

  1. Install 4.3.0
  2. Stage 4.4.0
  3. ??? I am unsure what triggered this issue.
  4. Notice you cannot complete the upgrade and the last logged step is indexing views.

Expected behavior After views are indexed, an install button should appear.

Environment

Additional context Manually changing the status of the upgrade log to "indexed" makes the button appear.

dianabarsan commented 1 year ago

I have not been able to replicate this. My suspicion is that, on the instance where we have seen this, some view request was just hanging and we should have just waited more. (view indexing took a few hours). Leaving this open though.

dianabarsan commented 2 months ago

Since this has been popping up more frequently recently, I started looking into it again. I managed to replicate this one time, it appear to happen if CouchDb crashes - or if haproxy believes CouchDb crashes - during indexing. Besides my local test, I also have logs from a training instance where this happened, and I'm seeing the same CouchDb - haproxy failure in the logs.