search

netgroup-polito / CrownLabs

Kubernetes-based Remote Laboratories
https://crownlabs.polito.it
Apache License 2.0
105 stars 41 forks source link

Upload to NextCloud no longer works #256

Closed giorio94 closed 4 years ago

giorio94 commented 4 years ago

Describe the bug It is impossible to upload new files to NextCloud (both via the web interface and through webdav). Indeed, the file appears to be uploaded but then NextCloud displays a popup reading "An unknown error has occurred" and the file is deleted. Instead, empty folders and files can be created from the web interface correctly.

To Reproduce Steps to reproduce the behavior:

  1. Go to https://crownlabs.polito.it/cloud
  2. Log-in
  3. Try to upload a new file
  4. NextCloud will show a popup reading "An unknown error has occurred"

Expected behavior The file should be uploaded correctly

Desktop (please complete the following information):

Additional context One of the database pods is returning strange logs (may or may not be related to this issue):

$ kubectl logs -n nextcloud nextcloud-db-cluster-1

/var/run/postgresql:5432 - rejecting connections
2020-05-15 09:07:29,766 INFO: Lock owner: nextcloud-db-cluster-2; I am nextcloud-db-cluster-1
2020-05-15 09:07:29,767 INFO: Still starting up as a standby.
2020-05-15 09:07:29,768 INFO: Lock owner: nextcloud-db-cluster-2; I am nextcloud-db-cluster-1
2020-05-15 09:07:29,768 INFO: does not have lock
2020-05-15 09:07:29,769 INFO: establishing a new patroni connection to the postgres cluster
2020-05-15 09:07:30,492 INFO: establishing a new patroni connection to the postgres cluster
2020-05-15 09:07:30,518 WARNING: Retry got exception: 'connection problems'
2020-05-15 09:07:30,519 INFO: Error communicating with PostgreSQL. Will try again later

Additionally, if I remember correctly, the nextcloud-db-cluster-1 pod has been killed forcefully while draining the node to upgrade the system O.S.

giorio94 commented 4 years ago

The problem related to file uploads in NextCoud should have been fixed. Hence, I am closing this issue. Please, feel free to reopen if it occurs again.

Apparently, there where two main problems (still unclear the root cause):

  1. The database cluster was in a corrupted state and one of the members (nextcloud-db-cluster-1) remained in starting state (as confirmed by the output of patronictl list). This issue has been fixed executing patronictl -c postgres.yml reinit nextcloud-db-cluster nextcloud-db-cluster-1 from inside the pod itself to reinitialize the member.
  2. The folders used by NextCloud to temporarily save the files during the upload where no longer present. These folders are configured in the php configuration here: https://github.com/netgroup-polito/CrownLabs/blob/2763f20ea323ce16ceaf84544d796124851e638e/infrastructure/file-sharing/manifests/nextcloud-php-configmap.yaml#L6-L7 After having created the two folders in the mounted PVC, the file upload started working again normally.