search

gigascience / gigadb-website

Source code for running GigaDB
http://gigadb.org
GNU General Public License v3.0
9 stars 15 forks source link

FUW local deployment on Ubuntu #1659

Closed luistoptal closed 7 months ago

luistoptal commented 9 months ago

User story

As a dev I want to deploy the project locally in Ubuntu and be able to follow the FUW flow without encountering errors, as described here: https://github.com/gigascience/gigadb-website/pull/1592

(please see the last issue reported at the bottom, Update (1-feb) heading)

I describe the steps I follow to encounter the errors:

Unit Tests (4) ------------------------------------------------------------------------------------------------------------------------------ Modules: Asserts, \Helper\Unit

✔ DatasetTest: Get authors (0.29s) ✔ DatasetTest: Get author names (0.07s) ✔ DatasetTest: Get curator name (0.07s)

Time: 1.86 seconds, Memory: 18.00MB

OK (4 tests, 7 assertions)

- To test the flow manually I run the up script again (I assume the automated tests change the local DB, because unless I run the script again, I no longer see the `200070` id)
- I access the dataset admin screen as admin (admin@gigadb.org) and see:
![image](https://github.com/gigascience/gigadb-website/assets/143437854/7644f238-70bd-40d6-b341-916217fca3b0)
- In http://gigadb.gigasciencejournal.com/adminDataset/update/id/668 I change Submitter to `user@gigadb.org` and Upload Status to `AssigningFTPBox` and save
![image](https://github.com/gigascience/gigadb-website/assets/143437854/1faff3d4-2df8-4c2c-9e22-803662070f3c)
- I see the dropbox icon in the admin page
![image](https://github.com/gigascience/gigadb-website/assets/143437854/a9e522dc-3f72-4dd4-b38c-4736ae7e910d)
- I click on the dropbox icon and the site shows this alert
![image](https://github.com/gigascience/gigadb-website/assets/143437854/84a47303-7e1e-4722-9b0b-9146c5e022e4)
- I use Portainer to inspect the container logs. Here Containers>deployment_application_1>Logs I see these logs:

The requesting user doesn't have admin role in /var/www/protected/components/FiledropService.php (77) in /var/www/protected/controllers/adminDataset/AssignFTPBoxAction.php (32) in /var/www/index.php (24)


Alternatively, if I run the same flow described above, I can also run into the following error (I can't reliably reproduce one or the other, 5 days ago I saw the one above, today after giving it another try I see the error below), on this page http://gigadb.gigasciencejournal.com/adminDataset/assignFTPBox/id/200070 (after clicking the dropbox icon)

I've seen the above issue once, the below issue multiple times across different days

GuzzleHttp\Exception\ServerException
Server error: `POST http://fuw-admin-api/filedrop-accounts` resulted in a `500 Internal Server Error` response:
{"name":"PHP Warning","message":"mkdir(): Permission denied","code":2,"type":"yii\\base\\ErrorException","file":"/app/ba (truncated...)

![image](https://github.com/gigascience/gigadb-website/assets/143437854/8a632836-1c56-47ba-9bff-8abf32e5e54d)

In Containers>deployment_application_1>Logs I see the same error:

GuzzleHttp\Exception\ServerException: Server error: POST http://fuw-admin-api/filedrop-accounts resulted in a 500 Internal Server Error response: {"name":"PHP Warning","message":"mkdir(): Permission denied","code":2,"type":"yii\base\ErrorException","file":"/app/ba (truncated...) in /var/www/vendor/guzzlehttp/guzzle/src/Exception/RequestException.php:113  Stack trace:

0 /var/www/vendor/guzzlehttp/guzzle/src/Middleware.php(65): GuzzleHttp\Exception\RequestException::create(Object(GuzzleHttp\Psr7\Request), Object(GuzzleHttp\Psr7\Response))

1 /var/www/vendor/guzzlehttp/promises/src/Promise.php(204): GuzzleHttp\Middleware::GuzzleHttp{closure}(Object(GuzzleHttp\Psr7\Response))

2 /var/www/vendor/guzzlehttp/promises/src/Promise.php(153): GuzzleHttp\Promise\Promise::callHandler(1, Object(GuzzleHttp\Psr7\Response), NULL)

3 /var/www/vendor/guzzlehttp/promises/src/TaskQueue.php(48): GuzzleHttp\Promise\Promise::GuzzleHttp\Promise{closure}()

4 /var/www/vendor/guzzlehttp/promises/src/Promise.php(248): GuzzleHttp\Promise\TaskQueue->run(true)

5 /var/www/vendor/guzzlehttp/promises/src/Promise.php(224): GuzzleHttp\Promise\Promise->invokeWaitFn()

6 /var/www/vendor/guzzlehttp/promises/src/Promise.php(269): GuzzleHttp\Promise\Promise->waitIfPending()

7 /var/www/vendor/guzzlehttp/promises/src/Promise.php(226): GuzzleHttp\Promise\Promise->invokeWaitList()

8 /var/www/vendor/guzzlehttp/promises/src/Promise.php(62): GuzzleHttp\Promise\Promise->waitIfPending()

9 /var/www/vendor/guzzlehttp/guzzle/src/Client.php(182): GuzzleHttp\Promise\Promise->wait()

10 /var/www/protected/components/FiledropService.php(98): GuzzleHttp\Client->request('POST', 'http://fuw-admi...', Array)

11 /var/www/protected/controllers/adminDataset/AssignFTPBoxAction.php(32): FiledropService->createAccount()

12 [internal function]: AssignFTPBoxAction->run('200070')

13 /var/www/vendor/yiisoft/yii/framework/web/actions/CAction.php(115): ReflectionMethod->invokeArgs(Object(AssignFTPBoxAction), Array)

14 /var/www/vendor/yiisoft/yii/framework/web/actions/CAction.php(74): CAction->runWithParamsInternal(Object(AssignFTPBoxAction), Object(ReflectionMethod), Array)

15 /var/www/vendor/yiisoft/yii/framework/web/CController.php(308): CAction->runWithParams(Array)

16 /var/www/vendor/yiisoft/yii/framework/web/filters/CFilterChain.php(134): CController->runAction(Object(AssignFTPBoxAction))

17 /var/www/vendor/yiisoft/yii/framework/web/filters/CFilter.php(40): CFilterChain->run()

18 /var/www/vendor/yiisoft/yii/framework/web/CController.php(1148): CFilter->filter(Object(CFilterChain))

19 /var/www/vendor/yiisoft/yii/framework/web/filters/CInlineFilter.php(58): CController->filterAccessControl(Object(CFilterChain))

20 /var/www/vendor/yiisoft/yii/framework/web/filters/CFilterChain.php(131): CInlineFilter->filter(Object(CFilterChain))

21 /var/www/vendor/yiisoft/yii/framework/web/CController.php(291): CFilterChain->run()

22 /var/www/vendor/yiisoft/yii/framework/web/CController.php(265): CController->runActionWithFilters(Object(AssignFTPBoxAction), Array)

23 /var/www/vendor/yiisoft/yii/framework/web/CWebApplication.php(282): CController->run('assignFTPBox')

24 /var/www/vendor/yiisoft/yii/framework/web/CWebApplication.php(141): CWebApplication->runController('adminDataset/as...')

25 /var/www/vendor/yiisoft/yii/framework/base/CApplication.php(185): CWebApplication->processRequest()

26 /var/www/index.php(24): CApplication->run()

27 {main}

REQUEST_URI=/adminDataset/assignFTPBox/id/200070 HTTP_REFERER=http://gigadb.gigasciencejournal.com/adminDataset/admin


---

Both errors seem related to missing permissions / authorization

Logging out and logging in again does not produce any changes

Switching browsers (e.g. Firefox / Chrome) doesn't produce any change

### System Specifications:
- **Operating System**: Ubuntu 20.04.3 LTS
- **Docker version** 25.0.1, build 29cf629
- **Docker Compose version** v2.17.3
- **Node.js Version**: v19.9.0

---

## Update (1-feb):

- I completely uninstalled docker and docker compose from my computer and reinstalled using Docker Desktop as suggested by @kencho51 

Docker version 25.0.2, build 29cf629
Docker Compose version v2.24.3-desktop.1

I follow the same process described above (with the expection of using the `sudo` command, it is not needed as Docker Desktop handles permissions) but encounter a different issue:

![image](https://github.com/gigascience/gigadb-website/assets/143437854/51b728f0-75c6-4b2d-8714-6df02041dec3)

The `deployment-application-1` container shows the following logs:

2024-02-01 17:13:55 GuzzleHttp\Exception\ServerException: Server error: POST http://fuw-admin-api/filedrop-accounts resulted in a 500 Internal Server Error response: 2024-02-01 17:13:55 {"name":"Exception","message":"Connection refused","code":0,"type":"Http\Client\Socket\Exception\ConnectionException (truncated...) 2024-02-01 17:13:55 in /var/www/vendor/guzzlehttp/guzzle/src/Exception/RequestException.php:113 2024-02-01 17:13:55 Stack trace: 2024-02-01 17:13:55 #0 /var/www/vendor/guzzlehttp/guzzle/src/Middleware.php(65): GuzzleHttp\Exception\RequestException::create(Object(GuzzleHttp\Psr7\Request), Object(GuzzleHttp\Psr7\Response)) 2024-02-01 17:13:55 #1 /var/www/vendor/guzzlehttp/promises/src/Promise.php(204): GuzzleHttp\Middleware::GuzzleHttp{closure}(Object(GuzzleHttp\Psr7\Response)) 2024-02-01 17:13:55 #2 /var/www/vendor/guzzlehttp/promises/src/Promise.php(153): GuzzleHttp\Promise\Promise::callHandler(1, Object(GuzzleHttp\Psr7\Response), NULL) 2024-02-01 17:13:55 #3 /var/www/vendor/guzzlehttp/promises/src/TaskQueue.php(48): GuzzleHttp\Promise\Promise::GuzzleHttp\Promise{closure}() 2024-02-01 17:13:55 #4 /var/www/vendor/guzzlehttp/promises/src/Promise.php(248): GuzzleHttp\Promise\TaskQueue->run(true) 2024-02-01 17:13:55 #5 /var/www/vendor/guzzlehttp/promises/src/Promise.php(224): GuzzleHttp\Promise\Promise->invokeWaitFn() 2024-02-01 17:13:55 #6 /var/www/vendor/guzzlehttp/promises/src/Promise.php(269): GuzzleHttp\Promise\Promise->waitIfPending() 2024-02-01 17:13:55 #7 /var/www/vendor/guzzlehttp/promises/src/Promise.php(226): GuzzleHttp\Promise\Promise->invokeWaitList() 2024-02-01 17:13:55 #8 /var/www/vendor/guzzlehttp/promises/src/Promise.php(62): GuzzleHttp\Promise\Promise->waitIfPending() 2024-02-01 17:13:55 #9 /var/www/vendor/guzzlehttp/guzzle/src/Client.php(182): GuzzleHttp\Promise\Promise->wait() 2024-02-01 17:13:55 #10 /var/www/protected/components/FiledropService.php(98): GuzzleHttp\Client->request('POST', 'http://fuw-admi...', Array) 2024-02-01 17:13:55 #11 /var/www/protected/controllers/adminDataset/AssignFTPBoxAction.php(32): FiledropService->createAccount() 2024-02-01 17:13:55 #12 [internal function]: AssignFTPBoxAction->run('200070') 2024-02-01 17:13:55 #13 /var/www/vendor/yiisoft/yii/framework/web/actions/CAction.php(115): ReflectionMethod->invokeArgs(Object(AssignFTPBoxAction), Array) 2024-02-01 17:13:55 #14 /var/www/vendor/yiisoft/yii/framework/web/actions/CAction.php(74): CAction->runWithParamsInternal(Object(AssignFTPBoxAction), Object(ReflectionMethod), Array) 2024-02-01 17:13:55 #15 /var/www/vendor/yiisoft/yii/framework/web/CController.php(308): CAction->runWithParams(Array) 2024-02-01 17:13:55 #16 /var/www/vendor/yiisoft/yii/framework/web/filters/CFilterChain.php(134): CController->runAction(Object(AssignFTPBoxAction)) 2024-02-01 17:13:55 #17 /var/www/vendor/yiisoft/yii/framework/web/filters/CFilter.php(40): CFilterChain->run() 2024-02-01 17:13:55 #18 /var/www/vendor/yiisoft/yii/framework/web/CController.php(1148): CFilter->filter(Object(CFilterChain)) 2024-02-01 17:13:55 #19 /var/www/vendor/yiisoft/yii/framework/web/filters/CInlineFilter.php(58): CController->filterAccessControl(Object(CFilterChain)) 2024-02-01 17:13:55 #20 /var/www/vendor/yiisoft/yii/framework/web/filters/CFilterChain.php(131): CInlineFilter->filter(Object(CFilterChain)) 2024-02-01 17:13:55 #21 /var/www/vendor/yiisoft/yii/framework/web/CController.php(291): CFilterChain->run() 2024-02-01 17:13:55 #22 /var/www/vendor/yiisoft/yii/framework/web/CController.php(265): CController->runActionWithFilters(Object(AssignFTPBoxAction), Array) 2024-02-01 17:13:55 #23 /var/www/vendor/yiisoft/yii/framework/web/CWebApplication.php(282): CController->run('assignFTPBox') 2024-02-01 17:13:55 #24 /var/www/vendor/yiisoft/yii/framework/web/CWebApplication.php(141): CWebApplication->runController('adminDataset/as...') 2024-02-01 17:13:55 #25 /var/www/vendor/yiisoft/yii/framework/base/CApplication.php(185): CWebApplication->processRequest() 2024-02-01 17:13:55 #26 /var/www/index.php(24): CApplication->run() 2024-02-01 17:13:55 #27 {main} 2024-02-01 17:13:55 REQUEST_URI=/adminDataset/assignFTPBox/id/200070 2024-02-01 17:13:55 HTTP_REFERER=http://gigadb.gigasciencejournal.com/adminDataset/admin 2024-02-01 17:13:55 ---


The `fuw-admin-1` container shows these logs:

2024-02-05 13:29:22 grep: /etc/pure-ftpd/passwd/pureftpd.passwd: No such file or directory 2024-02-05 13:29:22 grep: /etc/pure-ftpd/passwd/pureftpd.passwd: No such file or directory 2024-02-05 13:29:22 2024-02-05 06:29:22 [172.16.238.4][2][7e4a7415d2115a7223de2df0d1428a35][warning][application] !! Doing nothing, already exists: private/200070/uploader_token.txt 2024-02-05 13:29:22 in /app/backend/models/FiledropAccount.php:240 2024-02-05 13:29:22 in /app/backend/models/FiledropAccount.php:372 2024-02-05 13:29:22 in /app/backend/models/FiledropAccount.php:410 2024-02-05 13:29:22 2024-02-05 06:29:22 [172.16.238.4][2][7e4a7415d2115a7223de2df0d1428a35][warning][application] !! Doing nothing, already exists: private/200070/downloader_token.txt 2024-02-05 13:29:22 in /app/backend/models/FiledropAccount.php:240 2024-02-05 13:29:22 in /app/backend/models/FiledropAccount.php:373 2024-02-05 13:29:22 in /app/backend/models/FiledropAccount.php:410 2024-02-05 13:29:22 2024-02-05 06:29:22 [172.16.238.4][2][7e4a7415d2115a7223de2df0d1428a35][error][Http\Client\Socket\Exception\ConnectionException] Http\Client\Socket\Exception\ConnectionException: Connection refused in /app/vendor/php-http/socket-client/src/Client.php:114 2024-02-05 13:29:22 Stack trace: 2024-02-05 13:29:22 #0 /app/vendor/php-http/socket-client/src/Client.php(82): Http\Client\Socket\Client->createSocket(Object(GuzzleHttp\Psr7\Request), 'tcp://host.dock...', false) 2024-02-05 13:29:22 #1 /app/vendor/php-http/client-common/src/HttpClientDecorator.php(28): Http\Client\Socket\Client->sendRequest(Object(GuzzleHttp\Psr7\Request)) 2024-02-05 13:29:22 #2 /app/vendor/php-http/client-common/src/PluginClient.php(82): Http\Client\Common\EmulatedHttpAsyncClient->sendRequest(Object(GuzzleHttp\Psr7\Request)) 2024-02-05 13:29:22 #3 /app/vendor/php-http/client-common/src/Plugin/AddHostPlugin.php(64): Http\Client\Common\PluginClient->Http\Client\Common{closure}(Object(GuzzleHttp\Psr7\Request)) 2024-02-05 13:29:22 #4 /app/vendor/php-http/client-common/src/PluginClient.php(161): Http\Client\Common\Plugin\AddHostPlugin->handleRequest(Object(GuzzleHttp\Psr7\Request), Object(Closure), Object(Closure)) 2024-02-05 13:29:22 #5 /app/vendor/php-http/client-common/src/Plugin/DecoderPlugin.php(61): Http\Client\Common\PluginClient->Http\Client\Common{closure}(Object(GuzzleHttp\Psr7\Request)) 2024-02-05 13:29:22 #6 /app/vendor/php-http/client-common/src/PluginClient.php(161): Http\Client\Common\Plugin\DecoderPlugin->handleRequest(Object(GuzzleHttp\Psr7\Request), Object(Closure), Object(Closure)) 2024-02-05 13:29:22 #7 /app/vendor/php-http/client-common/src/Plugin/ContentLengthPlugin.php(34): Http\Client\Common\PluginClient->Http\Client\Common{closure}(Object(GuzzleHttp\Psr7\Request)) 2024-02-05 13:29:22 #8 /app/vendor/php-http/client-common/src/PluginClient.php(161): Http\Client\Common\Plugin\ContentLengthPlugin->handleRequest(Object(GuzzleHttp\Psr7\Request), Object(Closure), Object(Closure)) 2024-02-05 13:29:22 #9 /app/vendor/php-http/client-common/src/PluginClient.php(175): Http\Client\Common\PluginClient->Http\Client\Common{closure}(Object(GuzzleHttp\Psr7\Request)) 2024-02-05 13:29:22 #10 /app/vendor/php-http/client-common/src/PluginClient.php(88): Http\Client\Common\PluginClient->Http\Client\Common{closure}(Object(GuzzleHttp\Psr7\Request)) 2024-02-05 13:29:22 #11 /app/vendor/jane-php/open-api-runtime/Client/Psr7HttplugClient.php(48): Http\Client\Common\PluginClient->sendRequest(Object(GuzzleHttp\Psr7\Request)) 2024-02-05 13:29:22 #12 /app/vendor/docker-php/docker-php-api/src/Client.php(39): Jane\OpenApiRuntime\Client\Psr7HttplugClient->executePsr7Endpoint(Object(Docker\API\Endpoint\ContainerList), 'object') 2024-02-05 13:29:22 #13 /app/backend/models/DockerManager.php(110): Docker\API\Client->containerList() 2024-02-05 13:29:22 #14 /app/backend/models/DockerManager.php(162): backend\models\DockerManager->getContainer('/ftpd-1/') 2024-02-05 13:29:22 #15 /app/backend/models/FiledropAccount.php(277): backend\models\DockerManager->loadAndRunCommand('ftpd', Array) 2024-02-05 13:29:22 #16 /app/backend/models/FiledropAccount.php(412): backend\models\FiledropAccount->createFTPAccount(Object(backend\models\DockerManager), '200070') 2024-02-05 13:29:22 #17 /app/vendor/yiisoft/yii2/base/Model.php(351): backend\models\FiledropAccount->beforeValidate() 2024-02-05 13:29:22 #18 /app/vendor/yiisoft/yii2/db/ActiveRecord.php(564): yii\base\Model->validate(NULL) 2024-02-05 13:29:22 #19 /app/vendor/yiisoft/yii2/db/BaseActiveRecord.php(676): yii\db\ActiveRecord->insert(true, NULL) 2024-02-05 13:29:22 #20 /app/backend/actions/FiledropAccountController/CreateAction.php(58): yii\db\BaseActiveRecord->save() 2024-02-05 13:29:22 #21 [internal function]: backend\actions\FiledropAccountController\CreateAction->run() 2024-02-05 13:29:22 #22 /app/vendor/yiisoft/yii2/base/Action.php(93): call_user_func_array(Array, Array) 2024-02-05 13:29:22 #23 /app/vendor/yiisoft/yii2/base/Controller.php(178): yii\base\Action->runWithParams(Array) 2024-02-05 13:29:22 #24 /app/vendor/yiisoft/yii2/base/Module.php(552): yii\base\Controller->runAction('create', Array) 2024-02-05 13:29:22 #25 /app/vendor/yiisoft/yii2/web/Application.php(103): yii\base\Module->runAction('filedrop-accoun...', Array) 2024-02-05 13:29:22 #26 /app/vendor/yiisoft/yii2/base/Application.php(384): yii\web\Application->handleRequest(Object(yii\web\Request)) 2024-02-05 13:29:22 #27 /app/backend/web/index.php(17): yii\base\Application->run() 2024-02-05 13:29:22 #28 {main} 2024-02-05 13:29:22 172.16.238.10 - 05/Feb/2024:06:29:22 +0000 "POST /index.php" 500


After switching to Docker Desktop, running `docker-compose run --rm test ./vendor/codeception/codeception/codecept run --debug unit DatasetTest` from integration branch (not develop) shows one test erroring out

Codeception PHP Testing Framework v3.1.3 Powered by PHPUnit 5.7.27 by Sebastian Bergmann and contributors. Running with seed:

Unit Tests (4) ---------------------------------------------------------------------------------------------------- Modules: Asserts, \Helper\Unit

✔ DatasetTest: Get authors (0.50s) ✔ DatasetTest: Get author names (0.11s) ✔ DatasetTest: Get curator name (0.11s) E DatasetTest: Get available status list (0.53s)

Time: 1.35 seconds, Memory: 18.00MB

There was 1 error:

1) DatasetTest: Get available status list Test tests/unit/DatasetTest.php:testGetAvailableStatusList

[Cache\Adapter\Common\Exception\CachePoolException] Exception thrown when executing "save".

1 /var/www/vendor/cache/adapter-common/AbstractCachePool.php:338

2 /var/www/vendor/cache/adapter-common/AbstractCachePool.php:242

3 /var/www/vendor/cache/adapter-common/AbstractCachePool.php:446

4 /var/www/vendor/unleash/client/src/Client/DefaultRegistrationService.php:86

5 /var/www/vendor/unleash/client/src/Client/DefaultRegistrationService.php:69

6 /var/www/vendor/unleash/client/src/DefaultUnleash.php:120

7 /var/www/vendor/unleash/client/src/DefaultUnleash.php:53

8 /var/www/vendor/unleash/client/src/UnleashBuilder.php:354

9 /var/www/protected/components/FeatureFlagService.php:71

10 /var/www/vendor/yiisoft/yii/framework/base/CModule.php:394

ERRORS! Tests: 4, Assertions: 4, Errors: 1.


Running `./tests/functional_runner` from integration branch (not from develop) I see 6 failing tests that seem related to fuw and all look like the same issue:

E FiledropAccountCest: Checking an ftp account (0.03s) grep: /etc/pure-ftpd/passwd/pureftpd.passwd: No such file or directory grep: /etc/pure-ftpd/passwd/pureftpd.passwd: No such file or directory E FiledropAccountCest: Create ftp accounts (0.01s) grep: /etc/pure-ftpd/passwd/pureftpd.passwd: No such file or directory grep: /etc/pure-ftpd/passwd/pureftpd.passwd: No such file or directory E FiledropAccountCest: Create accounts database record (0.03s) grep: /etc/pure-ftpd/passwd/pureftpd.passwd: No such file or directory grep: /etc/pure-ftpd/passwd/pureftpd.passwd: No such file or directory E FiledropAccountCest: Send rest http post to create account (0.05s) grep: /etc/pure-ftpd/passwd/pureftpd.passwd: No such file or directory grep: /etc/pure-ftpd/passwd/pureftpd.passwd: No such file or directory E FiledropAccountCest: Send rest http delete to delete account (0.03s) grep: /etc/pure-ftpd/passwd/pureftpd.passwd: No such file or directory grep: /etc/pure-ftpd/passwd/pureftpd.passwd: No such file or directory E FiledropAccountCest: Send rest http put to update filedrop account and send email (0.02s)


As far as I know other team members do not see these errors

---

## 5-Feb

After following steps and running `./up.sh` (develop branch) I see this error, which I have never seen before:

ERROR: database "gigadb" already exists

Yii Migration Tool v1.0 (based on Yii v1.1.28)

Already at '300000_000000'. Nothing needs to be done.

Yii Migration Tool v1.0 (based on Yii v1.1.28)

Error: Unable to find the version '000000_000000'.

rija commented 8 months ago

Hi @luistoptal,

looking into the issues you are encountering is my top priority this week. Hopefully by tomorrow or Wednesday, I will be able to shed light on what's going on and provide a path forward.

luistoptal commented 8 months ago

Hi @rija thanks for looking into this. Please let me know if you need anything from me, e.g. screen sharing or anything else

rija commented 8 months ago

Hi @luistoptal,

I have replicated the 6 functional test failures and the error message

  [Http\Client\Socket\Exception\ConnectionException] php_network_getaddresses: getaddrinfo failed: Name or service not known

The code being tested that needs to manage the ftp account for the drop box, needs access to the docker daemon to send instructions to the ftpd container. It tries to access it using tcp://host.docker.internal:2375 which is a special hostname that only works on macOS.

The fix is easy and consists of making the backend use "unix:///var/run/docker.sock" as hostname to access the docker daemon and which should work on any platform. The fix should also prevent the error to happen when using the website (not just the tests).

The file to modify is ops/deployment/docker-compose.yml, and I've attached a fixed file to this comment. I'll eventually make a PR for it, but I'll wait until I'm confident you are not blocked on other things first.

docker-compose.yml.zip

luistoptal commented 8 months ago

@rija when running ./up.sh I keep seeing these errors:

+ ./ops/scripts/setup_devdb.sh dev
ERROR:  database "gigadb" already exists

Yii Migration Tool v1.0 (based on Yii v1.1.28)

Already at '300000_000000'. Nothing needs to be done.

Yii Migration Tool v1.0 (based on Yii v1.1.28)

Error: Unable to find the version '000000_000000'.

These are the logs of the application container when I try to access the app

2024-02-07 11:50:17 CDbException: The table "dataset" for active record class "Dataset" cannot be found in the database. in /var/www/vendor/yiisoft/yii/framework/db/ar/CActiveRecord.php:2391
2024-02-07 11:50:17 Stack trace:
2024-02-07 11:50:17 #0 /var/www/vendor/yiisoft/yii/framework/db/ar/CActiveRecord.php(413): CActiveRecordMetaData->__construct(Object(Dataset))
2024-02-07 11:50:17 #1 /var/www/vendor/yiisoft/yii/framework/db/ar/CActiveRecord.php(79): CActiveRecord->getMetaData()
2024-02-07 11:50:17 #2 /var/www/protected/controllers/SiteController.php(90): CActiveRecord->__construct()
2024-02-07 11:50:17 #3 /var/www/vendor/yiisoft/yii/framework/web/actions/CInlineAction.php(49): SiteController->actionIndex()
2024-02-07 11:50:17 #4 /var/www/vendor/yiisoft/yii/framework/web/CController.php(308): CInlineAction->runWithParams(Array)
2024-02-07 11:50:17 #5 /var/www/vendor/yiisoft/yii/framework/web/filters/CFilterChain.php(134): CController->runAction(Object(CInlineAction))
2024-02-07 11:50:17 #6 /var/www/vendor/yiisoft/yii/framework/web/filters/CFilter.php(40): CFilterChain->run()
2024-02-07 11:50:17 #7 /var/www/vendor/yiisoft/yii/framework/web/CController.php(1148): CFilter->filter(Object(CFilterChain))
2024-02-07 11:50:17 #8 /var/www/vendor/yiisoft/yii/framework/web/filters/CInlineFilter.php(58): CController->filterAccessControl(Object(CFilterChain))
2024-02-07 11:50:17 #9 /var/www/vendor/yiisoft/yii/framework/web/filters/CFilterChain.php(131): CInlineFilter->filter(Object(CFilterChain))
2024-02-07 11:50:17 #10 /var/www/vendor/yiisoft/yii/framework/web/CController.php(291): CFilterChain->run()
2024-02-07 11:50:17 #11 /var/www/vendor/yiisoft/yii/framework/web/CController.php(265): CController->runActionWithFilters(Object(CInlineAction), Array)
2024-02-07 11:50:17 #12 /var/www/vendor/yiisoft/yii/framework/web/CWebApplication.php(282): CController->run('')
2024-02-07 11:50:17 #13 /var/www/vendor/yiisoft/yii/framework/web/CWebApplication.php(141): CWebApplication->runController('')
2024-02-07 11:50:17 #14 /var/www/vendor/yiisoft/yii/framework/base/CApplication.php(185): CWebApplication->processRequest()
2024-02-07 11:50:17 #15 /var/www/index.php(24): CApplication->run()
2024-02-07 11:50:17 #16 {main}
2024-02-07 11:50:17 REQUEST_URI=/
2024-02-07 11:50:17 ---

image

Looks like something wrong with the DB

I've tried deleting all containers and images and re-run the script but that doesn't change anything

I've never seen this error before 5th February

After installing Docker Desktop I was able to locally deploy a few times so it must be something else, I guess it must be related to my local setup, but after cleanup the docker containers I wonder what else I can try

rija commented 8 months ago

Hi @luistoptal,

I'm not seeing this error on my ubuntu setup.

It's clearly a problem with the current state of the database

couple of things to suggest:

1) When shutting down the whole application, always use:

$ docker-compose down -v --remove-orphans

esp. the -v option is important as it will destroy all the volumes.

2) After ensuring the application is shut down as above, you can delete the PostgreSQL data directory. First identify where the application save generic runtime data

root@docker-ubuntu-s-1vcpu-1gb-sfo3-01:~/code/gigadb-website# cat .env | grep DATA
DATA_SAVE_PATH=~/.containers-data/default-gigadb

inside, you should see a postgres directory:

root@docker-ubuntu-s-1vcpu-1gb-sfo3-01:~/code/gigadb-website# ls -alrt ~/.containers-data/default-gigadb
total 20
drwxr-xr-x 3 root root 4096 Feb  5 14:51 ..
drwxr-xr-x 6 root root 4096 Feb  5 15:37 fuw
drwxr-xr-x 3 root root 4096 Feb  5 15:37 postgres
drwxr-xr-x 5 root root 4096 Feb  5 15:37 .
drwxr-xr-x 3 root root 4096 Feb  6 16:49 pure-ftpd

inside that directory there should a version number of PostgreSQL currently in use:

root@docker-ubuntu-s-1vcpu-1gb-sfo3-01:~/code/gigadb-website# ls -alrt ~/.containers-data/default-gigadb/postgres/
total 12
drwxr-xr-x 3 root root 4096 Feb  5 15:37 .
drwxr-xr-x 3 root root 4096 Feb  5 15:37 14.8
drwxr-xr-x 5 root root 4096 Feb  5 15:37 ..

Delete that directory

root@docker-ubuntu-s-1vcpu-1gb-sfo3-01:~/code/gigadb-website# rm -rf ~/.containers-data/default-gigadb/postgres/14.8

then you can restart the application with .up.sh and it will recreate a brand new database, making your issue disappear.

luistoptal commented 8 months ago

@rija what you proposed solved my issue about the database

Now when I replicate the fuw workflow I am back to this issue (even after using the docker-compose file you provided)

image

Application container logs:

2024-02-07 18:50:02 GuzzleHttp\Exception\ServerException: Server error: `POST http://fuw-admin-api/filedrop-accounts` resulted in a `500 Internal Server Error` response:
2024-02-07 18:50:02 {"name":"Exception","message":"Permission denied","code":0,"type":"Http\\Client\\Socket\\Exception\\ConnectionException" (truncated...)
2024-02-07 18:50:02  in /var/www/vendor/guzzlehttp/guzzle/src/Exception/RequestException.php:113
2024-02-07 18:50:02 Stack trace:
2024-02-07 18:50:02 #0 /var/www/vendor/guzzlehttp/guzzle/src/Middleware.php(65): GuzzleHttp\Exception\RequestException::create(Object(GuzzleHttp\Psr7\Request), Object(GuzzleHttp\Psr7\Response))
2024-02-07 18:50:02 #1 /var/www/vendor/guzzlehttp/promises/src/Promise.php(204): GuzzleHttp\Middleware::GuzzleHttp\{closure}(Object(GuzzleHttp\Psr7\Response))
2024-02-07 18:50:02 #2 /var/www/vendor/guzzlehttp/promises/src/Promise.php(153): GuzzleHttp\Promise\Promise::callHandler(1, Object(GuzzleHttp\Psr7\Response), NULL)
2024-02-07 18:50:02 #3 /var/www/vendor/guzzlehttp/promises/src/TaskQueue.php(48): GuzzleHttp\Promise\Promise::GuzzleHttp\Promise\{closure}()
2024-02-07 18:50:02 #4 /var/www/vendor/guzzlehttp/promises/src/Promise.php(248): GuzzleHttp\Promise\TaskQueue->run(true)
2024-02-07 18:50:02 #5 /var/www/vendor/guzzlehttp/promises/src/Promise.php(224): GuzzleHttp\Promise\Promise->invokeWaitFn()
2024-02-07 18:50:02 #6 /var/www/vendor/guzzlehttp/promises/src/Promise.php(269): GuzzleHttp\Promise\Promise->waitIfPending()
2024-02-07 18:50:02 #7 /var/www/vendor/guzzlehttp/promises/src/Promise.php(226): GuzzleHttp\Promise\Promise->invokeWaitList()
2024-02-07 18:50:02 #8 /var/www/vendor/guzzlehttp/promises/src/Promise.php(62): GuzzleHttp\Promise\Promise->waitIfPending()
2024-02-07 18:50:02 #9 /var/www/vendor/guzzlehttp/guzzle/src/Client.php(182): GuzzleHttp\Promise\Promise->wait()
2024-02-07 18:50:02 #10 /var/www/protected/components/FiledropService.php(98): GuzzleHttp\Client->request('POST', 'http://fuw-admi...', Array)
2024-02-07 18:50:02 #11 /var/www/protected/controllers/adminDataset/AssignFTPBoxAction.php(32): FiledropService->createAccount()
2024-02-07 18:50:02 #12 [internal function]: AssignFTPBoxAction->run('200070')
2024-02-07 18:50:02 #13 /var/www/vendor/yiisoft/yii/framework/web/actions/CAction.php(115): ReflectionMethod->invokeArgs(Object(AssignFTPBoxAction), Array)
2024-02-07 18:50:02 #14 /var/www/vendor/yiisoft/yii/framework/web/actions/CAction.php(74): CAction->runWithParamsInternal(Object(AssignFTPBoxAction), Object(ReflectionMethod), Array)
2024-02-07 18:50:02 #15 /var/www/vendor/yiisoft/yii/framework/web/CController.php(308): CAction->runWithParams(Array)
2024-02-07 18:50:02 #16 /var/www/vendor/yiisoft/yii/framework/web/filters/CFilterChain.php(134): CController->runAction(Object(AssignFTPBoxAction))
2024-02-07 18:50:02 #17 /var/www/vendor/yiisoft/yii/framework/web/filters/CFilter.php(40): CFilterChain->run()
2024-02-07 18:50:02 #18 /var/www/vendor/yiisoft/yii/framework/web/CController.php(1148): CFilter->filter(Object(CFilterChain))
2024-02-07 18:50:02 #19 /var/www/vendor/yiisoft/yii/framework/web/filters/CInlineFilter.php(58): CController->filterAccessControl(Object(CFilterChain))
2024-02-07 18:50:02 #20 /var/www/vendor/yiisoft/yii/framework/web/filters/CFilterChain.php(131): CInlineFilter->filter(Object(CFilterChain))
2024-02-07 18:50:02 #21 /var/www/vendor/yiisoft/yii/framework/web/CController.php(291): CFilterChain->run()
2024-02-07 18:50:02 #22 /var/www/vendor/yiisoft/yii/framework/web/CController.php(265): CController->runActionWithFilters(Object(AssignFTPBoxAction), Array)
2024-02-07 18:50:02 #23 /var/www/vendor/yiisoft/yii/framework/web/CWebApplication.php(282): CController->run('assignFTPBox')
2024-02-07 18:50:02 #24 /var/www/vendor/yiisoft/yii/framework/web/CWebApplication.php(141): CWebApplication->runController('adminDataset/as...')
2024-02-07 18:50:02 #25 /var/www/vendor/yiisoft/yii/framework/base/CApplication.php(185): CWebApplication->processRequest()
2024-02-07 18:50:02 #26 /var/www/index.php(24): CApplication->run()
2024-02-07 18:50:02 #27 {main}
2024-02-07 18:50:02 REQUEST_URI=/adminDataset/assignFTPBox/id/200070
2024-02-07 18:50:02 HTTP_REFERER=http://gigadb.gigasciencejournal.com/adminDataset/admin
2024-02-07 18:50:02 ---

This happens when I click on the dropbox icon and I'm redirected to http://gigadb.gigasciencejournal.com/adminDataset/assignFTPBox/id/200070

rija commented 8 months ago

Hi @luistoptal

can you also paste the log output from the fuw-admin container after you've clicked the dropbox icon?

luistoptal commented 8 months ago

@rija here it is:

2024-02-07 22:36:04 grep: /etc/pure-ftpd/passwd/pureftpd.passwd: No such file or directory
2024-02-07 22:36:04 grep: /etc/pure-ftpd/passwd/pureftpd.passwd: No such file or directory
2024-02-07 22:36:04 2024-02-07 15:36:04 [172.16.238.5][2][6f831006a1f10ebc9c3c1443ed823951][warning][application] !! Doing nothing, already exists: private/200070/uploader_token.txt
2024-02-07 22:36:04     in /app/backend/models/FiledropAccount.php:240
2024-02-07 22:36:04     in /app/backend/models/FiledropAccount.php:372
2024-02-07 22:36:04     in /app/backend/models/FiledropAccount.php:410
2024-02-07 22:36:04 2024-02-07 15:36:04 [172.16.238.5][2][6f831006a1f10ebc9c3c1443ed823951][warning][application] !! Doing nothing, already exists: private/200070/downloader_token.txt
2024-02-07 22:36:04     in /app/backend/models/FiledropAccount.php:240
2024-02-07 22:36:04     in /app/backend/models/FiledropAccount.php:373
2024-02-07 22:36:04     in /app/backend/models/FiledropAccount.php:410
2024-02-07 22:36:04 2024-02-07 15:36:04 [172.16.238.5][2][6f831006a1f10ebc9c3c1443ed823951][error][Http\Client\Socket\Exception\ConnectionException] Http\Client\Socket\Exception\ConnectionException: Permission denied in /app/vendor/php-http/socket-client/src/Client.php:114
2024-02-07 22:36:04 Stack trace:
2024-02-07 22:36:04 #0 /app/vendor/php-http/socket-client/src/Client.php(82): Http\Client\Socket\Client->createSocket(Object(GuzzleHttp\Psr7\Request), 'unix:///var/run...', false)
2024-02-07 22:36:04 #1 /app/vendor/php-http/client-common/src/HttpClientDecorator.php(28): Http\Client\Socket\Client->sendRequest(Object(GuzzleHttp\Psr7\Request))
2024-02-07 22:36:04 #2 /app/vendor/php-http/client-common/src/PluginClient.php(82): Http\Client\Common\EmulatedHttpAsyncClient->sendRequest(Object(GuzzleHttp\Psr7\Request))
2024-02-07 22:36:04 #3 /app/vendor/php-http/client-common/src/Plugin/AddHostPlugin.php(64): Http\Client\Common\PluginClient->Http\Client\Common\{closure}(Object(GuzzleHttp\Psr7\Request))
2024-02-07 22:36:04 #4 /app/vendor/php-http/client-common/src/PluginClient.php(161): Http\Client\Common\Plugin\AddHostPlugin->handleRequest(Object(GuzzleHttp\Psr7\Request), Object(Closure), Object(Closure))
2024-02-07 22:36:04 #5 /app/vendor/php-http/client-common/src/Plugin/DecoderPlugin.php(61): Http\Client\Common\PluginClient->Http\Client\Common\{closure}(Object(GuzzleHttp\Psr7\Request))
2024-02-07 22:36:04 #6 /app/vendor/php-http/client-common/src/PluginClient.php(161): Http\Client\Common\Plugin\DecoderPlugin->handleRequest(Object(GuzzleHttp\Psr7\Request), Object(Closure), Object(Closure))
2024-02-07 22:36:04 #7 /app/vendor/php-http/client-common/src/Plugin/ContentLengthPlugin.php(34): Http\Client\Common\PluginClient->Http\Client\Common\{closure}(Object(GuzzleHttp\Psr7\Request))
2024-02-07 22:36:04 #8 /app/vendor/php-http/client-common/src/PluginClient.php(161): Http\Client\Common\Plugin\ContentLengthPlugin->handleRequest(Object(GuzzleHttp\Psr7\Request), Object(Closure), Object(Closure))
2024-02-07 22:36:04 #9 /app/vendor/php-http/client-common/src/PluginClient.php(175): Http\Client\Common\PluginClient->Http\Client\Common\{closure}(Object(GuzzleHttp\Psr7\Request))
2024-02-07 22:36:04 #10 /app/vendor/php-http/client-common/src/PluginClient.php(88): Http\Client\Common\PluginClient->Http\Client\Common\{closure}(Object(GuzzleHttp\Psr7\Request))
2024-02-07 22:36:04 #11 /app/vendor/jane-php/open-api-runtime/Client/Psr7HttplugClient.php(48): Http\Client\Common\PluginClient->sendRequest(Object(GuzzleHttp\Psr7\Request))
2024-02-07 22:36:04 #12 /app/vendor/docker-php/docker-php-api/src/Client.php(39): Jane\OpenApiRuntime\Client\Psr7HttplugClient->executePsr7Endpoint(Object(Docker\API\Endpoint\ContainerList), 'object')
2024-02-07 22:36:04 #13 /app/backend/models/DockerManager.php(110): Docker\API\Client->containerList()
2024-02-07 22:36:04 #14 /app/backend/models/DockerManager.php(162): backend\models\DockerManager->getContainer('/ftpd-1/')
2024-02-07 22:36:04 #15 /app/backend/models/FiledropAccount.php(277): backend\models\DockerManager->loadAndRunCommand('ftpd', Array)
2024-02-07 22:36:04 #16 /app/backend/models/FiledropAccount.php(412): backend\models\FiledropAccount->createFTPAccount(Object(backend\models\DockerManager), '200070')
2024-02-07 22:36:04 #17 /app/vendor/yiisoft/yii2/base/Model.php(351): backend\models\FiledropAccount->beforeValidate()
2024-02-07 22:36:04 #18 /app/vendor/yiisoft/yii2/db/ActiveRecord.php(564): yii\base\Model->validate(NULL)
2024-02-07 22:36:04 #19 /app/vendor/yiisoft/yii2/db/BaseActiveRecord.php(676): yii\db\ActiveRecord->insert(true, NULL)
2024-02-07 22:36:04 #20 /app/backend/actions/FiledropAccountController/CreateAction.php(58): yii\db\BaseActiveRecord->save()
2024-02-07 22:36:04 #21 [internal function]: backend\actions\FiledropAccountController\CreateAction->run()
2024-02-07 22:36:04 #22 /app/vendor/yiisoft/yii2/base/Action.php(93): call_user_func_array(Array, Array)
2024-02-07 22:36:04 #23 /app/vendor/yiisoft/yii2/base/Controller.php(178): yii\base\Action->runWithParams(Array)
2024-02-07 22:36:04 #24 /app/vendor/yiisoft/yii2/base/Module.php(552): yii\base\Controller->runAction('create', Array)
2024-02-07 22:36:04 #25 /app/vendor/yiisoft/yii2/web/Application.php(103): yii\base\Module->runAction('filedrop-accoun...', Array)
2024-02-07 22:36:04 #26 /app/vendor/yiisoft/yii2/base/Application.php(384): yii\web\Application->handleRequest(Object(yii\web\Request))
2024-02-07 22:36:04 #27 /app/backend/web/index.php(17): yii\base\Application->run()
2024-02-07 22:36:04 #28 {main}
2024-02-07 22:36:04 172.16.238.10 -  07/Feb/2024:15:36:04 +0000 "POST /index.php" 500
rija commented 8 months ago

@luistoptal

thanks, I've now replicated the exact issue on my side. I'll get back to you when I've figured out what's going on

rija commented 8 months ago

Hi @luistoptal

If you're docker daemon is not accessible to the outside world (that's the default with Docker Desktop normally), you can run 

$ chmod 777 /var/run/docker.sock

And it should get rid of that error you are seeing.

rija commented 8 months ago

Additionally, @luistoptal, you will want to run these commands too before executing ./up.sh:

$ chmod -R 777 fuw/app/console/runtime
$ chmod -R 777 fuw/app/frontend/runtime
$ chmod -R 777 fuw/app/backend/runtime
$ chmod -R 777 ~/.containers-data/default-gigadb/fuw

to avoid similar errors on other part of the workflow

luistoptal commented 8 months ago

@rija

I don't have a /var/run/docker.sock file in my system, however I do have $HOME/.docker/desktop/docker.sock, so I tried to give this one full permissions and I also tried to use that path instead of the /var/run in the docker-compose file

In both cases I still get the last error I reported.

Not having a /var/run/docker.sock file is probably not usual, I guess something related to how Docker desktop is configured (I didn't change any of the default settings)

Will do some more research tomorrow

rija commented 8 months ago

@luistoptal

I see, the important thing is what the container think is /var/run/docker.sock.

So if $HOME/.docker/desktop/docker.sock turns out to be the one used, you could replace in docker-compose.yml all (the two of them) the lines that says: 

- /var/run/docker.sock:/var/run/docker.sock:ro

with 

- $HOME/.docker/desktop/docker.sock:/var/run/docker.sock:ro

or 

- ~/.docker/desktop/docker.sock:/var/run/docker.sock:ro

and run the chmod command against $HOME/.docker/desktop/docker.sock

luistoptal commented 8 months ago

@rija These are the steps I'm following

I get the same error

image

fuw-admin container logs:

2024-02-08 11:02:22 grep: /etc/pure-ftpd/passwd/pureftpd.passwd: No such file or directory
2024-02-08 11:02:22 grep: /etc/pure-ftpd/passwd/pureftpd.passwd: No such file or directory
2024-02-08 11:02:22 2024-02-08 04:02:22 [172.16.238.6][2][86b091d498c38640dc77e1da0f49169b][warning][application] !! Doing nothing, already exists: private/200070/uploader_token.txt
2024-02-08 11:02:22     in /app/backend/models/FiledropAccount.php:240
2024-02-08 11:02:22     in /app/backend/models/FiledropAccount.php:372
2024-02-08 11:02:22     in /app/backend/models/FiledropAccount.php:410
2024-02-08 11:02:22 2024-02-08 04:02:22 [172.16.238.6][2][86b091d498c38640dc77e1da0f49169b][warning][application] !! Doing nothing, already exists: private/200070/downloader_token.txt
2024-02-08 11:02:22     in /app/backend/models/FiledropAccount.php:240
2024-02-08 11:02:22     in /app/backend/models/FiledropAccount.php:373
2024-02-08 11:02:22     in /app/backend/models/FiledropAccount.php:410
2024-02-08 11:02:22 2024-02-08 04:02:22 [172.16.238.6][2][86b091d498c38640dc77e1da0f49169b][error][Http\Client\Socket\Exception\ConnectionException] Http\Client\Socket\Exception\ConnectionException: Connection refused in /app/vendor/php-http/socket-client/src/Client.php:114
2024-02-08 11:02:22 Stack trace:
2024-02-08 11:02:22 #0 /app/vendor/php-http/socket-client/src/Client.php(82): Http\Client\Socket\Client->createSocket(Object(GuzzleHttp\Psr7\Request), 'unix:///var/run...', false)
2024-02-08 11:02:22 #1 /app/vendor/php-http/client-common/src/HttpClientDecorator.php(28): Http\Client\Socket\Client->sendRequest(Object(GuzzleHttp\Psr7\Request))
2024-02-08 11:02:22 #2 /app/vendor/php-http/client-common/src/PluginClient.php(82): Http\Client\Common\EmulatedHttpAsyncClient->sendRequest(Object(GuzzleHttp\Psr7\Request))
2024-02-08 11:02:22 #3 /app/vendor/php-http/client-common/src/Plugin/AddHostPlugin.php(64): Http\Client\Common\PluginClient->Http\Client\Common\{closure}(Object(GuzzleHttp\Psr7\Request))
2024-02-08 11:02:22 #4 /app/vendor/php-http/client-common/src/PluginClient.php(161): Http\Client\Common\Plugin\AddHostPlugin->handleRequest(Object(GuzzleHttp\Psr7\Request), Object(Closure), Object(Closure))
2024-02-08 11:02:22 #5 /app/vendor/php-http/client-common/src/Plugin/DecoderPlugin.php(61): Http\Client\Common\PluginClient->Http\Client\Common\{closure}(Object(GuzzleHttp\Psr7\Request))
2024-02-08 11:02:22 #6 /app/vendor/php-http/client-common/src/PluginClient.php(161): Http\Client\Common\Plugin\DecoderPlugin->handleRequest(Object(GuzzleHttp\Psr7\Request), Object(Closure), Object(Closure))
2024-02-08 11:02:22 #7 /app/vendor/php-http/client-common/src/Plugin/ContentLengthPlugin.php(34): Http\Client\Common\PluginClient->Http\Client\Common\{closure}(Object(GuzzleHttp\Psr7\Request))
2024-02-08 11:02:22 #8 /app/vendor/php-http/client-common/src/PluginClient.php(161): Http\Client\Common\Plugin\ContentLengthPlugin->handleRequest(Object(GuzzleHttp\Psr7\Request), Object(Closure), Object(Closure))
2024-02-08 11:02:22 #9 /app/vendor/php-http/client-common/src/PluginClient.php(175): Http\Client\Common\PluginClient->Http\Client\Common\{closure}(Object(GuzzleHttp\Psr7\Request))
2024-02-08 11:02:22 #10 /app/vendor/php-http/client-common/src/PluginClient.php(88): Http\Client\Common\PluginClient->Http\Client\Common\{closure}(Object(GuzzleHttp\Psr7\Request))
2024-02-08 11:02:22 #11 /app/vendor/jane-php/open-api-runtime/Client/Psr7HttplugClient.php(48): Http\Client\Common\PluginClient->sendRequest(Object(GuzzleHttp\Psr7\Request))
2024-02-08 11:02:22 #12 /app/vendor/docker-php/docker-php-api/src/Client.php(39): Jane\OpenApiRuntime\Client\Psr7HttplugClient->executePsr7Endpoint(Object(Docker\API\Endpoint\ContainerList), 'object')
2024-02-08 11:02:22 #13 /app/backend/models/DockerManager.php(110): Docker\API\Client->containerList()
2024-02-08 11:02:22 #14 /app/backend/models/DockerManager.php(162): backend\models\DockerManager->getContainer('/ftpd-1/')
2024-02-08 11:02:22 #15 /app/backend/models/FiledropAccount.php(277): backend\models\DockerManager->loadAndRunCommand('ftpd', Array)
2024-02-08 11:02:22 #16 /app/backend/models/FiledropAccount.php(412): backend\models\FiledropAccount->createFTPAccount(Object(backend\models\DockerManager), '200070')
2024-02-08 11:02:22 #17 /app/vendor/yiisoft/yii2/base/Model.php(351): backend\models\FiledropAccount->beforeValidate()
2024-02-08 11:02:22 #18 /app/vendor/yiisoft/yii2/db/ActiveRecord.php(564): yii\base\Model->validate(NULL)
2024-02-08 11:02:22 #19 /app/vendor/yiisoft/yii2/db/BaseActiveRecord.php(676): yii\db\ActiveRecord->insert(true, NULL)
2024-02-08 11:02:22 #20 /app/backend/actions/FiledropAccountController/CreateAction.php(58): yii\db\BaseActiveRecord->save()
2024-02-08 11:02:22 #21 [internal function]: backend\actions\FiledropAccountController\CreateAction->run()
2024-02-08 11:02:22 #22 /app/vendor/yiisoft/yii2/base/Action.php(93): call_user_func_array(Array, Array)
2024-02-08 11:02:22 #23 /app/vendor/yiisoft/yii2/base/Controller.php(178): yii\base\Action->runWithParams(Array)
2024-02-08 11:02:22 #24 /app/vendor/yiisoft/yii2/base/Module.php(552): yii\base\Controller->runAction('create', Array)
2024-02-08 11:02:22 #25 /app/vendor/yiisoft/yii2/web/Application.php(103): yii\base\Module->runAction('filedrop-accoun...', Array)
2024-02-08 11:02:22 #26 /app/vendor/yiisoft/yii2/base/Application.php(384): yii\web\Application->handleRequest(Object(yii\web\Request))
2024-02-08 11:02:22 #27 /app/backend/web/index.php(17): yii\base\Application->run()
2024-02-08 11:02:22 #28 {main}
2024-02-08 11:02:22 172.16.238.10 -  08/Feb/2024:04:02:22 +0000 "POST /index.php" 500

I see there are unix:///var/run/docker.sock in the docker-compose file, do I also need to udpate those? If yes, I already tried and getting same results

rija commented 8 months ago

I see there are unix:///var/run/docker.sock in the docker-compose file, do I also need to udpate those? If yes, I already tried and getting same results

Those must not change as they are used in-congtainer, and the container should only know about /var/run/docker.sock Which is why we do the mount binding

rija commented 8 months ago

@luistoptal But the error you are seeing has changed. Before it was “Permission denied”, now it says “Connection refused”

rija commented 8 months ago

Do you run as root on your setup?

luistoptal commented 8 months ago

@rija You are right, it's a different error

I got this error while changing these lines /var/run/docker.sock:/var/run/docker.sock:ro, but not unix:///var/run/docker.sock

No, I'm not running as root. With Docker Desktop I shouldn't need to.

When I run as root with sudo:

sudo ./up.sh                                
[sudo] password for luis: 
+ dbSet=dev
+ echo 'Starting all services...'
Starting all services...
++ uname
+ '[' Linux == Darwin ']'
+ '[' -f ./.env ']'
+ git describe --always
+ docker-compose run --rm config
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

this is because /var/run/docker.sock doesn't exist, I presume.

I might try to create a symlink

rija commented 8 months ago

As per our conference call, I have decided to disable the FTP functionality as it secondary and has no intersection with Luis' work. To do so I have disabled all the calls to docker-php functions in FiledropAccount.php by commenting calls to DockerManager.php wrapper methods.

The modifications to docker-compose.yml are no longer necessary. You should be unblocked now.

In parallel, I will work on a PR that completely remove the need for docker-php and for inter-container communication.

luistoptal commented 8 months ago

@rija I don't know if it matters or not for my work (probably not) but when I click on update a dataset as a user, I get an error image http://gigadb.gigasciencejournal.com/datasetSubmission/datasetManagement/id/668 image the delete button works

only1chunts commented 8 months ago

Thanks Luis, I've create a ticket from your discovery, we will discuss it at the next story refinement workshop. I'm not entirely sure how this button fits in with the new File Upload wizard, the tech team will advise on that.

kencho51 commented 7 months ago

Closed as fixed in PR #1688