search

whiskyechobravo / kerkoapp

A web application that provides a faceted search interface for bibliographies managed with Zotero.
https://whiskyechobravo.github.io/kerko/
BSD 3-Clause "New" or "Revised" License
62 stars 23 forks source link

docker install error. curl: (52) Empty reply from server #43

Closed hengxt closed 9 months ago

hengxt commented 9 months ago

### I use docker to install app. from step.1 to step.5, no problem image

but in step.6 -> Page not accessible

### error is : image image

How can I solve this problem?

davidlesieur commented 9 months ago

Not sure. I have just repeated the installation steps to ascertain that they still worked, and they did for me.

With the worker exiting like that, might be a problem in .secrets.toml or config.toml.

Could you try the following commands?

make shell

That should open an interactive shell within the Docker container. From there:

flask kerko config

Does the above command output KerkoApp's configuration?

flask --debug kerko sync

Does the above command output something like below?

root@623cf52ace5d:/kerkoapp# flask --debug kerko sync
 * Loading configuration file /kerkoapp/instance/config.toml
 * Loading configuration file /kerkoapp/instance/.secrets.toml
[2023-11-25 16:05:26,277] INFO in cache: Starting cache sync...
[2023-11-25 16:05:26,284] INFO in zotero: Requesting all item types...
[2023-11-25 16:05:26,439] INFO in zotero: Requesting all collections...
[2023-11-25 16:05:27,037] INFO in zotero: Requesting fields for items of type 'artwork'...
[2023-11-25 16:05:27,179] INFO in zotero: Requesting fields for items of type 'audioRecording'...
[2023-11-25 16:05:27,367] INFO in zotero: Requesting fields for items of type 'bill'...
[2023-11-25 16:05:27,511] INFO in zotero: Requesting fields for items of type 'blogPost'...
[2023-11-25 16:05:27,646] INFO in zotero: Requesting fields for items of type 'book'...
...
[2023-11-25 16:05:38,802] INFO in zotero: Requesting creator types for items of type 'thesis'...
[2023-11-25 16:05:38,954] INFO in zotero: Requesting creator types for items of type 'videoRecording'...
[2023-11-25 16:05:39,096] INFO in zotero: Requesting creator types for items of type 'webpage'...
[2023-11-25 16:05:39,472] INFO in zotero: Requesting updated text content since version 189...
[2023-11-25 16:05:39,638] INFO in zotero: Found 0 item(s) with updated text content.
[2023-11-25 16:05:39,640] INFO in zotero: Requesting up to 100 updated items since version 189, starting at position 0...
[2023-11-25 16:05:39,973] INFO in zotero: Requesting up to 100 trashed items since version 189, starting at position 0...
[2023-11-25 16:05:40,161] INFO in cache: Cache sync successful, now at version 189 (0 item(s) processed).
[2023-11-25 16:05:40,162] INFO in index: Starting index sync...
[2023-11-25 16:05:40,192] DEBUG in index: Item 1 updated (F8FUWSY5, conferencePaper): Design recommendations for hierarchical faceted search interfaces
[2023-11-25 16:05:40,195] DEBUG in index: Item 2 updated (FHICFQI2, journalArticle): Finding the flow in web site search
[2023-11-25 16:05:40,199] DEBUG in index: Item 3 updated (P8FA6F38, conferencePaper): Hierarchical Faceted Metadata in Site Search Interfaces
...
[2023-11-25 16:05:40,703] INFO in index: Index sync successful, now at version 189 (64 top level item(s) processed).
[2023-11-25 16:05:40,705] INFO in attachments: Starting attachment files sync...
[2023-11-25 16:05:40,719] DEBUG in attachments: Keeping attachment HGU6JNVV (parent item: TWDYFYPQ).
[2023-11-25 16:05:40,719] INFO in attachments: Attachment files sync successful (1 file(s) processed).
[2023-11-25 16:05:40,720] INFO in cli: Execution time: 14 seconds
hengxt commented 9 months ago

okey，Here is my command output

root@d84cef4b7276:/kerkoapp# flask --debug kerko sync
 * Loading configuration file /kerkoapp/instance/config.toml
 * Loading configuration file /kerkoapp/instance/.secrets.toml
[2023-11-27 09:11:09,719] INFO in cache: Starting cache sync...
[2023-11-27 09:11:09,735] INFO in zotero: Requesting all item types...
[2023-11-27 09:11:10,958] INFO in zotero: Requesting all collections...
[2023-11-27 09:11:12,917] INFO in zotero: Requesting fields for items of type 'artwork'...
[2023-11-27 09:11:14,018] INFO in zotero: Requesting fields for items of type 'audioRecording'...
[2023-11-27 09:11:14,734] INFO in zotero: Requesting fields for items of type 'bill'...
[2023-11-27 09:11:15,473] INFO in zotero: Requesting fields for items of type 'blogPost'...
[2023-11-27 09:11:16,205] INFO in zotero: Requesting fields for items of type 'book'...
[2023-11-27 09:11:16,969] INFO in zotero: Requesting fields for items of type 'bookSection'...
[2023-11-27 09:11:17,696] INFO in zotero: Requesting fields for items of type 'case'...
[2023-11-27 09:11:18,400] INFO in zotero: Requesting fields for items of type 'conferencePaper'...
[2023-11-27 09:11:19,106] INFO in zotero: Requesting fields for items of type 'dataset'...
[2023-11-27 09:11:19,865] INFO in zotero: Requesting fields for items of type 'dictionaryEntry'...
[2023-11-27 09:11:20,579] INFO in zotero: Requesting fields for items of type 'document'...

..........
[2023-11-27 09:12:55,917] INFO in cache: Cache sync successful, now at version 1199 (82 item(s) processed).
[2023-11-27 09:12:55,917] INFO in index: Starting index sync...
[2023-11-27 09:12:56,543] DEBUG in index: Item 1 updated (8M94YCII, journalArticle): DeepChain: Auditable and Privacy-Preserving Deep Learning with Blockchain-based Incentive
..........

[2023-11-27 09:13:07,907] DEBUG in attachments: Requesting attachment KGPL2QHM (parent item: FBL5KRZP)...
[2023-11-27 09:13:11,632] DEBUG in attachments: Requesting attachment XW6WD7XF (parent item: MS6C4RIY)...
[2023-11-27 09:13:14,989] DEBUG in attachments: Requesting attachment 6JRKAZHY (parent item: 9Z8A4EC3)...
[2023-11-27 09:13:18,061] DEBUG in attachments: Requesting attachment VN8G2E7G (parent item: 9Z8A4EC3)...
[2023-11-27 09:13:21,254] DEBUG in attachments: Requesting attachment FYXPZD7X (parent item: 29I3PDN2)...
[2023-11-27 09:13:24,276] DEBUG in attachments: Requesting attachment KVR6QH9D (parent item: QT3LZDX2)...
[2023-11-27 09:13:27,345] INFO in attachments: Attachment files sync successful (17 file(s) processed).
[2023-11-27 09:13:27,345] INFO in cli: Execution time: 2 minutes 18 seconds

But I still can't access the port in docker shell

root@d84cef4b7276:/kerkoapp# curl 127.0.0.1:80
curl: (7) Failed to connect to 127.0.0.1 port 80 after 0 ms: Couldn't connect to server

in server shell（centos）

[root@iZ2zecj51bg87sd5l4jbc1Z ~]# curl 127.0.0.1:8080
curl: (56) Recv failure: Connection reset by peer

the next is flask kerko config config.txt

davidlesieur commented 9 months ago

At least sync works, that's a good start.

The right port would be 8080, so no need to try 80. When you run docker ps, do you get something like below? (especially the ports)

CONTAINER ID   IMAGE                      COMMAND                  CREATED         STATUS         PORTS                                   
0acfa036613a   whiskyechobravo/kerkoapp   "gunicorn --threads …"   2 minutes ago   Up 2 minutes   0.0.0.0:8080->80/tcp, :::8080->80/tcp

Could it be that some other process or container is already using port 8080? Or that some system configuration is blocking 8080?

hengxt commented 9 months ago

this is output after docker ps image it look like only COMMAND is different

davidlesieur commented 9 months ago

The command doesn't look right. Could you make sure the container is stopped, then run the following command from the directory where your Makefile is located? (i.e., the directory that's parent to the instance directory created during the installation procedure)

docker run --name kerkoapp --rm -p 8080:80 -v ./instance:/kerkoapp/instance whiskyechobravo/kerkoapp
hengxt commented 9 months ago

There are still problems here image image

davidlesieur commented 9 months ago

It's those workers exiting with code 255... We need to find out why that happens!

Could you try the following?

make shell

And then, from within the container:

flask --debug run -h 0.0.0.0 -p 80

Hopefully this will output more info.

hengxt commented 9 months ago

Sure,It does provide more information. PermissionError: [Errno 1] Operation not permitted: '/usr/local/bin/python But to enter the container this way, I don't have any Permission

[root@iZ2zecj51bg87sd5l4jbc1Z app]# make shell
docker run -it --rm -p 8080:80 -v /root/zotero/app/instance:/kerkoapp/instance -v /tmp/kerkoapp-dev-log:/dev/log whiskyechobravo/kerkoapp bash
root@4cdba0bf15a7:/kerkoapp# flask --debug run -h 0.0.0.0 -p 80
 * Loading configuration file /kerkoapp/instance/config.toml
 * Loading configuration file /kerkoapp/instance/.secrets.toml
 * Debug mode: on
[2023-12-05 01:01:18,797] INFO in _internal: WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on all addresses (0.0.0.0)
 * Running on http://127.0.0.1:80
 * Running on http://172.17.0.3:80
[2023-12-05 01:01:18,797] INFO in _internal: Press CTRL+C to quit
[2023-12-05 01:01:18,798] INFO in _internal:  * Restarting with stat
Traceback (most recent call last):
  File "/usr/local/bin/flask", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flask/cli.py", line 1063, in main
    cli.main()
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/decorators.py", line 84, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flask/cli.py", line 923, in run_command
    run_simple(
  File "/usr/local/lib/python3.11/site-packages/werkzeug/serving.py", line 1097, in run_simple
    run_with_reloader(
  File "/usr/local/lib/python3.11/site-packages/werkzeug/_reloader.py", line 456, in run_with_reloader
    sys.exit(reloader.restart_with_reloader())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/werkzeug/_reloader.py", line 273, in restart_with_reloader
    exit_code = subprocess.call(args, env=new_environ, close_fds=False)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/subprocess.py", line 389, in call
    with Popen(*popenargs, **kwargs) as p:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/subprocess.py", line 1026, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/local/lib/python3.11/subprocess.py", line 1838, in _execute_child
    self._posix_spawn(args, executable, env, restore_signals,
  File "/usr/local/lib/python3.11/subprocess.py", line 1782, in _posix_spawn
    self.pid = os.posix_spawn(executable, args, env, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 1] Operation not permitted: '/usr/local/bin/python
root@4cdba0bf15a7:/# ls -la
ls: cannot access 'boot': Operation not permitted
ls: cannot access 'sys': Operation not permitted
ls: cannot access 'media': Operation not permitted
ls: cannot access '.': Operation not permitted
ls: cannot access 'home': Operation not permitted
......
ls: cannot access 'root': Operation not permitted
ls: cannot access 'kerkoapp': Operation not permitted
ls: cannot access '.dockerenv': Operation not permitted
total 0
davidlesieur commented 9 months ago

I suggest maybe rebuilding the KerkoApp image? 

make clean_image
make build

I rarely use CentOS, and have never run Docker under CentOS. Could your Docker install be missing some permissions?

Can you confirm that /root/zotero/app/instance exists on the host machine and contains subdirectories such as /root/zotero/app/instance/kerko/cache and /root/zotero/app/instance/kerko/index? (those normally exist once Kerko has synchronized from Zotero).

hengxt commented 9 months ago

I reinstalled it, and it's the same problem

[root@iZ2zecj51bg87sd5l4jbc1Z app]# make run
docker run --rm -p 8080:80 -v /root/zotero/app/instance:/kerkoapp/instance -v /tmp/kerkoapp-dev-log:/dev/log whiskyechobravo/kerkoapp
Unable to find image 'whiskyechobravo/kerkoapp:latest' locally
latest: Pulling from whiskyechobravo/kerkoapp
d52e4f012db1: Pull complete 
7dd206bea61f: Pull complete 
2320f9be4a9c: Pull complete 
6e5565e0ba8d: Pull complete 
d3797e13cc41: Pull complete 
70f90dfe001b: Pull complete 
bd75605de417: Pull complete 
3d0e1a4b14bc: Pull complete 
39519f6f9bd2: Pull complete 
76c06473969b: Pull complete 
91f5990eb977: Pull complete 
fe467dd31d99: Pull complete 
Digest: sha256:81e8116177a1a970e329280bac4026f1d19641d587cc14161bac6daec55eb396
Status: Downloaded newer image for whiskyechobravo/kerkoapp:latest
[2023-12-05 02:29:50 +0000] [1] [INFO] Starting gunicorn 21.2.0
[2023-12-05 02:29:50 +0000] [1] [INFO] Listening at: http://0.0.0.0:80 (1)
[2023-12-05 02:29:50 +0000] [1] [INFO] Using worker: gthread
[2023-12-05 02:29:50 +0000] [5] [INFO] Booting worker with pid: 5
[2023-12-05 02:35:21 +0000] [1] [ERROR] Worker (pid:5) exited with code 255
[2023-12-05 02:35:21 +0000] [1] [ERROR] Worker (pid:5) exited with code 255.

this is /root/zotero/app/instance/kerko/cache and ``/root/zotero/app/instance/kerko/index image

davidlesieur commented 9 months ago

It really looks like the container has insufficient permissions. Do the following commands help?

docker run --privileged --name kerkoapp --rm -p 8080:80 -v ./instance:/kerkoapp/instance whiskyechobravo/kerkoapp

or 

docker run --cap-add=ALL --name kerkoapp --rm -p 8080:80 -v ./instance:/kerkoapp/instance whiskyechobravo/kerkoapp

Reference: Runtime privilege and Linux capabilities (Docker docs)

hengxt commented 9 months ago

thanks!!!! This commands solved my problem!!🧡🧡. image image

davidlesieur commented 9 months ago

Phew, finally! :grin:

That solution is perfectly adequate for local/testing purposes, but if you intend to run the Docker container in production, I suggest that you find which privileges really need to be enabled with --cap-add, instead of using --privileged, because the latter gives full privileges to the Docker container.

In the meantime, enjoy! :grin: