Help needed for installation

[sistar2020]

I'm running Ubuntu 19.3. I tried to install matcher as below:

$ git clone https://github.com/Merck/matcher.git $ cd matcher $ git submodule init $ git submodule update $ docker-compose up ERROR: The Compose file './docker-compose.yml' is invalid because: Unsupported config option for services: 'database' Unsupported config option for networks: 'matcher-network'

What did I do wrong?

[hooveran]

Hi @sistar2020 , thanks for reporting this issue.

Could you please give it another try, starting with removing your current matcher repo, then repeating the commands that you pasted above?

In my most recent commit, 136e75d4e6fa6cbb1099d667f010aa0a306943b6, I added a version declaration to the docker-compose.yml file, which I expect to help make installation more robust.

If the same issue still persists after that, then please let me know your docker version ($ docker --version), so that I could then attempt to reproduce the issue prior to further troubleshooting.

For your reference, I have successfully installed and launched matcher, using code from the above commit to the main branch, with Docker version 20.10.18, build b40c2f6, on Ubuntu 20.04.2.

Thanks!

[zemanet2020]

Thank you very much for your prompt answer. My docker version is as below: Docker version 20.10.21, build 20.10.21-0ubuntu1~18.04.2

What I tried as follows:

$ git clone https://github.com/Merck/matcher.git $ cd matcher/ $ git submodule init $ git submodule update $ sudo systemctl start docker $ sudo docker-compose up

Then docker was running OK and I was able to connect to http://0.0.0.0:8000 on a web browser.

I drew a molecule in the editor, selected a part of molecule and pressed the "Set Variable Atoms". Then I selected the rest of the molecule and pressed the "Set Constant Atoms". When I clicked on the "Select Properties" button, I got a following message: No properties have been loaded to the database. Please use mmpdb loadprops to load properties, then reload the page.

Pressing the "Submit Query" button didn't do anything. Did I do something wrong?

[hooveran]

Did you see the below message in the docker-compose output, before navigating to localhost:8000 in your browser?

matcher-backend-1 | INFO: Uvicorn running on http://0.0.0.0:8001 (Press CTRL+C to quit)

As noted in the README Quick Start section, the installation is only complete when the above message is outputted. With the default installation data, It can take a minute, or a few minutes, for the installation to proceed, prior to observing the above output.

In that case, all you would need to do is wait for the installation to finish as above. Then, when navigating to localhost:8000, the "Select Properties" button will properly display properties, the submit button will work (assuming you have entered valid input), and the pre-installed example queries will work.

Additionally, if you prematurely navigated to localhost:8000 before observing that the installation finished, you will need to refresh the browser page after the installation finishes.

Please let me know if that doesn't work, and if so, let me know what output is observed in the docker-compose console for the matcher-backend container.

[sistar2020]

$ docker --version Docker version 20.10.21, build 20.10.21-0ubuntu1~18.04.2

$ sudo docker-compose up

Starting matcher_database_1 ...
Starting matcher_database_1 ... done
Starting matcher_backend_1 ...
Starting matcher_backend_1 ... done
Starting matcher_frontend_1 ...
Starting matcher_frontend_1 ... done
Attaching to matcher_database_1, matcher_backend_1, matcher_frontend_1
database_1  |
database_1  | PostgreSQL Database directory appears to contain a database; Skipping initialization
database_1  |
backend_1   | mmpdb build was done in a previous container startup, skipping...
database_1  | 2023-04-14 03:16:36.017 UTC [1] LOG:  starting PostgreSQL 13.10 (Debian 13.10-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by
 gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
database_1  | 2023-04-14 03:16:36.018 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
database_1  | 2023-04-14 03:16:36.018 UTC [1] LOG:  listening on IPv6 address "::", port 5432
database_1  | 2023-04-14 03:16:36.032 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
database_1  | 2023-04-14 03:16:36.050 UTC [28] LOG:  database system was interrupted; last known up at 2023-04-14 03:12:30 UTC
backend_1   | INFO:     Started server process [17]
backend_1   | INFO:uvicorn.error:Started server process [17]
backend_1   | INFO:     Waiting for application startup.
backend_1   | INFO:uvicorn.error:Waiting for application startup.
backend_1   | INFO:     Application startup complete.
backend_1   | INFO:uvicorn.error:Application startup complete.
backend_1   | INFO:     Uvicorn running on http://0.0.0.0:8001 (Press CTRL+C to quit)
backend_1   | INFO:uvicorn.error:Uvicorn running on http://0.0.0.0:8001 (Press CTRL+C to quit)
frontend_1  | [2023-04-14 03:16:37 +0000] [14] [INFO] Starting gunicorn 20.1.0
frontend_1  | [2023-04-14 03:16:37 +0000] [14] [INFO] Listening at: http://0.0.0.0:8000 (14)
frontend_1  | [2023-04-14 03:16:37 +0000] [14] [INFO] Using worker: gevent
frontend_1  | [2023-04-14 03:16:37 +0000] [15] [INFO] Booting worker with pid: 15
frontend_1  | [2023-04-14 03:16:37 +0000] [16] [INFO] Booting worker with pid: 16
database_1  | 2023-04-14 03:16:38.013 UTC [28] LOG:  database system was not properly shut down; automatic recovery in progress
database_1  | 2023-04-14 03:16:38.019 UTC [28] LOG:  redo starts at 0/16A5980
database_1  | 2023-04-14 03:16:38.261 UTC [28] LOG:  invalid record length at 0/4BB6700: wanted 24, got 0
database_1  | 2023-04-14 03:16:38.261 UTC [28] LOG:  redo done at 0/4BB66C8
database_1  | 2023-04-14 03:16:38.623 UTC [1] LOG:  database system is ready to accept connections

It seems that the database was not initialized. So I removed the volumes:

$ sudo docker-compose down --volumes

Then I re-built it:

$ sudo docker-compose up --build

Creating network "matcher_matcher-network" with the default driver
Building database
Step 1/3 : FROM postgres:13
 ---> ab3945c8cf71
Step 2/3 : RUN apt-get update -y && apt-get install postgresql-13-rdkit -y
 ---> Using cache
 ---> f1cc5a2c1923
Step 3/3 : COPY load-extensions.sh /docker-entrypoint-initdb.d/
 ---> Using cache
 ---> f1ef6c550f33
Successfully built f1ef6c550f33
Successfully tagged matcher_database:latest
Building backend
Step 1/11 : FROM condaforge/miniforge3
 ---> 7b1f59c35ea5
Step 2/11 : USER root
 ---> Using cache
 ---> f1eaa27cd50a
Step 3/11 : RUN apt-get update -y && apt-get install libxrender1 -y
 ---> Using cache
 ---> 16797dee5d91
Step 4/11 : WORKDIR /opt
 ---> Using cache
 ---> 487c254a84b2
Step 5/11 : COPY environment.yml requirements.txt ./
 ---> Using cache
 ---> f3961003bcfe
Step 6/11 : RUN conda env create -f /opt/environment.yml
 ---> Using cache
 ---> 21e288a61206
Step 7/11 : ADD mmpdb ./mmpdb
 ---> Using cache
 ---> af96da3f8858
Step 8/11 : ADD initialize_db ./initialize_db
 ---> Using cache
 ---> 649e3cc13074
Step 9/11 : COPY backend_api.py models.py search_algorithm.py smiles_syntax.py ss_select.py ./
 ---> Using cache
 ---> 25c08cc9c6ac
Step 10/11 : EXPOSE 8001
 ---> Using cache
 ---> 9dc670c4c522
Step 11/11 : ENTRYPOINT ["bash", "-c", "chmod +x ./initialize_db/quick_start.sh && ./initialize_db/quick_start.sh"]
 ---> Using cache
 ---> 3b79019121f3
Successfully built 3b79019121f3
Successfully tagged matcher_backend:latest
Building frontend
Step 1/11 : FROM condaforge/miniforge3
 ---> 7b1f59c35ea5
Step 2/11 : WORKDIR /opt
 ---> Using cache
 ---> 9bfcb025abce
Step 3/11 : COPY environment.yml requirements.txt ./
 ---> Using cache
 ---> c70fac3975b1
Step 4/11 : RUN conda env create -f /opt/environment.yml
 ---> Using cache
 ---> 084ac78bb0f8
Step 5/11 : COPY frontend_api.py matcher.html monkey.py pairplot.py ./
 ---> Using cache
 ---> 61005ecb0e2e
Step 6/11 : COPY examples/ ./examples/
 ---> Using cache
 ---> 6f446021780f
Step 7/11 : COPY css/ ./css/
 ---> Using cache
 ---> 0286a3cb1363
Step 8/11 : COPY js/* ./js/
 ---> Using cache
 ---> 0627312183b0
Step 9/11 : COPY ketcher/ ./ketcher/
 ---> Using cache
 ---> d84a653cc6d5
Step 10/11 : EXPOSE 8000
 ---> Using cache
 ---> 2e1bbbb2562b
Step 11/11 : ENTRYPOINT ["conda", "run", "--no-capture-output", "-n", "matcher-frontend", "gunicorn", "--worker-class", "gevent", "--workers
", "2", "--bind", "0.0.0.0:8000", "monkey:server"]
 ---> Using cache
 ---> 0e66c41f08b0
Successfully built 0e66c41f08b0
Successfully tagged matcher_frontend:latest
Creating matcher_database_1 ...
Creating matcher_database_1 ... done
Creating matcher_backend_1 ...
Creating matcher_backend_1 ... done
Creating matcher_frontend_1 ...
Creating matcher_frontend_1 ... done
Attaching to matcher_database_1, matcher_backend_1, matcher_frontend_1
database_1  | The files belonging to this database system will be owned by user "postgres".
database_1  | This user must also own the server process.
database_1  |
database_1  | The database cluster will be initialized with locale "en_US.utf8".
database_1  | The default database encoding has accordingly been set to "UTF8".
database_1  | The default text search configuration will be set to "english".
database_1  |
database_1  | Data page checksums are disabled.
database_1  |
database_1  | fixing permissions on existing directory /var/lib/postgresql/data ... ok
database_1  | creating subdirectories ... ok
database_1  | selecting dynamic shared memory implementation ... posix
database_1  | selecting default max_connections ... 100
database_1  | selecting default shared_buffers ... 128MB
database_1  | selecting default time zone ... Etc/UTC
database_1  | creating configuration files ... ok
database_1  | running bootstrap script ... ok
database_1  | performing post-bootstrap initialization ... ok
backend_1   | Preparing record 0[03:19:28] SMILES Parse Error: syntax error while parsing: SMILES
backend_1   | [03:19:28] SMILES Parse Error: Failed parsing SMILES 'SMILES' for input: 'SMILES'
database_1  | syncing data to disk ... ok
database_1  |
database_1  |
database_1  | Success. You can now start the database server using:
database_1  |
database_1  |     pg_ctl -D /var/lib/postgresql/data -l logfile start
database_1  |
database_1  | initdb: warning: enabling "trust" authentication for local connections
database_1  | You can change this by editing pg_hba.conf or using the option -A, or
database_1  | --auth-local and --auth-host, the next time you run initdb.
database_1  | waiting for server to start....2023-04-14 03:19:29.339 UTC [48] LOG:  starting PostgreSQL 13.10 (Debian 13.10-1.pgdg110+1) on
x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
frontend_1  | [2023-04-14 03:19:29 +0000] [15] [INFO] Starting gunicorn 20.1.0
frontend_1  | [2023-04-14 03:19:29 +0000] [15] [INFO] Listening at: http://0.0.0.0:8000 (15)
frontend_1  | [2023-04-14 03:19:29 +0000] [15] [INFO] Using worker: gevent
frontend_1  | [2023-04-14 03:19:29 +0000] [16] [INFO] Booting worker with pid: 16
database_1  | 2023-04-14 03:19:29.345 UTC [48] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
database_1  | 2023-04-14 03:19:29.365 UTC [49] LOG:  database system was shut down at 2023-04-14 03:19:27 UTC
database_1  | 2023-04-14 03:19:29.373 UTC [48] LOG:  database system is ready to accept connections
frontend_1  | [2023-04-14 03:19:29 +0000] [17] [INFO] Booting worker with pid: 17
database_1  |  done
database_1  | server started
database_1  | CREATE DATABASE
database_1  |
database_1  |
database_1  | /usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/load-extensions.sh
database_1  | CREATE EXTENSION
database_1  |   oid  | extname | extowner | extnamespace | extrelocatable | extversion | extconfig | extcondition
database_1  | -------+---------+----------+--------------+----------------+------------+-----------+--------------
database_1  |  13431 | plpgsql |       10 |           11 | f              | 1.0        |           |
database_1  |  16385 | rdkit   |       10 |         2200 | t              | 3.8        |           |
database_1  | (2 rows)
database_1  |
database_1  |
database_1  | 2023-04-14 03:19:30.040 UTC [48] LOG:  received fast shutdown request
database_1  | waiting for server to shut down....2023-04-14 03:19:30.046 UTC [48] LOG:  aborting any active transactions
database_1  | 2023-04-14 03:19:30.047 UTC [48] LOG:  background worker "logical replication launcher" (PID 55) exited with exit code 1
database_1  | 2023-04-14 03:19:30.047 UTC [50] LOG:  shutting down
database_1  | 2023-04-14 03:19:30.170 UTC [48] LOG:  database system is shut down
database_1  |  done
database_1  | server stopped
database_1  |
database_1  | PostgreSQL init process complete; ready for start up.
database_1  |
database_1  | 2023-04-14 03:19:30.269 UTC [1] LOG:  starting PostgreSQL 13.10 (Debian 13.10-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by
 gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
database_1  | 2023-04-14 03:19:30.270 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
database_1  | 2023-04-14 03:19:30.270 UTC [1] LOG:  listening on IPv6 address "::", port 5432
database_1  | 2023-04-14 03:19:30.282 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
database_1  | 2023-04-14 03:19:30.296 UTC [66] LOG:  database system was shut down at 2023-04-14 03:19:30 UTC
database_1  | 2023-04-14 03:19:30.305 UTC [1] LOG:  database system is ready to accept connections
Started indexing at time = 1681442383.3004518
backend_1   | Finished loading fragments, seconds taken = 0.14402246475219727
backend_1   | Starting to find pairs at 1681442383.4444742
backend_1   | Starting to write pairs at 1681442383.5421417
backend_1   | Starting postgres writing at 1681442385.3619423
backend_1   | 2258 rule_smiles took 0.013167142868041992 seconds to write
backend_1   | 3913 rules took 0.053819894790649414 seconds to write
backend_1   | 5646 efps took 0.03582453727722168 seconds to write
backend_1   | 28639 rule_envs took 0.4128587245941162 seconds to write
backend_1   | 1089 compounds took 0.012606620788574219 seconds to write
backend_1   | 2769 constant_smiles took 0.015533924102783203 seconds to write
backend_1   | 52092 pairs took 1.262298583984375 seconds to write
backend_1   | Matcher-specific postgres extension took (seconds): 8.497
backend_1   | Started loadprops_command at 2023-04-14 03:19:57
backend_1   | args
backend_1   | Namespace(quiet=False, databases=['public$postgres'], properties='./initialize_db/quick_props.csv', metadata='./initialize_db/
quick_metadata.csv', command=<function loadprops_command at 0x7f9011ed2ee0>, subparser=ArgumentParser(prog='mmpdb.py loadprops', usage=None,
 description=None, formatter_class=<class 'argparse.RawDescriptionHelpFormatter'>, conflict_handler='error', add_help=True))
backend_1   | Reconnecting in loadprops_command at: 2023-04-14 03:19:57
backend_1   | Using dataset: MMPs from './initialize_db/quick_structures.fragments'
backend_1   | Reading properties from './initialize_db/quick_props.csv'
backend_1   | WARNING: the identifier column in the properties file (column 1) has a header of 'CMPD_CHEMBLID'; should be 'id', 'ID', 'Name'
, or 'name'
Read 2 properties for 1089 compounds from './initialize_db/quick_props.csv'
Read metadata for 2 properties from './initialize_db/quick_metadata.csv'
backend_1   | Finished downloading compound_property rows at 2023-04-14 03:19:57
backend_1   | Starting compound_property writes for CYP3A4 at 2023-04-14 03:19:57
Imported 368 'CYP3A4' records (368 new, 0 updated).
backend_1   | Starting compound_property writes for hERG_pIC50 at 2023-04-14 03:19:57
Imported 749 'hERG_pIC50' records (749 new, 0 updated).
backend_1   | Finished compound_property writes at 2023-04-14 03:19:57
backend_1   | Starting reaggregate_properties at 2023-04-14 03:19:57
Finished aggregating statistics at 2023-04-14 03:19:57
Generated 29422 rule statistics (28639 rule environments, 2 properties)
Getting information about which rule statistics exist...Reconnecting in reaggregate_properties at: 2023-04-14 03:19:57
backend_1   | Started batch statistics writing at 2023-04-14 03:19:57
Finished batch statistics writing at 2023-04-14 03:19:58
Number of rule statistics added: 29422 updated: 0 deleted: 0
Commiting changed ...Exiting db.atomic and committing writes at 2023-04-14 03:19:58
backend_1   | Exited db.atomic at 2023-04-14 03:19:58
Loaded all properties and re-computed all rule statistics.
backend_1   | Started loadprops_command at 2023-04-14 03:19:57
backend_1   | Finished downloading compound_property rows at 2023-04-14 03:19:57
backend_1   | Starting compound_property writes for CYP3A4 at 2023-04-14 03:19:57
backend_1   | Starting compound_property writes for hERG_pIC50 at 2023-04-14 03:19:57
backend_1   | Finished compound_property writes at 2023-04-14 03:19:57
backend_1   | Starting reaggregate_properties at 2023-04-14 03:19:57
backend_1   | Finished aggregating statistics at 2023-04-14 03:19:57
backend_1   | Aggregate statistics computing took (in seconds): 0.6149516105651855
backend_1   | Started batch statistics writing at 2023-04-14 03:19:57
backend_1   | Finished batch statistics writing at 2023-04-14 03:19:58
backend_1   | Updating statistics took (in seconds): 1.1457304954528809
backend_1   | Deleting old statistics took (in seconds): 8.559226989746094e-05
backend_1   | Exiting db.atomic and committing writes at 2023-04-14 03:19:58
backend_1   | Exited db.atomic at 2023-04-14 03:19:58
database_1  | 2023-04-14 03:20:00.145 UTC [77] WARNING:  nonstandard use of escape in a string literal at character 119
database_1  | 2023-04-14 03:20:00.145 UTC [77] HINT:  Use the escape string syntax for escapes, e.g., E'\r\n'.
backend_1   | INFO:     Started server process [252]
backend_1   | INFO:uvicorn.error:Started server process [252]
backend_1   | INFO:     Waiting for application startup.
backend_1   | INFO:uvicorn.error:Waiting for application startup.
backend_1   | INFO:     Application startup complete.
backend_1   | INFO:uvicorn.error:Application startup complete.
backend_1   | INFO:     Uvicorn running on http://0.0.0.0:8001 (Press CTRL+C to quit)
backend_1   | INFO:uvicorn.error:Uvicorn running on http://0.0.0.0:8001 (Press CTRL+C to quit)

But it I got an error message as above.

[hooveran]

The output you pasted looks like expected behavior for a successful installation.

What happens when, after observing the last line of your output, you navigate to localhost:8000 in a fresh browser tab?

You can test whether matcher is working by clicking "Run Example Search" at the top of the page, then clicking on any of the example queries on the page that loads:

[zemanet2020]

OK. It worked fine this time. Thanks a lot.