SORMAS-Foundation / SORMAS-Project

SORMAS (Surveillance, Outbreak Response Management and Analysis System) is an early warning and management system to fight the spread of infectious diseases.
https://sormas.org
GNU General Public License v3.0
292 stars 142 forks source link

Server Installation and configuration #1876

Closed scottboyde closed 4 years ago

scottboyde commented 4 years ago

Hi

I have been asked by HSC in Northern Ireland to Built a system and install Sormas. I have deployed a ubuntu system and installed java, postgres and beleive that I have installed the Sormas without any issues.

I have attempted to configure Apache but I am not sure how to link apache and sormas to display the website. Any assistance is appreciated.

Thanks Scott

MateStrysewske commented 4 years ago

@HolgerReiseVSys Can you assist with this?

scottboyde commented 4 years ago

does the install directory need to be /root/deploy/sormas/$(date +%F) as when running the install it fails with access permission denied

could not change directory to "/root/deploy/sormas/2020-04-21": Permission denied setup.sql: No such file or directory

HolgerReiseVSys commented 4 years ago

I have noticed that in SERVER_SETUP.md#apache-web-server the ports of the ProxyPass and ProxyPassReverse directives do not match the default port base specified in server-setup.sh. It should be 6080 instead of 5080. Maybe this could already explain your issues.

For the install directory, this may happen if the postgres user is not allowed to accces the directory. Either you adjust the permissions or you coud try and replace su postgres -c "psql -p ${DB_PORT} -f setup.sql" with su postgres -c "psql -p ${DB_PORT} < setup.sql" in server-setup.sh.

scottboyde commented 4 years ago

“This email is covered by the disclaimer found at the end of the message.”


Hi

I noticed that myself and changed but cannot connect.

I am using a different location to install sormas and get to the database setup but get an error about the schema_version psql:setup.sql:14: NOTICE: relation "schema_version" does not exist, skipping

Thanks Scott

From: HolgerReiseVSys [mailto:notifications@github.com] Sent: 22 April 2020 02:31 To: hzi-braunschweig/SORMAS-Project Cc: Scott Boyde; Author Subject: Re: [hzi-braunschweig/SORMAS-Project] Server Installation and configuration (#1876)

I have noticed that in SERVER_SETUP.md#apache-web-serverhttps://github.com/hzi-braunschweig/SORMAS-Project/blob/v1.38.0/SERVER_SETUP.md#apache-web-server the ports of the ProxyPass and ProxyPassReverse directives do not match the default port base specified in server-setup.sh. It should be 6080 instead of 5080. Maybe this could already explain your issues.

For the install directory, this may happen if the postgres user is not allowed to accces the directory. Either you adjust the permissions or you coud try and replace su postgres -c "psql -p ${DB_PORT} -f setup.sql" with su postgres -c "psql -p ${DB_PORT} < setup.sql" in server-setup.sh.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/hzi-braunschweig/SORMAS-Project/issues/1876#issuecomment-617467465, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AGL2V3VXRAKQBMRNPFXL2BDRNZCFVANCNFSM4MMXYDEA.


“The information contained in this email and any attachments is confidential and intended solely for the attention and use of the named addressee(s). No confidentiality or privilege is waived or lost by any mistransmission. If you are not the intended recipient of this email, please inform the sender by return email and destroy all copies. Any views or opinions presented are solely those of the author and do not necessarily represent the views of HSCNI. The content of emails sent and received via the HSC network may be monitored for the purposes of ensuring compliance with HSC policies and procedures. While HSCNI takes precautions in scanning outgoing emails for computer viruses, no responsibility will be accepted by HSCNI in the event that the email is infected by a computer virus. Recipients are therefore encouraged to take their own precautions in relation to virus scanning. All emails held by HSCNI may be subject to public disclosure under the Freedom of Information Act 2000.”

scottboyde commented 4 years ago

I manually created the databases and ran the sql scripts from SORMAS-Project/sormas-backend/src/main/resources/sql/

I have installed SORMAS from a different directory

/software/deploy/sormas/$(date +%F)/server-setup.sh

It appears to have installed correctly and do not get any errors, however it says to run sormas-update.sh but I assume that this means server-update.sh

So I have also ran server-update.sh

Do I need to add docroot to Apache config to point to sormas location

HolgerReiseVSys commented 4 years ago

I guess the payara should be OK. You can check by calling curl http://127.0.0.1:6080/sormas-ui/. This should return a redirect to /sormas-ui/login. curl http://127.0.0.1:6080 should show if the app server is running at all.

The ProxyPass directive belongs directly in the VirtualHost. This is used instead of DocumentRoot, as we do not deliver static files, but proxy calls to the application server.

scottboyde commented 4 years ago

Hi

When curl I get the following

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>Payara Server 5.192 #badassfish- Error report</title><style type="text/css"><!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P{font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}HR {color : #525D76;}--></style> </head><body><h1>HTTP Status 404 - </h1><hr/><p><b>type</b> Status report </p><p><b>message </b></p><p><b>description </b>The requested resource is not available.</p><hr/><h3>Payara Server 5.192 #badassfish</h3></body></html>

HolgerReiseVSys commented 4 years ago

This probably just means that theserver is up, but the ear and the war have not yet been deployed.

scottboyde commented 4 years ago

Apologies but how do I deploy the ear and war, thanks.

scottboyde commented 4 years ago

I have ran through the install again and when running the server-update.sh I get to part that starts the payara-sormas service but it fails as port 6048 is already being used. Though I am not sure why

scottboyde commented 4 years ago

So going through this again and the service is not being stopped during the server-update script so this fails to complete. I have changed the command and have been able to get the script to continue but I get a lot of java errors now. Should I cose this and start up another ticket

aleshchik commented 4 years ago

@scottboyde Hi, did you manage to run the sormas locally and load the UI? I'm doing the same now and run into problems as well. Maybe you know some kind of discord channel for this purpose?

scottboyde commented 4 years ago

I am now able to run the server-update.sh and get further,

`[2020-04-22T18:17:26.069+0100] [Payara 5.192] [INFO] [] [org.hibernate.Version] [tid: _ThreadID=23 _ThreadName=payara-executor-service-scheduled-task] [timeMillis: 1587575846069] [levelValue: 800] [[ HHH000412: Hibernate Core {5.4.3.Final}]]

[2020-04-22T18:17:26.277+0100] [Payara 5.192] [INFO] [] [org.hibernate.annotations.common.Version] [tid: _ThreadID=23 _ThreadName=payara-executor-service-scheduled-task] [timeMillis: 1587575846277] [levelValue: 800] [[ HCANN000001: Hibernate Commons Annotations {5.1.0.Final}]]

[2020-04-22T18:17:26.872+0100] [Payara 5.192] [SEVERE] [] [javax.enterprise.system.core] [tid: _ThreadID=23 _ThreadName=payara-executor-service-scheduled-task] [timeMillis: 1587575846872] [levelValue: 1000] [[ Exception while preparing the app]]

[2020-04-22T18:17:26.873+0100] [Payara 5.192] [SEVERE] [NCLS-CORE-00026] [javax.enterprise.system.core] [tid: _ThreadID=23 _ThreadName=payara-executor-service-scheduled-task] [timeMillis: 1587575846873] [levelValue: 1000] [[ Exception during lifecycle processing java.lang.NoSuchMethodError: sun.security.ssl.Handshaker.setApplicationProtocols([Ljava/lang/String;)V at sun.security.ssl.SSLSocketImpl.initHandshaker(SSLSocketImpl.java:1350) at sun.security.ssl.SSLSocketImpl.doneConnect(SSLSocketImpl.java:708) at sun.security.ssl.SSLSocketImpl.(SSLSocketImpl.java:583) at sun.security.ssl.SSLSocketFactoryImpl.createSocket(SSLSocketFactoryImpl.java:110) at org.postgresql.ssl.WrappedFactory.createSocket(WrappedFactory.java:42) at org.postgresql.ssl.MakeSSL.convert(MakeSSL.java:36) at org.postgresql.core.v3.ConnectionFactoryImpl.enableSSL(ConnectionFactoryImpl.java:441) at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:94) at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:192) at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49) at org.postgresql.jdbc.PgConnection.(PgConnection.java:195) at org.postgresql.Driver.makeConnection(Driver.java:458) at org.postgresql.Driver.connect(Driver.java:260) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:247) at org.postgresql.ds.common.BaseDataSource.getConnection(BaseDataSource.java:97) at org.postgresql.ds.common.BaseDataSource.getConnection(BaseDataSource.java:82) at org.postgresql.ds.PGConnectionPoolDataSource.getPooledConnection(PGConnectionPoolDataSource.java:58) at com.sun.gjc.spi.CPManagedConnectionFactory.createManagedConnection(CPManagedConnectionFactory.java:136) at com.sun.enterprise.resource.allocator.LocalTxConnectorAllocator.createResource(LocalTxConnectorAllocator.java:87) at com.sun.enterprise.resource.pool.ConnectionPool.createSingleResource(ConnectionPool.java:920) at com.sun.enterprise.resource.pool.ConnectionPool.createResource(ConnectionPool.java:1209) at com.sun.enterprise.resource.pool.datastructure.RWLockDataStructure.addResource(RWLockDataStructure.java:98) at com.sun.enterprise.resource.pool.ConnectionPool.addResource(ConnectionPool.java:287) at com.sun.enterprise.resource.pool.ConnectionPool.createResourceAndAddToPool(ConnectionPool.java:1532) at com.sun.enterprise.resource.pool.ConnectionPool.createResources(ConnectionPool.java:957) at com.sun.enterprise.resource.pool.ConnectionPool.initPool(ConnectionPool.java:235) at com.sun.enterprise.resource.pool.ConnectionPool.internalGetResource(ConnectionPool.java:528) at com.sun.enterprise.resource.pool.ConnectionPool.getResource(ConnectionPool.java:386) at com.sun.enterprise.resource.pool.PoolManagerImpl.getResourceFromPool(PoolManagerImpl.java:244) at com.sun.enterprise.resource.pool.PoolManagerImpl.getResource(PoolManagerImpl.java:171) at com.sun.enterprise.connectors.ConnectionManagerImpl.getResource(ConnectionManagerImpl.java:360) at com.sun.enterprise.connectors.ConnectionManagerImpl.internalGetConnection(ConnectionManagerImpl.java:307) at com.sun.enterprise.connectors.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:196) at com.sun.enterprise.connectors.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:171) at com.sun.enterprise.connectors.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:166) at com.sun.gjc.spi.base.AbstractDataSource.getConnection(AbstractDataSource.java:113) at org.hibernate.engine.jdbc.connections.internal.DatasourceConnectionProviderImpl.getConnection(DatasourceConnectionProviderImpl.java:122) at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator$ConnectionProviderJdbcConnectionAccess.obtainConnection(JdbcEnvironmentInitiator.java:180) at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:68) at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:35) at org.hibernate.boot.registry.internal.StandardServiceRegistryImpl.initiateService(StandardServiceRegistryImpl.java:101) at org.hibernate.service.internal.AbstractServiceRegistryImpl.createService(AbstractServiceRegistryImpl.java:263) at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:237) at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:214) at org.hibernate.id.factory.internal.DefaultIdentifierGeneratorFactory.injectServices(DefaultIdentifierGeneratorFactory.java:152) at org.hibernate.service.internal.AbstractServiceRegistryImpl.injectDependencies(AbstractServiceRegistryImpl.java:286) at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:243) at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:214) at org.hibernate.boot.internal.InFlightMetadataCollectorImpl.(InFlightMetadataCollectorImpl.java:175) at org.hibernate.boot.model.process.spi.MetadataBuildingProcess.complete(MetadataBuildingProcess.java:118) at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.metadata(EntityManagerFactoryBuilderImpl.java:900) at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.build(EntityManagerFactoryBuilderImpl.java:931) at org.hibernate.jpa.HibernatePersistenceProvider.createContainerEntityManagerFactory(HibernatePersistenceProvider.java:141) at org.glassfish.persistence.jpa.PersistenceUnitLoader.loadPU(PersistenceUnitLoader.java:207) at org.glassfish.persistence.jpa.PersistenceUnitLoader.(PersistenceUnitLoader.java:114) at org.glassfish.persistence.jpa.JPADeployer$1.visitPUD(JPADeployer.java:225) at org.glassfish.persistence.jpa.JPADeployer$PersistenceUnitDescriptorIterator.iteratePUDs(JPADeployer.java:525) at org.glassfish.persistence.jpa.JPADeployer.createEMFs(JPADeployer.java:240) at org.glassfish.persistence.jpa.JPADeployer.prepare(JPADeployer.java:170) at com.sun.enterprise.v3.server.ApplicationLifecycle.prepareModule(ApplicationLifecycle.java:986) at org.glassfish.javaee.full.deployment.EarDeployer.prepareBundle(EarDeployer.java:309) at org.glassfish.javaee.full.deployment.EarDeployer.access$200(EarDeployer.java:88) at org.glassfish.javaee.full.deployment.EarDeployer$1.doBundle(EarDeployer.java:155) at org.glassfish.javaee.full.deployment.EarDeployer$1.doBundle(EarDeployer.java:152) at org.glassfish.javaee.full.deployment.EarDeployer.doOnBundles(EarDeployer.java:232) at org.glassfish.javaee.full.deployment.EarDeployer.doOnAllTypedBundles(EarDeployer.java:241) at org.glassfish.javaee.full.deployment.EarDeployer.doOnAllBundles(EarDeployer.java:267) at org.glassfish.javaee.full.deployment.EarDeployer.prepare(EarDeployer.java:152) at com.sun.enterprise.v3.server.ApplicationLifecycle.prepareModule(ApplicationLifecycle.java:986) at com.sun.enterprise.v3.server.ApplicationLifecycle.prepare(ApplicationLifecycle.java:432) at org.glassfish.deployment.admin.DeployCommand.execute(DeployCommand.java:540) at com.sun.enterprise.v3.admin.CommandRunnerImpl$2$1.run(CommandRunnerImpl.java:557) at com.sun.enterprise.v3.admin.CommandRunnerImpl$2$1.run(CommandRunnerImpl.java:553) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:360) at com.sun.enterprise.v3.admin.CommandRunnerImpl$2.execute(CommandRunnerImpl.java:552) at com.sun.enterprise.v3.admin.CommandRunnerImpl$3.run(CommandRunnerImpl.java:583) at com.sun.enterprise.v3.admin.CommandRunnerImpl$3.run(CommandRunnerImpl.java:575) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:360) at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:574) at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:1483) at com.sun.enterprise.v3.admin.CommandRunnerImpl.access$1300(CommandRunnerImpl.java:119) at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1865) at org.glassfish.deployment.autodeploy.AutoOperation.run(AutoOperation.java:149) at org.glassfish.deployment.autodeploy.AutoDeployer.deploy(AutoDeployer.java:532) at org.glassfish.deployment.autodeploy.AutoDeployer.deployAll(AutoDeployer.java:437) at org.glassfish.deployment.autodeploy.AutoDeployer.run(AutoDeployer.java:371) at org.glassfish.deployment.autodeploy.AutoDeployer.run(AutoDeployer.java:362) at org.glassfish.deployment.autodeploy.AutoDeployService$1.run(AutoDeployService.java:231) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) ]]

[2020-04-22T18:17:26.915+0100] [Payara 5.192] [WARNING] [NCLS-DEPLOYMENT-02036] [javax.enterprise.system.tools.deployment.autodeploy] [tid: _ThreadID=23 _ThreadName=payara-executor-service-scheduled-task] [timeMillis: 1587575846915] [levelValue: 900] [[ Error occurred during deployment: Exception while preparing the app. Please see server.log for more details.]]

[2020-04-22T18:17:26.916+0100] [Payara 5.192] [WARNING] [NCLS-DEPLOYMENT-02036] [javax.enterprise.system.tools.deployment.autodeploy] [tid: _ThreadID=23 _ThreadName=payara-executor-service-scheduled-task] [timeMillis: 1587575846916] [levelValue: 900] [[ [AutoDeploy] Autodeploy failed : /opt/domains/sormas/autodeploy/sormas-ear.ear.]]

[2020-04-22T18:17:26.917+0100] [Payara 5.192] [INFO] [NCLS-DEPLOYMENT-02027] [javax.enterprise.system.tools.deployment.autodeploy] [tid: _ThreadID=23 _ThreadName=payara-executor-service-scheduled-task] [timeMillis: 1587575846917] [levelValue: 800] [[ Selecting file /opt/domains/sormas/autodeploy/sormas-rest.war for autodeployment]]

`

HolgerReiseVSys commented 4 years ago

I guess you are connecting to a remote postgres via ssl? We have experienced a very similar problem on the development systems. This seems to be a bug in payara, it is not compatible with the current java 8. It helped to downgrade java 8 to 8u202 (Zulu 8.36.0.1), see #1765. Something like sudo apt-get install zulu-8=8.36.0.1 should do the job. Maybe sudo apt-mark hold zulu-8 is needed, too. Using java 11 may work too, but it may require the latest payara (201).

scottboyde commented 4 years ago

Thanks, I am now able to complete the installation and get a web page displayed. I have tried to login but the site doesnt do anything. It appears not to even attempt to login.

scottboyde commented 4 years ago

@scottboyde Hi, did you manage to run the sormas locally and load the UI? I'm doing the same now and run into problems as well. Maybe you know some kind of discord channel for this purpose? post a ticket on here with what stage you are at and any errors etc and the folks here will help out, mention me in it as well and I will comment as best I can to help.

HolgerReiseVSys commented 4 years ago

Thanks, I am now able to complete the installation and get a web page displayed. I have tried to login but the site doesnt do anything. It appears not to even attempt to login.

Does that mean that you see the login window and when you enter e.g 'admin' as username and 'sadmin' as password and then press 'LOG IN', nothing happens?

Have you accessed the server via http instead of https? Then the apache directive Header edit Set-Cookie "(?i)^((?:(?!;\s?Secure).)+)$" "$1;Secure" from the manual would cause the browser to ignore the session cookies.

Otherwise you may want to check the web console and network monitoring view of your browser for problems.

scottboyde commented 4 years ago

I am only accessing the site via http but once I commented out the following I am able to login ok

`

Header edit Set-Cookie "(?i)^((?:(?!;\s?HttpOnly).)+)$" "$1;HttpOnly"

Header edit Set-Cookie "(?i)^((?:(?!;\s?Secure).)+)$" "$1;Secure"

`

If I enable the ssl settings for the site I get requested url not found

HolgerReiseVSys commented 4 years ago

Looks like an issue with your apache config. apache2ctl -S may help you with debugging. There should be one NameVirtualHost for port 80 and one for 443.

scottboyde commented 4 years ago

This is my apache config

scottboyde commented 4 years ago

output from apache2ctl -S

AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1. Set the 'ServerName' directive globally to suppress this message VirtualHost configuration: :80 sormas.hscni.net (/etc/apache2/sites-enabled/sormas.hscni.net.conf:1) :443 sormas.hscni.net (/etc/apache2/sites-enabled/sormas.hscni.net.conf:44) ServerRoot: "/etc/apache2" Main DocumentRoot: "/var/www/html" Main ErrorLog: "/var/log/apache2/error.log" Mutex watchdog-callback: using_defaults Mutex rewrite-map: using_defaults Mutex ssl-stapling-refresh: using_defaults Mutex ssl-stapling: using_defaults Mutex proxy: using_defaults Mutex ssl-cache: using_defaults Mutex default: dir="/var/run/apache2/" mechanism=default PidFile: "/var/run/apache2/apache2.pid" Define: DUMP_VHOSTS Define: DUMP_RUN_CFG User: name="www-data" id=33 Group: name="www-data" id=33 root@d1lvsorm-web01:/etc/apache2#

HolgerReiseVSys commented 4 years ago

It looks like you have configured Port 80 for https. So https://sormas.hscni.net:80/sormas-ui/ might even work. Please have a look at https://github.com/hzi-braunschweig/SORMAS-Docker/blob/master/apache2/vhost.conf.sh This is the script that is used to create the apache config for Docker.

scottboyde commented 4 years ago

Doh, please accept my apologies its been a long time since I had this much envolvement with linux systems and had to dig deep. I beleive that thats it all runninng now.

PS in the script there is a mistake for the apache error log location

ErrorLog /usr/local/apache2/error.log

Is there any links for system administration, user documents. and Many Thanks

HolgerReiseVSys commented 4 years ago

Great to hear. Some information can be found e.g. at https://sormasorg.helmholtz-hzi.de/sormas-docs.html. Also, I would recommend to get in contact with the SORMAS team at the HZI.