Install Guide

[chcokm]

Is there an install guide for Calypso available? We have a OMOP instance at Colorado Children's Hospital and would like to see if we can install Calypso on top.

[chrisknoll]

Sadly, no. However, I may be able to assist you directly, and in the process create a guide. Could you give me some details as to your environment? What database is the OMOP instance hosted on, and what existing infrastructure do you have for hosting a web application? (Such as a windows server with IIS or a linux server with Apache?)

-Chris

[chcokm]

Hi Chris, OMOP is on an Oracle Database. We have a linux server with Apache available for web hosting.

Kevin Matthews | Research Informatics | ETL Programmer | Children’s Hospital Colorado 13123 East 16th Avenue, Box B321 | Aurora, CO 80045 | Phone: (720) 777-3207 | Fax: (720) 777-7276 | Kevin.Matthews@childrenscolorado.orgmailto:Kevin.Matthews@childrenscolorado.org Connect with Children's Hospital Colorado on Facebookhttp://www.facebook.com/thechildrenshospital and Twitterhttp://twitter.com/childrenscolo

From: Chris Knoll [mailto:notifications@github.com] Sent: Monday, February 22, 2016 5:50 PM To: OHDSI/Calypso Cc: Matthews, Kevin Subject: Re: [Calypso] Install Guide (#14)

Sadly, no. However, I may be able to assist you directly, and in the process create a guide. Could you give me some details as to your environment? What database is the OMOP instance hosted on, and what existing infrastructure do you have for hosting a web application? (Such as a windows server with IIS or a linux server with Apache?)

-Chris

—

Reply to this email directly or view it on GitHubhttps://github.com/OHDSI/Calypso/issues/14#issuecomment-187456410.

CONFIDENTIALITY NOTICE: This e-mail, including any attachments, is for the sole use of the intended recipient and may contain confidential and privileged information. If you are not an intended recipient, or the person responsible for delivering this message to an intended recipient, you are hereby notified that reading, copying, using or distributing this message is prohibited. If you are not an intended recipient, please contact the sender by reply email and destroy all copies of the original message from your computer system.

---

[chrisknoll]

@chcokm , Let's start with getting the webapi set up. I'm going to work on creating a wiki page on OHDSI/WebAPI that describes seting up the databases (I think I can provide instructions for mssql, postgresql and oracle), and then the maven instructions for configuring your own environment (database connection strings, db user/password, etc), and once that's set up, installing any of the OHDSI apps should be simple. I'll let you know when to check out the OHDSI/WebAPI wiki.

[chcokm]

Thanks Chris, we look forward to hearing back from you on this.

Kevin Matthews | Research Informatics | ETL Programmer | Children’s Hospital Colorado 13123 East 16th Avenue, Box B321 | Aurora, CO 80045 | Phone: (720) 777-3207 | Fax: (720) 777-7276 | Kevin.Matthews@childrenscolorado.orgmailto:Kevin.Matthews@childrenscolorado.org Connect with Children's Hospital Colorado on Facebookhttp://www.facebook.com/thechildrenshospital and Twitterhttp://twitter.com/childrenscolo

From: Chris Knoll [mailto:notifications@github.com] Sent: Wednesday, February 24, 2016 7:20 AM To: OHDSI/Calypso Cc: Matthews, Kevin Subject: Re: [Calypso] Install Guide (#14)

@chcokmhttps://github.com/chcokm , Let's start with getting the webapi set up. I'm going to work on creating a wiki page on OHDSI/WebAPI that describes seting up the databases (I think I can provide instructions for mssql, postgresql and oracle), and then the maven instructions for configuring your own environment (database connection strings, db user/password, etc), and once that's set up, installing any of the OHDSI apps should be simple. I'll let you know when to check out the OHDSI/WebAPI wiki.

—

Reply to this email directly or view it on GitHubhttps://github.com/OHDSI/Calypso/issues/14#issuecomment-188273755.

CONFIDENTIALITY NOTICE: This e-mail, including any attachments, is for the sole use of the intended recipient and may contain confidential and privileged information. If you are not an intended recipient, or the person responsible for delivering this message to an intended recipient, you are hereby notified that reading, copying, using or distributing this message is prohibited. If you are not an intended recipient, please contact the sender by reply email and destroy all copies of the original message from your computer system.

---

[chrisknoll]

@chcokm , I've published a first draft of installation instructions for WebAPI here: https://github.com/OHDSI/WebAPI/wiki/WebAPI-Installation-Guide

If you could try to go through those instructions and see if you have success, I'd appreciate it. Please note: Oracle is the db platform I am most unfamiliar with. I got a local XE instance running and had the tables created successfully, but I haven't done extensive testing on Oracle. I'd appreciate any insights you could provide for gaps in the documentation or any troubleshooting hints you used when figuring out problems during installation.

Only Oracle info is in this WIKI, but i'll be adding details for PostgreSQL and MsSQL soon. Once you are all set up on WebAPI we'll have an easy time setting up Calypso.

[chcokm]

Hi Chris, We have our servers installed and have downloaded the WebApi from Github and are now in the process of attempting to configure it. Where do we get the Settings.xml file? Is this something we have to create or do we locate it somewhere. We searched on the server but were not able to find it. I have a co-worker with some Linux experience but don't have much myself so I'm struggling with understanding some of the commands. Thanks.

[chrisknoll]

Hi, @chcokm , Right, the settings.xml file is something you produce yourself, but you should be able to take the example found in the guide (https://github.com/OHDSI/WebAPI/wiki/WebAPI-Installation-Guide) and copy paste that into a file. Just modify the elements in the XML to point to your own environment and save it as oracle_settings.xml or settings.xml (whichever name you choose will be used in the command for building the WebAPI via Maven).

mvn clean package -s /WebAPIConfig/settings.xml -P {profile id}

In your case, you will set the profile's id xml tag to 'webapi-oracle' as per the example and you'll refernce the value of this id in the {profile id} parameter of the mvn command.

In addition, you will also need to execute the maven commands to register the JDBC driver for oracle into the local maven repository (this is necessary for any proprietary JDBC drivers that we can not host in the git repository).

mvn install:install-file -Dfile=ojdbc.jar -DgroupId=ojdbc -DartifactId=ojdbc -Dversion=6.0.0 -Dpackaging=jar

Instructions of downloading and installing maven is in the setup guide, as well as the jdbc driver I believe you can find in the oracle guide.

Let me know if you have any problems!

-Chris

[chcokm]

Chris, We deployed the .war file and were able to connect to the database but we are running into an error. Does WebAPI need an oracle user that has privileges to create a schema? The error is;

Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flyway' defined in class path resource [org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]: Invocation of init method failed; nested exception is org.flywaydb.core.api.FlywayException: Unable to create schema "omop"

[chrisknoll]

@chcokm, You do need to have specific schemas set up (and in oracle-world, a user owns a schema so creating a schema is akin to creating a user).

Instructions on oracle DB specific setup can be found here: https://github.com/OHDSI/WebAPI/wiki/Oracle-DB-Setup-Guide

I do not work natively in oracle, so YMMV on these instructions. If you find any gaps in these instructions, please let me know!

-Chris

[chcokm]

Chris, We are still running into an issue. We've created the Ohdsi user with resource and connect privileges matching the Oracle DB set up guide but are still running into an error.

The error is "Caused by: java.sql.SQLSyntaxErrorException: ORA-01031: insufficient privileges" from "Unable to create schema "ohdsi" Caused by: org.flywaydb.core.api.FlywayException: Unable to create schema "ohdsi"

Can you point us or send us the scripts that the WebApi is actually running, we can't tell what is actually erroring out and it appears the user is set up right.

[chrisknoll]

The error you are receiving is that the database migration is attempting to create the schema 'ohdsi'. That's part of the migration tool, not a script that you execute. However, in the DB guide, part of the instructions are to create the ohdsi schema as a new user: https://github.com/OHDSI/WebAPI/wiki/Oracle-DB-Setup-Guide#creating-the-ohdsi-schema

If you are getting this error, then the user/schema 'ohdsi' is not set up (and the db migration will attempt to do this for you, but I don't recommend this, you should set up the users, roles, permissions and schemas in advance). Can you try to log in as the ohdsi user and try to create a table manually, it could be a case-sensitive issue where the user you created isn't cased the same as in your configuration (Ohdsi vs. ohdsi for example?)

I want to caution you about trying to execute the migration scripts manually because it will interfere with the automated migration, but if you want to see what's executing you can find the migration scripts in the folder: https://github.com/OHDSI/WebAPI/tree/master/src/main/resources/db/migration/oracle

This does work on the Oracle XE instance I tested this on, and it's clear from the above error message that the user set up as the flyway user is not granted permisisons to create the schema (but in this case, the flyway user should actually be the schema, so I'm not sure why a schema is needing to be created).

Can you tell me if there's any differences between your own environment and the setup described in the Oracle DB setup in the guide?

-Chris

[chcokm]

Chris, good news, it looks like we've got the WebApi working. There were two different spots for the OHDSI username and datasource and one of them was lowercase. After we fixed that it now appears to be working. We were not sure at first because we could not launch the page in Internet Explorer but other browsers work.

Not we are seeing; [{"sourceId":1,"sourceName":"My Cdm","sourceDialect":"oracle","sourceKey":"MY_CDM","daimons":[{"sourceDaimonId":1,"daimonType":"CDM","tableQualifier":"cdm","priority":"0"},{"sourceDaimonId":2,"daimonType":"Vocabulary","tableQualifier":"cdm","priority":"0"},{"sourceDaimonId":3,"daimonType":"Results","tableQualifier":"ohdsi","priority":"0"}]},{"sourceId":2,"sourceName":"Default vocabulary","sourceDialect":"oracle","sourceKey":"vocab","daimons":[{"sourceDaimonId":4,"daimonType":"Vocabulary","tableQualifier":"cdm","priority":"1"}]}]

At this point we can attempt to install Calypso? Do you have instructions available for that?

[chrisknoll]

Haven't gotten formal installation guide for Calypso, but it's fairly straight forward:

1. git clone the calypso repo. Place the contents of the repo (except for .git) into a directory called calpyso under the webroot of your websererver. If you are running on tomcat, that's under ht_docs. If you are hosting webapps on IIS, it's under wwwroot.
2. Edit the config.js file under /js It looks something like this.

define([], function () {
    var config = {};
    config.webAPIRoot = "http://api.ohdsi.org/WebAPI/";
    return config;
});

Note, this is our www.ohdsi.org/calypso configuration. You will change webAPIRoot to the URL of your hosted WebAPI. It should be the URL that you used to get the soruces list from your message above, all the way up to (and including the trailing slash) of the WebAPI/

Let me know if you have any problems.

-Chris

[chcokm]

Chris, The OHDSI user has access to the omop tables, we entered the source information in the source table, however when we launch the application it only comes up with the feasibility study list and help links only (and the help link does not work). One of our questions is with where we cloned the calypso code, there was no ht_docs directory in tomcat but so we put the code under webapps, is this correct?”

[chrisknoll]

Hi, @chcokm ,
I'm not entirely sure about your setup. But i read this: http://stackoverflow.com/questions/3954621/deploying-just-html-css-webpage-to-tomcat

It has some instructions about how to set up the HTML pages. However, if you are getting the main screen with the links then it's serving html, so I think you got that part right. Next thing I'd do is check your Chrome console (ctrl-shift-I) and look for errors. Something like a 404 error or some other kind of error indicating a problem. I'll need more information about the errors you're getting in order to provide more specific instructions.

-Chris

[chcokm]

Chris, I'm attaching a document with a screenshot with what I'm seeing. There are 404 errors in the "Console" tab when I use (ctrl-shift-I). Calypso Errors.docx

[chrisknoll]

Great, that's good information. Here's what I see: It's trying to load data from WebAPI from the following URL: http://omopd01:8080/calypso/undefinedsource/sources/

The first thing that looks definitely odd is the WebAPI url should end with /WebAPI. In the calypso config (found in /calypso/js/appConfig.js) you should have something like the following:

define([], function () {
    var config = {};
    config.webAPIRoot = "http://omopd01:8080/WebAPI/";
    return config;
});

The webAPIRoot is the path to the WebAPI service (which you installed in tomcat with the webAPI.war). You put the calyspo path in the config, but we're not accessing the service via /calypso/source/sources, we're acessing the service as /WebAPI/source/sources. In the prior message you sent me where you saw the JSON output for the /source/sources, that URL to get to that JSON should be the same root that you have in config.webAPIRoot.

The second thing that looks odd is the 'undefined' in the url path above, as if it was a NULL value in the javascript. Here's the source code for that call:

    var loadPromise = $.ajax({
            url: config.webAPIRoot + 'source/sources/',
            error: function (error) {
                console.log("Error: " + error);
            }
    });

As you can see it just makes a post to the configured WebAPI root + 'source/sources/'. It means that your calypso/js/appConfig.js file should have "http://omopd01:8080/WebAPI/" as the root (note, the trailing slash is important).

-Chris

[chcokm]

Chris, I updated the config.js with the URL "http://omopd01:8080/WebAPI/" but I'm still seeing an issue. I've attached an updated screenshot.

[chcokm]

Calypso Errors 2.docx

[chrisknoll]

I think you have a cache issue? it still says 'calypso' in the url, and also this 'undefined' part of the URL which I'm not sure how that could be comming up. Do you think you could set up a screen share at some point so that I could see what is happening?

[chcokm]

Chris, thanks. That would be helpful. Today I'm wide open but the rest of the week is a little more cluttered. If you have something this afternoon that would be great.

[chrisknoll]

Ok, we can skype. Can I add you as a contact?

[chcokm]

Yes Please. I have a Department account that I can use. The name is ResearchInformatics

Kevin Matthews | Research Informatics | ETL Programmer | Children’s Hospital Colorado 13123 East 16th Avenue, Box B321 | Aurora, CO 80045 | Phone: (720) 777-3207 | Fax: (720) 777-7276 | Kevin.Matthews@childrenscolorado.orgmailto:Kevin.Matthews@childrenscolorado.org Connect with Children's Hospital Colorado on Facebookhttp://www.facebook.com/thechildrenshospital and Twitterhttp://twitter.com/childrenscolo

From: Chris Knoll [mailto:notifications@github.com] Sent: Monday, May 02, 2016 1:27 PM To: OHDSI/Calypso Cc: Matthews, Kevin; Mention Subject: Re: [OHDSI/Calypso] Install Guide (#14)

Ok, we can skype. Can I add you as a contact?

— You are receiving this because you were mentioned.

Reply to this email directly or view it on GitHubhttps://github.com/OHDSI/Calypso/issues/14#issuecomment-216336369

CONFIDENTIALITY NOTICE: This e-mail, including any attachments, is for the sole use of the intended recipient and may contain confidential and privileged information. If you are not an intended recipient, or the person responsible for delivering this message to an intended recipient, you are hereby notified that reading, copying, using or distributing this message is prohibited. If you are not an intended recipient, please contact the sender by reply email and destroy all copies of the original message from your computer system.

---

[chcokm]

Chris, is there a good time for you? What is your Skype name?

[chrisknoll]

You can add me at cknoll.jj Also, I'm out of office until tomorrow, but if you add me as a contact, I can ping you and see if you are available, or I'll let you know tomorrow what time looks clear.

I was thinking about the error you've received so far, and i suspect that the configuration has a NULL value in the configuration which is why we get the rul with /calypsoundefined/source/sources. So I'd defintely doublecheck that, but I'll review your configuration directlywhen we connect over skype.

[chcokm]

Thanks Chris, any chance you are available at 11:15 Mountain Time? I'll be in a meeting for the next hour but could ping you then.

[chcokm]

Chris, Thanks. I was in the process of writing you to let you know that it had not worked but I tried it a second time and now it appears to be working without errors! Thanks for helping with this.

Tomorrow is a busy day for me, I'm in meetings until 3:30 Mountain Time, but I have more time on Thursday afternoon if you wanted to regroup and show me what you planned. Thanks!

[chrisknoll]

Thursday afternoon is fine with me.You can ping me on skype, i'll let you know what times I am available tomorrow (i'm not at the office ATM).

-Chris

[chcokm]

Hi Chris, We have loaded the Observation_period table with Start_Date's and End_Date's for all patients. I tried running the study we created again in Calypso but I'm still not returning any results, so there might be something else we are missing. I wanted to see what your availability might be to continue where we left off the rest of this week. Thanks!

[chcokm]

Hi Chris, can you let me know if you got my last post? Also, if I wanted to attempt to install Atlas as well, can I use the same server that we installed Calypso on or does it need to be a stand alone install for some reason? Thanks!

[chrisknoll]

Hi, sorry for the delayed reply. So since we last connected, we've had new releases of WebAPI and Atlas, and now Atlas has the calypso-like functionality so it might be worthwhile to just install Atlas. Installing atlas is very similar to installing calypso: pull the atlas repo down, put the content of the repo under your website root in a folder /atlas and you'll just have to modify the /atlas/js/config.js file in the same way you did for CALYPSO: to point to the correct URL for the webAPI service.

Would you have time to connect sometime next week?

[chcokm]

Hi Chris, I could make sometime early next week, Monday or Tuesday, and then the rest of the week gets a little busier. I'm going to try and see if we can install Atlas. We have some instructions on how to do it from another Atlas User and are going through it. It looks like we have to install R but I don't have the permissions to do that so I'm trying to see if there is someone who does. I'll let you know how we progress.

[chrisknoll]

Ok, Thanks.

Note on R: it's not a requirement for Atlas but it is for things like Achilles (which is exposed through Atlas, but without running it it just means you will find no data). R is a cornerstone of the ohdsi tools tho, so if you can get R installed that'd be good.

[chrisknoll]

How does Monday, July 11th, 1pm EST sound?

[chcokm]

Hi Chris, Sorry for the delay. I was out sick on Friday. I have a meeting starting at 1:30 EST, would later in the afternoon be possible? Did you still want to troubleshoot the Calypso issues?

[chrisknoll]

Sure would 3pm EST be OK with you? I think it would be best to just focus on Atlas, it's mostly the same functionality. We just need to understand what is failing while we generate a cohort based on a simple cohort definition. We can work on getting Atlas installed on your environment, provided you have the proper rights to set up a webapp on your webserver.

[chcokm]

3pm EST will work great. I do have the rights to install webapps on the server. I will be on Skype at 3. Thanks!