September 2018 Hackathon Goals

[stevencox]

Green Team : September

ICEES: Scientific Use Cases

• [ ] Validate integrated feature tables #136, #122
• [ ] Evaluate ICEES API output
• [ ] ICEES API usability with respect to Gamma Reasoner
• [ ] Expand temporally to include additional years of patient data (2012-2017), #146
• [ ] Expand feature variables to include additional clinical variables (e.g., pulmonary function tests, FA-relevant measures [metformin, HCT, PLT, WBC]), CMAQ airborne pollutants data, DOT roadway data, and ACS Socioenvironmental data, #146
• [ ] Develop plans for expanding to include new cohorts (e.g., pain, DILI, diabetes), #146
• [ ] Publish approach and preliminary findings, #145

X-CTSA Translator Clinical Data Pipeline

• Present a proof of concept translating EHRs across CTSAs into a single de-identified format sharable as a smartAPI. Publish a pipeline from i2b2 to FHIR to ICEES:
  • [ ] Publish a unified ICEES repo https://github.com/NCATS-Tangerine/icees-api preprocessing https://github.com/xu-hao/datatrans/tree/master/spark #138
  • [ ] Generate FHIR from i2b2 #137
  • [ ] Document and share across Translator/CTSAs, #140

(Socio)Environmental Exposures Knowledge Sources

• [ ] Document and share, including an architecture diagram #139
• [ ] Create new CMAQ airborne pollutants exposures API, #141
• [ ] Create ACS socioenvironmental exposures API, #142
• [ ] Create DOT roadway exposures API, #143

Translator Platform Services

• [x] BioNames v2 #147
• [x] Onto v2 #148
• [x] CTD v2 #149
• [ ] OmniCorp v2
• [x] Rosetta v2 #150
• [x] Write Translator Platform doc #151
• [ ] Collaborate on development of NIEHS Tox21 API
• [ ] Collaborate on development of M2M similarity API

[karafecho]

@stevencox: Perhaps add OmniCorp to Translator Platform Services?

[karafecho]

Update from Emily on plan for ICEES integrated feature tables and X-CTSA clinical data pipeline, minor corrections provided by Kara:

• Emily to create a HuSH+ datamart on our i2b2 server that James will be able to access during the Hackathon if needed. (No open access—James can log in if needed.)
• James to use that datamart prior to the hackathon to produce FHIR files for all patients in the HuSH+ cohort—our “pile of FHIR” (Open access)
• James to have CAMP FHIR available on his laptop for use in collaborating with others (Not yet ready to share code, but could share executable ?)
• Emily to create ICEES tables (clinical variables only) for 2012 and 2013, to add to existing tables for 2010, 2011, 2015. (Not for sharing; intermediate data still contains identifiers)
• James to merge Emily’s clinical ICEES tables from 2012, 2013, 2015 with other data (roadway, CMAQ, etc.) where available to create final ICEES tables.
• Hao to take final ICEES tables and host them on the RENCI box where the 2010 and 2011 tables reside. (Open access); open questions include whether Translator teams will be permitted access to ICEES only via the ICEES UI/API, or whether Translator teams will be permitted access to the underlying integrated feature tables.

[stevencox]

@karafecho @empfff @xu-hao @jameschump

• This is great.
• I get how the things labeled "Open access" are for the Hackathon. The others are probably meaningful as internal milestones but it's not clear how they'll be useful to people at the Hackathon. What am I missing?
• Any particular reason we're not ready to share CAMP FHIR code? Sharing is pretty much the purpose for this piece.
• Hao's on vacation this week but is the right person to do the last bullet about hosting ICEES tables when he's back.

[empfff]

@stevencox @karafecho @xu-hao @jameschump yep, so the non-open-access lines are internal milestones--but some of those (like the identified ICEES tables) are dependencies for the open-access stuff (the final ICEES tables), so I thought they belonged on this list.

We can share the CAMP FHIR code--I just didn't want it going too far before we got a green team publication out of it. I defer to the group.

[stevencox]

Ten four to all of that.

[karafecho]

@stevencox @empfff : For what it's worth, I agree with Emily that we should not share the CAMP FHIR code until we have a publication (or at least a submitted manuscript).

[stevencox]

I'm fine with doing that.

Also, at some point, I'd like to have an over-nachos discussion about this for my edification. Is hiding code really the secret to publication? I expect we'll want to publish something about Robokop (@cbizon) - that's all way public. I'm pretty sure @balhoff's stuff is all entirely open. He likes to publsh things. That lot doesn't seem to fear being scooped.

[balhoff]

I wasn't going to stick my nose in... however it's my preference to develop in the open from the first commit. If someone else has time to look at the code you are probably lucky, and maybe they will even end up collaborating.

[karafecho]

I don't think anything has to be published in a peer-reviewed journal article, but I do think one needs to demonstrate ownership of a product and restrict broad dissemination (meaning large-scale socialization or advertising) if one wishes to publish a peer-reviewed journal article. WRT the latter point, most (regarded) journals simply will not publish 'old news', especially these days. So, yeah, there's a balance between being completely open and being a tad bit guarded. The balance depends on your personal goals. For instance, academics remain 'old school'. Chris and Jim are not traditional academics (i.e., tenure-track faculty).

Furthermore, open-source anything opens the door to academic and industry 'theft' of products plus IP. Sorry, but my cynical view of the world is rooted in experience.

[karafecho]

And I'm not the only one who shares these beliefs.

Here's Charles' perspective on a potential meeting with Gamma and BD (excerpt from an email): BD should be interesting (too bad Perry is retired, at least that's what his linkedin says). I think BD could be interested in Reasoner (probably not Translator) - but, putting my cynical hat on it could come across as yet another biomed knowledge system (e.g., Pathway Studio, Watson) without a clear differentiator. Being even more cynical, if they like the idea a lot it looks like there are zero barriers to them stealing it (I suspect the code license allows them to freely sell Reasoner/Translator).

[balhoff]

Open source is absolutely no barrier to peer-reviewed publication. Traditional journals usually want some sort of scientific application to carry a software article, and that will extend well beyond the source code itself. For direct publishing of the software as a product itself, JOSS is an interesting new choice. Since we typically publish software with a license like MIT or BSD, no one can steal it—they are freely given a license to use it.

The free and open nature of Translator is to me the primary and very important differentiator from those other systems.

[cbizon]

?I agree with Jim 100%.

Chris

---

From: Jim Balhoff notifications@github.com Sent: Wednesday, September 5, 2018 6:33 PM To: ResearchSoftwareInstitute/greendatatranslator Cc: Chris Bizon; Mention Subject: Re: [ResearchSoftwareInstitute/greendatatranslator] September 2018 Hackathon Goals (#135)

Open source is absolutely no barrier to peer-reviewed publication. Traditional journals usually want some sort of scientific application to carry a software article, and that will extend well beyond the source code itself. For direct publishing of the software as a product itself, JOSShttp://joss.theoj.org is an interesting new choice. Since we typically publish software with a license like MIT or BSD, no one can steal it-they are freely given a license to use it.

The free and open nature of Translator is to me the primary and very important differentiator from those other systems.

- You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/ResearchSoftwareInstitute/greendatatranslator/issues/135#issuecomment-418902973, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AKPyHiDBGYlXfWcHYvPzCHZxoRvSN5viks5uYFEjgaJpZM4UjDkB.

[karafecho]

I guess my point relates more to the application space for a particular piece of software code. My argument is that we keep the initial application of CAMP FHIR somewhat restricted until some sort of publication is released (JOSS?), one that all team members agree upon and provides proper attribution to team members. I agree that subsequent applications of the same (or slightly adapted) software code also are publishable in peer-reviewed journal articles, but Green Team loses provenance over the original code the more things get recycled. In fact, many academic groups make their livelihood off of this sort of thing.

I'd also like to remind you that the Translator CTS commentaries are being considered 'confidential' until they are published.

I fully support the Translator program (and I REALLY support the program, in probably more ways than you are aware of). I also completely agree with you that the open nature of the program is the main differentiator between Translator and other academic or commercial programs/products.

However, I recognize that the Translator program relies on people, not just software, and therein lies the challenge. The Translator program is by no means free of questionable behavior by team members. Trust me.

I guess my concerns are that the nuances of peer review, publication, IP, and licensing are tricky and in flux. All I'm suggesting is to be a bit guarded. Open, yet guarded, if that makes sense.

Apologies for the cynicism, but my experience warrants it.

Kara

[karafecho]

Especially when it comes to open access to clinical data.

We're breaking new ground here, so let's be a bit guarded.

[stevencox]

@karafecho - I get the concern about bad actors and poaching and all of that. What's our timeline for submitting an abstract?

Here's Harold Solbrig's proposed clinical profile specification scheme published - on GitHub. It overlaps substantially with our work (FHIR). And uses crepe which seems to do OMOP.

As @balhoff said, for the kind of code we're building here, there are no secrets algorithms. We'll be better off and stay closer to the cutting edge the more open we're able to be.

With Orange willing to share fairly innovative work like Harold's, we stand to look awkward showing up with secret code to a Hackathon whose purpose is sharing and collaboration.

[karafecho]

@stevencox : Your thoughts are well taken.

However, you all need to consider sustainability of the Translator program. NCATS has provided a one-year extension in funding during the 'feasibility' phase of the program and no guarantee of future funding thereafter. (For what it's worth, the OTA award mechanism CANNOT by definition extend beyond the feasibility phase of the program.)

'Open-everything' is fine, but you'd be foolish to NOT consider additional options.

In terms of sustainability, there's room to accommodate 'open', 'semi-open', AND 'closed'. We just need to be careful and define the 'space', 'time-frame', and 'scope' of 'openness'.

Just my two cents...

[karafecho]

One other thing: In terms of attribution, I relinquished first-authorship on the Translator CTS commentaries in exchange for attributing authorship to 'The Biomedical Data Translator Consortium'. And convincing the journal that all contributors deserved recognition took a bit of work.

I point this out only to demonstrate that I truly am a 'team player', just a pragmatic one. ;-)

[karafecho]

One other-other thing: sharing 'stuff' broadly among Translator team members, as @stevencox suggests, is perfectly fine and in the spirit of the program, but the terms and conditions need to be clearly defined. In fact, this very issue has been the topic of discussion among several Green Team members in response to recent events.

[empfff]

Interesting thread. I'll just add that we do have an abstract on CAMP FHIR already in to AMIA, waiting for a decision--we should hear back by Halloween. If we are accepted, at that point I feel entirely comfortable sharing the code. For me, the code isn't what needs protecting prior to publication--it's the idea/concept behind it. (Converting relational data to JSON, which is all CAMP FHIR really does, isn't the novel part--but WHY we're doing it and HOW we're applying it is.)

The angel on my right shoulder tends to side with the "everything should be open from day 1" crowd. The devil on my left has seen some pretty gauche behavior from certain academics, and is much more of a cynic. I usually split the difference by saying I'll happily share everything and more once at least one abstract or manuscript is in the works. I don't think anyone loses out that way, but maybe that's just me.

[stevencox]

Thank you all for an excellent Hackathon, y'all!

@karafecho @empfff @jameschump @balhoff @xu-hao @lstillwe @ColinKCurtis @StanAhalt @arunacs @karlgustafson