Open Discussion: Compiled Code As Data

[max-mapper]

Lets do a community call on version controlling compiled code as data (for greater reproducibility and automation).

Sometimes you need to move code to data, other times you can afford to move the data. Lately we've realized we can treat containers as datasets (since they are just Linux filesystems) and build one set of tools (a distributed filesystem that syncs) to address both cases.

We are interested in hearing other perspectives and about other use cases that people are excited about. We'e invited @freeman-lab, a Neuroscientist and @ekg, a Bioinformatician to share their use cases.

YouTube link: http://youtu.be/YI-S3hnqrew Gitter link (chat/ask questions): https://gitter.im/datproject/discussions

Where: Google Hangouts on Air When: Friday, October 23rd Time: 1pm EST (17:00 GMT) 12:30pm EST (16:30 GMT)

Background reading:

• http://begriffs.com/posts/2015-10-16-internet-of-code.html
• http://www.linux.com/news/enterprise/storage/860948-exclusive-interview-max-ogden-of-hyperos
• http://blog.tonicdev.com/2015/09/10/time-traveling-in-node.js-notebooks.html

Agenda:

• Quick overview of the Dat use case (HyperOS)
• Overview of @ekg's use case (Bioinformatics)
• Overview of @freeman-lab's use case (Jupyter notebooks plus Spark)
• General discussion/Q&A
• Suggest other topics in the comments below

[max-mapper]

We decided to start a little early, will be getting going in the next 10 minutes

[freeman-lab]

Here are some slides on binder, which we discussed during the meeting

http://thefreemanlab.com/talk-nyc-fall-2015/

[flyingzumwalt]

@freeman-lab can you give a link to the slides that you walked through about the Thousand Genome project? The video doesn't show the slides.

[freeman-lab]

@flyingzumwalt those slides were from @ekg, i bet he can post them here

[ekg]

@flyingzumwalt Here they are: https://docs.google.com/presentation/d/1SVkYx4VeKuNX6Yq-nkL86qdKDdMX2qwgK4BEDZlkwv4/edit?usp=sharing

I'm at the Sanger Institute now (Cambridge student). These were from a few years back. The final phase of the 1000 Genomes Project was just published: check out the main 1000G paper and structural variation companion.

[flyingzumwalt]

:+1: I just got to the part of the video where @freeman-lab starts talking about binder and realized that I had confused @ekg with @freeman-lab. Either way, thanks for the links. Interesting work all around!