search

hammerlab / cycledash

Variant Caller Analysis Dashboard and Data Management System
Other
35 stars 2 forks source link

How do we know when we are successful? #655

Closed jaclynperrone closed 9 years ago

jaclynperrone commented 9 years ago

What does success look like? And how do we know when we've achieved it?

We can break this down into long term and short term goals. This is also dependent on who we are developing CycleDash for [#654]

hammer commented 9 years ago

We'll know we're successful when

jaclynperrone commented 9 years ago

Yes! I like these.

"Cycledash has active contributors from outside of our lab" "We have published a paper about Cycledash and it's highly cited"

^^I especially like these because they have a metric we can measure.

What are some ways we can measure the other ones?

Some thoughts:

"Cycledash is an intermediate destination on the critical path for data being analyzed for an important research or clinical project outside of Mount Sinai"

We could get a survey of projects going on outside of Mt. Sinai and find ways in which CycleDash can be integrated in their process. We would need to determine a) what institution(s) we would like to follow b) gather the contacts we have there c) learn about their workflow and see where CycleDash can fill any needs. I know this is less organic than what your point implies, but we need to start somewhere to see how we well we can adapt to outside organizations' workflows.

"Our Cycledash deployment is managed as a critical production system"

Not entirely sure what you mean here. Do you mean internally to Hammer Lab? And what defines "critical"?

"Complementary software is written with Cycledash in mind"

How will we know when this happens? What falls in the category of "complementary", and how can we get other contributors to let us know when they've created it? Maybe a baby step would be to talk more about our goals as open-source contributors in the About section on Cycledash. Perhaps start the conversation there and encourage others to keep in contact with us.

hammer commented 9 years ago

Let me add a few success states that have been driving a lot of Cycledash development so far. I am hoping to de-emphasize these success states as the project matures. I think these success states go a long way to answering the question "Why are we building Cycledash?" that @danvk has asked a few times.

hammer commented 9 years ago

We could get a survey of projects going on outside of Mt. Sinai and find ways in which CycleDash can be integrated in their process. We would need to determine a) what institution(s) we would like to follow b) gather the contacts we have there c) learn about their workflow and see where CycleDash can fill any needs.

For sure. And, honestly, we should do the same thing for inside Mount Sinai first. Each disease area at Sinai (e.g. healthy subjects, cancer, autism, IBD) has its own workflow that is slightly different from the one supported by the core facilities. I've been thinking about building a Cycledash customer advisory board where we pluck people form the core facilities and each disease area and meet regularly with them. We've done a customer advisory board at Cloudera with great success and we're getting close to needing one for Cycledash too.

hammer commented 9 years ago

Also a nice staging of the Cycledash rollout would be within Sinai, within NYC, and finally outside NYC. That said, if we have a lab outside NYC that's particularly excited about Cycledash, we shouldn't try to force this staging.

hammer commented 9 years ago

Another potential success state:

ihodes commented 9 years ago

I like the idea of a customer advisory board.

I think building for a production workflow use-case would maybe be an ideal way to reach our success states. It gives us users, feedback, and would be high-value & high-volume.

The success state that has been driving Cycledash has been "aiding guacamole developers in improving variant calling". The fact that that use-case has dissipated in the past few months has led us to ask who else we should be building it for. There's a single new open source library in Cycledash (React), and it was chosen to speed up development of the product, not so that we could learn more about it.