Adding document to serve a destination for KBase feature requests

[cshenry]

The structure can be greatly modified. This is just my initial feel for what belongs in this document. Also, I simply listed things as an example. IT is by no means a complete list, and I will probably personally add alot to this, once we've fully settled on format.

[fperez]

Since this cuts into management, I think it would be good to have a bit of feedback before we merge it from @aparkin, @scanon, @dangunter, @kbase/project-leads and others...

A few thoughts come to mind:

• a field for 'acceptance status' or similar, reflecting whether it's just been proposed or accepted and scheduled by the Executive Committee. Major features that require allocating multiple folks for a substantial amount of time will probably require EC decision, but it's still a good idea to record things here even in the proposed stage.
• not sure we should list here the assigned team as part of the doc: that's very likely to change over time, and probably makes more sense once the major task is broken into more specific work units, to just have individual tickets assigned. I would change the 'assigned team' instead to perhaps only a single person 'point of contact', 'project lead' or somesuch. That's less likely to change and evolve.

Otherwise this is an excellent start, thanks! Let's get some feedback and we'll merge it.

[fperez]

closed by accident, misclick. sorry.

[aparkin]

Yes-- likely this should be requested team at the box level, and requested timeline. Exec would then hear from the box lead to summarize the request, and their suggestions for roadmapping. Something like that.

[mlhenderson]

I'm very confused by this PR.

[aparkin]

@mlhenderson, this appeared as a ticket in JIRA and it was decided by Chris, Fernando and Shane that such large feature requests (We are assuming this is large) should be requested elsewhere than JIRA for various reasons. As a stopgap, the idea was to put BIG feature requests here.

If this WAS a production request then it SHOULD be in JIRA in my opinion. But if it is next-gen then not so much perhaps.

What is confusing is the claim that this could be done by August. So I think what Fernando was getting at was that if this is considered to big for production (e.g. UI/Services rejects this particular one as Next-gen) then suggesting dates and teams is premature.

A

[mlhenderson]

@aparkin, thanks for the backstory.

Shouldn't this go somewhere in nextgen? project_guides is a really weird place for it, that was part of my confusion.

On Thu, Apr 23, 2015 at 3:51 PM, Adam Arkin notifications@github.com wrote:

@mlhenderson https://github.com/mlhenderson, this appeared as a ticket in JIRA and it was decided by Chris, Fernando and Shane that such large feature requests (We are assuming this is large) should be requested elsewhere than JIRA for various reasons. As a stopgap, the idea was to put BIG feature requests here.

If this WAS a production request then it SHOULD be in JIRA in my opinion. But if it is next-gen then not so much perhaps.

What is confusing is the claim that this could be done by August. So I think what Fernando was getting at was that if this is considered to big for production (e.g. UI/Services rejects this particular one as Next-gen) then suggesting dates and teams is premature.

A

— Reply to this email directly or view it on GitHub https://github.com/kbase/project_guides/pull/33#issuecomment-95739092.

[aparkin]

Possibly-- we haven't really figured out where to put our overall roadmapping efforts. I think this falls between nextgen and production so there is some confusion.

On Thu, Apr 23, 2015 at 3:56 PM, Matt Henderson notifications@github.com wrote:

@aparkin, thanks for the backstory.

Shouldn't this go somewhere in nextgen? project_guides is a really weird place for it, that was part of my confusion.

On Thu, Apr 23, 2015 at 3:51 PM, Adam Arkin notifications@github.com wrote:

@mlhenderson https://github.com/mlhenderson, this appeared as a ticket in JIRA and it was decided by Chris, Fernando and Shane that such large feature requests (We are assuming this is large) should be requested elsewhere than JIRA for various reasons. As a stopgap, the idea was to put BIG feature requests here.

If this WAS a production request then it SHOULD be in JIRA in my opinion. But if it is next-gen then not so much perhaps.

What is confusing is the claim that this could be done by August. So I think what Fernando was getting at was that if this is considered to big for production (e.g. UI/Services rejects this particular one as Next-gen) then suggesting dates and teams is premature.

A

— Reply to this email directly or view it on GitHub https://github.com/kbase/project_guides/pull/33#issuecomment-95739092.

— Reply to this email directly or view it on GitHub https://github.com/kbase/project_guides/pull/33#issuecomment-95739839.

Adam Paul Arkin

Dean A. Richard Newton Memorial Chair

Director, Physical Biosciences Division E.O. Lawrence Berkeley National Laboratory

Professor, Department of Bioengineering University of California Berkeley, CA, 94720

CEO/CSO, DOE Systems Biology Knowledgebase, http://kbase.us Director, Berkeley Synthetic Biology Institute, http://synbio.berkeley.edu PI and Co-Director, ENIGMA SFA, http://enigma.lbl.gov http://vimss.lbl.gov Investigator, Energy Biosciences Institute, http://energybiosciencesinstitute.org

Office: 512C Energy Biosciences Building (Berkeley Campus) Mailing address: E.O. Lawrence Berkeley National Laboratory 1 Cyclotron Road, MS 955-512L Berkeley, CA 94720

Contact: W: http://genomics.lbl.gov V: 510-495-2366 C: 510-206-1389 F: 510-486-6219

Assistant: Gwyneth A. Terry V: 510-495-2116

E: GATerry@lbl.gov

[fperez]

Yes, I actually suggested to @cshenry that he drop it somewhere so we could figure out a place for 'roadmap' level big features to be documented at. I figured, once we have it somewhere, it's much easier to discuss if we want to move it later to a different location.

But that way, we can have a concrete discussion on the idea, content, etc. If we later think this actual file should go elsewhere, it's easy enough to move.

[fperez]

So I think for now, this is as good as any other place, given it kind of straddles.

But the main point was that for very large, 'roadmap' ideas, a single JIRA ticket might not be the best way to communicate. I do agree with that: in IPython, we tend to keep those in documents like this one. We will then make issues for concrete, smaller chunks, but we don't really bother with one godzilla-size issue for each of those things.

[cshenry]

No. It's not next gen. These are features requested by users that we could implement in the current system without difficulty. Bill and I agree, the map viz is not hard and we have an advanced prototype that just barely got cut from the previous release.

I just don't want jira tickets accumulating assigned to me for three months. Also, these things get limited exposure in jira for discussion by the exec. Here, additions to this list will be discussed.

Sent from my iPhone

On Apr 23, 2015, at 5:56 PM, Matt Henderson notifications@github.com wrote:

@aparkin, thanks for the backstory.

Shouldn't this go somewhere in nextgen? project_guides is a really weird place for it, that was part of my confusion.

On Thu, Apr 23, 2015 at 3:51 PM, Adam Arkin notifications@github.com wrote:

@mlhenderson https://github.com/mlhenderson, this appeared as a ticket in JIRA and it was decided by Chris, Fernando and Shane that such large feature requests (We are assuming this is large) should be requested elsewhere than JIRA for various reasons. As a stopgap, the idea was to put BIG feature requests here.

If this WAS a production request then it SHOULD be in JIRA in my opinion. But if it is next-gen then not so much perhaps.

What is confusing is the claim that this could be done by August. So I think what Fernando was getting at was that if this is considered to big for production (e.g. UI/Services rejects this particular one as Next-gen) then suggesting dates and teams is premature.

A

— Reply to this email directly or view it on GitHub https://github.com/kbase/project_guides/pull/33#issuecomment-95739092.

— Reply to this email directly or view it on GitHub.

[aparkin]

I'm not entirely sure where the right place for this is. Theoretically this should then appear in the WBS that the leads are preparing and it would ultimately appear as a topic or label on a set of tasks assigned to staff in those boxes. I recognize there are experts on the project who might not at the moment be part of the appropriate production team. But then theoretically that production lead would ask for time from the willing expert maybe? Another model is that Services production is really about lowering the barrier to service incorporation and execution and the Services themselves are assigned the expert designer of that service and their "team". Then when third party developers (one day I hope) begin to truly join us, each of them becomes essentially the maintainer of their service. Production Services might report on how that is is happening and try to corral those individuals in doing what is requested and if they do not meet standard, decommissioning the service.

But I am not sure how to run that as KBase personnel because we would soon have too many people as particular service specialists. :) Obviously, we could have "sprints" (I know -- dirty word) where a KBase team builds a core service and then passes it to production when we thought it was necessary and then the lead developer would be an internal expert consultant to the production team. Hm.

[cshenry]

Well, the way I looked at this is this document can be used as a pool of potential needed functionality mine from our tickets and elsewhere, some of which we might promote to become WBS line items.

Consider them candidate WBS line items. Thing is, I could list a whole bunch of functionality that we can�t really put on the WBS yet, but I have tickets or user interactions where people have asked specifically for them. This document seems like to place to do that.

On Apr 23, 2015, at 8:28 PM, Adam Arkin notifications@github.com wrote:

I'm not entirely sure where the right place for this is. Theoretically this should then appear in the WBS that the leads are preparing and it would ultimately appear as a topic or label on a set of tasks assigned to staff in those boxes. I recognize there are experts on the project who might not at the moment be part of the appropriate production team. But then theoretically that production lead would ask for time from the willing expert maybe? Another model is that Services production is really about lowering the barrier to service incorporation and execution and the Services themselves are assigned the expert designer of that service and their "team". Then when third party developers (one day I hope) begin to truly join us, each of them becomes essentially the maintainer of their service. Production Services might report on how that is is happening and try to corral those individuals in doing what is requested and if they do not meet standard, decommissioning the service .

But I am not sure how to run that as KBase personnel because we would soon have too many people as particular service specialists. :) Obviously, we could have "sprints" (I know -- dirty word) where a KBase team builds a core service and then passes it to production when we thought it was necessary and then the lead developer would be an internal expert consultant to the production team. Hm.

� Reply to this email directly or view it on GitHub.

[aparkin]

I don't really disagree, Chris. I guess I am thinking about how we ensure that these get looked at and considered by the right people in a timely manner, you know?

I suppose I am mixing threads of thought. But this happens to sit on a list I've been punting around.

At the end of the principals of services document I list a bunch of basic functionality that I think should be on our near term plan roughly in order (though this is just a first pass-- not an actual deeply thought out list and plan)

Here it is:

Genome and Metagenome Assembly And Annotation

• Need to support eukaryotic and metagenome assembly (At least minimally).
• Need to support metagenome binning methods (multiple)
• Need to support multiple structural and annotation systems.
• Need to support eukaryotic annotation properly.
• Advanced Genome Browser/Sequence Browser

Phylogenetic and Comparative Analysis

• Need Basic MSA
• Need to get variation pipelines up and running for eukaryotes and microbes
• Need homology and orthology pipelines up asap
• Need Powerful Tree Browser
• Need phylogenetic Profiler.
• Need Operon and Regulon callers
• Need basic regulatory motif caller \ Abundance and Rate Data Analysis **
• Should have RNA-SEQ pipelines in place ASAP (for ubiquitousness)
• Should secondarily have priority for metabolite profile (for uniqueness)
• Tertiary is proteomics

Association Analysis

• Basic clustering tools should be available asap.
• Need basic ontological enrichment analysis tools.
• Need powerful cluster browser

Cellular Network and Community Modeling

• Run an externally generated metabolic model. (map biochemistry? map genes?)
• Need to rationalize metabolic models to allow multiple annotations to be mapped to biochemistry for analysis.
• Need basic model editor.
• Need powerful metabolic pathway browser

Genetic Association and Population Genetic Modeling

• Basic GWAS Analysis
• Basic QTL Analysis

How do we discuss and prioritize these well?

[dangunter]

In agile/scrum terms, any set of issues too large to complete in a single sprint is an "epic". In other words, the tools understand this. I would vote for using an agile tool so we don't have to roll our own version of this whole thing with a JIRA/github hybrid. I doubt this is a popular stance, but I wanted to throw it out there.

[kkellerlbl]

I will try really hard to get my test JIRA upgrade environment up soon, so that we can start testing out the Agile JIRA plugin on a recent version of JIRA. FWIW these are tickets in JIRA.

https://atlassian.kbase.us/browse/KBASE-1861 https://atlassian.kbase.us/browse/KBASE-802

[aparkin]

Interesting.. has anyone any experience using these tools?

[kkellerlbl]

Which, JIRA Agile? No, I don't think so, at least not here, which is why people have been asking to try it out (we can get an eval copy free, and if we can get an open-source JIRA license we should be able to get the Agile plugin free there as well).

My wife uses JIRA Agile at her work and she says they use it all the time for their scrums. I didn't go into specifics but I can ask. (I think we'll probably get more useful info from playing with the plugin, but I am able talk to her before I'll be able to get the eval plugin set up.)

Dan (or anyone really), do you know other Agile tools we might investigate?

[dangunter]

@aparkin what do you mean by "near term"?

[scanon]

Sorry I am joining this conversation so late.

Chris: Thanks for adding this here. We can figure out if this is the best place for it long term.

I would prefer to keep this document structured as a list of requested features. Once something is approved, then we can assign a target date, resources, etc. So maybe replace the "assigned team" with "potential contributors". And replace Timeline with "Desired Timeline". I would expect the people and timeline to be determined by the leads. Which leads would depend on which team takes responsibility for the feature (Production, NextGen, or Science Engagements). Hopefully this sounds reasonable.

Regarding Agile. I'm anxious to try it out. My hope is these features would become backlog items towards a release, then broken down into a series of sprint items. But having a single place to look for outstanding feature requests will be useful.

I'm actually okay if those are in Jira, but I can understand why Chris would want to avoid having hundreds of tickets with his name assigned if he can't do the work. We could enter these as Jira tickets and assign them to someone (possibly me) who would be responsible for grooming the list to generate the roadmap.

[fperez]

Since it seems we're in broad agreement that this is a good starting point (if not finished), I propose I merge this now as-is, and we can then make further edits on the various fine-tunings that have been proposed (such as Shane's comments immediately above).

That should keep things moving forward, and we can continue iterating on the text.

I'll wait til the end of the day in case anyone fundamentally disagrees with that course of action.

@cshenry, if in the meantime you have a minute to make some of these edits already, that would be great. I think some of the proposed adjustments are reasonable and easy to make. But if you can't no worries, we can merge and iterate to avoid holding it up much further.

[fperez]

OK, merging now so we can keep the ball in motion...