Open apetkau opened 5 years ago
Relies on #211
Imported from GitLab. Originally posted on 2017/06/22 02:26PM
This is blocked on updating the files data model.
This is sufficiently detailed that I'm going to keep it open.
Imported from GitLab. Originally posted on 2015/12/02 03:28PM Originally posted by Franklin Bristow
Ok great. That all sounds really good, Aaron. Let me know if you need any assistance with anything.
Imported from GitLab. Originally posted on 2015/06/03 01:33PM Originally posted by Franklin Bristow
Yes, I was only concerned about newly uploaded data. With about 10,000 samples now running them all would clog up our cluster for weeks. We can selectively run assemblies on older data if it is requested.
For a quick estimate, looking through the assemblies we already have, it's taking about an average of 4 hours per assembly. Our cluster should be big enough to run all the samples from a miseq run at once (about 30), so only about 4 hours per miseq run to finish all the assemblies. Of course, some will take longer, some will fail, but that's not even too bad.
Imported from GitLab. Originally posted on 2015/06/03 01:30PM
To clarify 1: are you creating something to run assemblies on already uploaded data, or are you just writing something that will submit newly uploaded data? I think we should focus on doing newly uploaded data and then we can write something that would run assemblies on already uploaded data later.
Imported from GitLab. Originally posted on 2015/06/03 01:16PM Originally posted by Franklin Bristow
Okay, that all makes sense to me. I'm thinking, for the order to work on these:
I'm just working on the initial stages of updating the model right now.
Imported from GitLab. Originally posted on 2015/06/03 01:11PM
canUpdateSample
permission on a file) to assign the analysis back to a file. We can update the canReadAnalysisSubmission
and canReadAnalysis
permissions to check to see if the analysis has been assigned to a file, then ask about the permissions on the file given the current user to decide if the user can read them. These shouldn't be major changes to the data model or the permissions model.For which to do first, they both share so much functionality and changes that I think either is fine to choose.
Imported from GitLab. Originally posted on 2015/06/03 11:46AM Originally posted by Franklin Bristow
Awesome, that makes sense. Having a service method like findAssembliesForSample
would work out. If a pipeline gets loaded up that needs assemblies, we can use it to get a list of assemblies.
For associating an assembly to a pair of files:
FileProcessingChain
makes sense to me. However, a few issues I can see with this method:
SequenceFilePair
? Just anyone? Just the MANAGER of a Project containing the Sample containing the SequenceFilePair
(can a pair of files belong to multiple Samples?).AnalysisSubmission
. This currently only has an owner. If they share the results back to a SequenceFilePair
, we may have to modify this to allow any user read access? Or any user who has read access to the SequenceFilePair
?Given all these issues, I'm wondering if it makes sense to start out with the Manual
version, and work our way to the Automated
version? However, all those permission problems will still get in the way.
Imported from GitLab. Originally posted on 2015/06/03 09:38AM
If the assembly is associated with the file/pair of files, then we have a transitive relationship with the sample. We can write a service method something like findAssembliesForSample
that would return a list of assemblies attached to files that are attached to the sample.
I propose two methods of assigning an assembly to a file/pair:
FileProcessingChain
to handle creating the submission automatically. The default assembly might just be "good enough" and doesn't need to be changed.How does that sound?
Imported from GitLab. Originally posted on 2015/06/03 09:16AM Originally posted by Franklin Bristow
So, Matt suggested that we should probably associate assemblies with the exact sequence files used to construct them (so with a SequenceFilePair
or a SequenceFile
).
For how the files are used after that, I do feel they should also be associated with the Sample
in some way. Possibly, a Sample
could have/return a Set<AssembledGenome>
of all the assemblies associated with any sequence files in that Sample
. My reason is that it was requested at one point to be able to associate an assembly with a sample (that's what issue #211 is on). This would then allow us to easily complete this issue by letting users select samples to use for a pipeline and listing (possibly giving the option to choose) which assembly to use for each Sample, similar to how we give the option to choose different sequence files.
What I am still unclear about is how users would associate assemblies with samples right now. In the long run, it makes sense to be automated (similar to FastQC analysis, triggered on upload), but in the short term we may have to let users manually run assemblies and possibly select different parameters. I am going to see if I can discuss it a bit more with Matt.
Imported from GitLab. Originally posted on 2015/06/03 09:06AM
SequencingObject
I could see the overall process of submitting workflows the exact same as we have it now. However, instead of using a
SequenceFile
as input, we would use anAnalysisSubmission
, or specifically anAnalysisAssemblyAnnotation
(though we want to keep track of the submission as well). We would pick one of the output files fromAnalysisAssemblyAnnotation
which could be used as input for other workflows (for example, the contigs file with repeats). See the attached screenshot:To store this information within an
AnalysisSubmission
we would probably have to create different types of submissions (one that takes as input sequence reads and another that takes as input assemblies).Imported from GitLab issue #245. Originally posted on 2015/06/01 08:42AM