umccr / nextflow-stack

AWS stack to run Nextflow pipelines on Batch using shared resources
0 stars 0 forks source link

Added new script for running oncoanalyser through orcabus #115

Open alexiswl opened 2 weeks ago

alexiswl commented 2 weeks ago
scwatts commented 1 week ago

Thanks so much, and truly next level use of jq!

Have you tested this in dev? If not happy to give it a go

scwatts commented 1 week ago

Were there any further changes needed to switch from subject ID to library ID?

alexiswl commented 1 week ago

Have you tested this in dev? If not happy to give it a go

No not yet, just getting the structure right first, we will need to also manually give the oncoanalyser roles permissions to access the data-account buckets as well before we can start testing

alexiswl commented 1 week ago

Were there any further changes needed to switch from subject ID to library ID?

I don't think so, for now we can just continue to use the SBJ nfrom the linked individual object, this is set via the orchestration rather than the nextflow stack too so we can change this down the track without needing to make changes here

scwatts commented 1 week ago

Great, that makes it easy though I will probably change variable names from subject to library or similar when that time comes

For testing is there anything stopping us from using the current S3 bucket to avoid permissions issues? e.g. manually stage inputs into our temp dev bucket then provide an appropriate manifest to the run-v2.sh script. I'd love to test this early in the dev process if possible

victorskl commented 1 week ago

(subscribed)

victorskl commented 2 days ago

@scwatts We had a discussion/update yesterday in devops. Just want to loop you in.

This PR intertwined with OrcaBus O3 release.

I understood that changes contain in this PR, added extension to OrcaBus automation (and doest not affect to existing ops with Portal automation).

So, we probably need to deploy this to prod after testing has performed (tbc in #arteria-dev and #alerts-dev for events).

FYI