Keep flow cell data in a single directory

[seallard]

Description

Maintaining two directories for flow cell data complicates the data flow and logic for processing flow cells. There's a need to simplify this structure.

Note: the backup logic would need to be adjusted as to not backup both raw and converted data.

Suggested solution

Merge the two directories (/home/proj/production/flowcells and /home/proj/production/demultiplexed-runs) into a single location, ensuring data consistency and simplified processing flow.

• Patch demultiplexing workflow to output data into /home/proj/production/flowcells only
• Patch any dependencies relying on /home/proj/production/demultiplexed-runs to exist
• Remove /home/proj/production/demultiplexed-runs

This can be closed when

We have a single location for flow cell data

Blocked by

Supporting the new novaseqx machines has higher prio

[seallard]

For first refinement:

• Do we want to do this?
• How important is it?
• How much effort is it?

For subsequent refinement:

• Identify dependencies
• Create smaller subtasks
• Discuss technical implementation

[seallard]

• Do we want to do this? Gain: simplified logic in processing flow for flow cells, regardless of machine type.

• How much effort would it be? A lot of logic depends on the current structure.

Refine further once higher prioritised projects are completed.

[seallard]

No clear gain and will not be prioritized or done soon.