Y24-172 - As PSD we want to make the developer experience of enabling/disabling pipeline in WIP identical between SequenceScape and Limber so that we reduce confusion and mistakes.

[TWJW-SANGER]

As PSD we want to make the developer experience of enabling/disabling pipelines in WIP identical between SequenceScape and Limber so that we reduce confusion and mistakes

• [ ] The developer experience of enabling/disabling pipelines in WIP is identical between SequenceScape and Limber.
• [ ] Documentation is provided and linked to that describes the features and the impact on the enabling WIP pipeline (database changes that are difficult to revert?)

[KatyTaylor]

Technical note:

The WIP feature in Sequencescape is actually from a PSD gem, 'record loader', so relevant to more repos than just SS. The current WIP (and dev) behaviour is well documented in the README - https://github.com/sanger/record_loader/blob/master/README.md#dev-and-wip

It's not only to do with enabling / disabling pipelines, it's to do with enabling / disabling the insert of any records into the database (but we use it most when developing new pipelines and inserting new purposes, request types, submission templates etc.).

I can't find documentation for the Limber feature after a quick search, but I'm happy to make some to document the current setup if that would be a good first step.

[KatyTaylor]

Just to add -

I think the desired behaviour is the following, but definitely need to check with the team:

• WIP is enabled by default in development and UAT environments (this is how Limber currently works, but not Record Loader)
• WIP files can be enabled / disabled singly or all together, using environment variables on deployment (Record Loader currently allows enabling single, but not all)

Also, the Record Loader Readme already suggests a way of integrating this with an existing feature flag system (and deploying all WIP files at once):

If you have an existing feature flag system you can use this instead by adding a wip_list method to RecordLoader::ApplicationRecordLoader which returns an array of enabled feature names. For example:

def wip_list
    FeatureFlags.active.pluck(:name)
  end

I haven't tested this but think this is suggesting, say our feature flag was called scrna_core_pipeline, then we could name all our related record loader files after that, e.g.

request_types > scrna_core_pipeline.wip.yml
submission_templates > scrna_core_pipeline.wip.yml
library_types > scrna_core_pipeline.wip.yml