RFC: Override NRCS Data in the Sofie GUI

[hummelstrand]

About me

This RFC is posted by SuperFly.tv on behalf of the BBC.

Use case

Background

Traditionally, the end users have not been able to modify or override any aspects of the rundown data. On behalf of the BBC, SuperFly.tv would like to add the ability for the GUI to modify aspects of the data coming from the NRCS. The initial set of features that we would like to add to Sofie can be described by the following four use cases, but we believe these need to be discussed with vested parties in order to find an optimal implementation strategy.

For this discussion, the term "override" will be used to indicate any data that doesn't have it's origin in the NRCS.

Use Case 1 – Editing Piece/Part/Segment Parameters

As a user I want to be able to interact with pieces/parts/segments in the GUI and override the following parameters:

• Source Layer parameters

  • Timing parameters:

    • Start or end (the piece boundaries, editable by grabbing and dragging handles on either end of the piece).

    • Both start and end at the same time (preferably with a drag handle on the actual pieces).

    • In/out (for VT's and other types where the piece boundaries might not be identical to the content's in/outpoints). One UX idea here is to be able to set in/out point of a VT by marking them while hover-scrubbing.

  • Source Layer type (example: from a Camera to an AUX)

  • Mapping (example: Change Camera 1 to Camera 3). For certain types such as VTs, this will need to include a browser with search and filtering of some sort.

  • Setting the piece as one of Sofie's infinite types.

• Deletion of a piece and possible even a part or segment.

• Move piece to another part (either within the same segment, or freely using drag-handle).

The GUI for this will most likely be handles on the pieces in the views, in combination with some sort of Inspector panel that should probably live in the right Side Bar.

Upon push of MOS data from NRCS (e.g. due to the user saving the story) GUI-modified pieces and parts should revert to the NRCS version.

Options per source layer type should be configured via blueprints and be customisable per part / piece.

Use Case 1-specific Discussion Points

• How does Sofie connect to and display the sources for parameters such as VTs which might be a list of thousands of results?

• Should it be possible to multi-select and edit parameters of multiple items at once?

• The current model for the Sofie GUI interaction is that you can't really change anything, so how do we avoid that a user inadvertently changes order and placement of pieces just by scrolling or moving the cursor of the GUI?

• Could Duplication be a useful operation?

Use Case 2 – Locking Pieces/Parts/Segments

As a user I want to be able to prevent NRCS updates to selected pieces/parts/segments.

Where updates have been made to pieces or parts by manipulating them on the GUI, they should revert to the NRCS version upon a push of MOS data. This is desired behaviour, except sometimes during busy moments where the Sofie operator may have updated elements on the GUI, whilst the journalist is also resaving presenter scripts. We need a way for the operator to 'lock this element from NRCS update' and in doing this for the locked piece, part or segment to hold it's new/updated status, and to not revert to reflect the NRCS page upon a push of MOS data.

Locked items need to have their locked status displayed somehow, and their lock state needs to be editable in the GUI, most likely via the aforementioned Inspector panel.

Use Case 2-specific Discussion Points

• Should it be possible to multi-select items to be locked/unlocked?

Use Case 3 – Create Pieces from MOS Plugin

As a user I want to be able to add a new piece in a Sofie playlist/rundown by dragging it from a MOS-plugin directly into the Sofie GUI.

Use Case 3-specific Discussion Points

• As the GUI needs to give the user a good understanding of where the new piece will be created -- possibly including giving a preview of the duration of the new piece influences the existing parts.

• Just like with the other use cases; how are updates from the NRCS handled when the segment is floated/unfloated?

• Should it be possible to create new parts and/or segments, or only to add to an existing part?

• What about replacing one piece with the new piece? If that is possible, are there any parameters of the old piece that should be retained, for example duration?

General Override Discussion Points

• Where would these overrides be stored in Sofie, and how would the data model need to be changed?

• What current assumptions in Sofie need to be revisited if we abandon the "NRCS is King" model?

• Could overrides be limited to Sofie instances, and could/should we instance the entire playlist? If implemented for only the current and Next part, the implications and development time might be significantly lower.

• How are overrides they persisted?

• Is write-back to the NRCS something to consider, either short-term or long-term?

• Could copy/cut/paste be an alternative to drag-and-drop?

• How to handle dragging items where the target is not visible within the current viewport?

• Should there be a way for a user to see if any overrides have been applied to a playlist? If so, would that be shown in the Lobby view, too?

• How does a user cancel/remove all overrides to go back to a playlist that reflects only the data originating from the NRCS?

Proposal

We would like to initiate a discussion around these topics in order to be able to assess the amount of work required.

Process

The Sofie Team will evaluate this RFC and open up a discussion about it, usually within a week.

• [x] RFC created
• [x] Sofie Team has evaluated the RFC
• [x] A workshop has been planned
• [x] RFC has been discussed in a workshop
• [x] A conclusion has been reached, see comments in thread

[nytamin]

Thank you for submitting this RFC! (If you haven’t already, please give our contribution guidelines a read.)

The Sofie Team thinks that this is a very interesting idea, it is something we've been discussing internally before as well.

This type of change would likely affect a fairly large chunk of the ingest logic in Sofie, therefore we think that several workshops would be needed to make an implementation plan, to ensure we come up with something that works well and is sustainable.

We propose a first workshop on Monday, January 22nd at 14:00 CET (13:00 GMT), where we can discuss the use cases and possible solutions.

Agenda, Workshop 1

1. Presentation of the RFC from BBC
2. Outlining of use cases Discuss the use cases, ideate other use cases and potential edge cases/pitfalls.
   • Which would be required initially?
   • Which would be optional / for the future?
3. Initial (short) discussion of possible architectural solutions.
4. Planning of next workshop

If anyone else wishes to join the workshop, pleas email me at johan.nyman@nrk.no

[bevand10]

Great to finally see our brand name in lights @hummelstrand - it's been a long-time coming (thinking back to those open CasparCG sessions in Birmingham and London in the previous decade!)

[MicKiloMic]

Hi - just joining this thread. Great meeting today, thanks all.

[nytamin]

Meeting notes, Workshop January 22nd

Use cases

We discussed and listed various possible use cases

Editing Pieces

• Change Piece from camera 1 to camera 2 (no side effects)
• Change Piece from camera 1 to camera 2 (side effects) Where the change has side effects on other Pieces, eg a studio light and mic is ON when on camera 2.
• Change Piece from camera 1 to OS (“outside source”) 2 Side effect: mics
• re-timing (or remove) a gfx Piece in a Part
  • using a MOS plugin
  • using dragging of the Piece in Sofie GUI
• Trimming VT in/out points
• Replace a VT with another The new VT Piece should automatically get new timings according to the new VT.
• Update spelling in a gfx (possibly using a MOS plugin?)

Editing Parts

• Move a Piece from a Part into another Part Does this change things about the piece? I.e. is it also editing pieces?
• Change Part from camera 1 to Full VT
• Lock the (edited) contents of a Part from NRCS updates
  • Protect from changes
  • Protect from deletions
• Moving a Part within a rundown
• Changing a clip from a “package”/VT to “voice-over” Side effect: mic levels
• Delete a Part (and un-delete a previously deleted Part)
• Make a Part “invalid” (this is useful in order to skip a Part)

Creating new content

• Insert/Create a new Part from “nothing” or “something”
  • Create a Part from a “piece” or mos-object (dragging in from NRCS)
• Insert/Create a new Piece from “nothing” or “something”
• Add/remove/edit Transitions
• Add/remove/edit Adlibs

Editing the rundown

• Dragging/moving Parts
• Move/Add/edit/delete Segments
• Add/remove/edit Global Adlibs

Initial Architectural thoughts

A VERY short discussion on possible solutions.

“Editing the Next PartInstance”

Easy implementation, but very limited useability, might be a short term solution?

“Instantiate the whole Rundown as PartInstances”

Hard to envision how it could work? Likely a dead end?

“Add manipulation stage(s) to the ingest data flow”

A large addition to the data flow. Significant effort needed in implementing this, but possibly doable.

“Rework the ingest data flow to have fine grained reactions to every NRCS update”

A complete change in how we take in updates. Significant effort needed in implementing this, but possibly doable.

“Write back changes to the NRCS”

Keeps the "single source of truth". Likely to be a problem with some NRCSes that don't support write-back.

Other notes from the discussions

• It should be possible to define “editing rules”, eg “user should not be able to change a cam1 to a VT”.
• It would be good to define what is the scope of this work, to have a stopping poind and avoid feature creep.

Next workshop

Another workshop is planned for Monday, January 29th at 14:00 CET (13:00 GMT).

In preparation for next workshop

BBC and NRK will rank the use cases from blocker → important → later → not important

Agenda:

1. Architectural discussions
2. Relate solutions to use cases

If anyone else wishes to join the series of workshops, please email me at johan.nyman@nrk.no

[nytamin]

Meeting notes, Workshop January 29th

Use case prioritizations

BBC's priorities are

• Most important:
  • cases related to changing properties of Pieces (ie cam 1 to cam 3)
  • cases rekated to changing the type of Story (ie change a VT with full audio to "No audio"
  • Disable/Remove a gfx Piece
• Nice-to-have
  • Update spelling in a gfx (eg using mos plugin)
  • change timing a gfx Piece

It was noted that side-effects (ie a mic fader changes due to a changed camera) are likely to happen in all relevant use cases.

MOS Plugin

Discussion on the subject of MOS Plugin support in Sofie to edit for example gfx Pieces.

• Because the gfx Piece content is defined by the gfx system, it will be hard/unsafe to edit it directly in Sofie. One solution could be for Sofie to support loading a MOS Plugin in an iframe which is used to edit the gfx Piece. The mos-protocol API for MOS Plugins (using iframe postMessage) could be used for this.
• We should probably support both editing a Piece by using an internal inspector and/or a MOS Plugin to edit data.
• Question: When implementing the inspector, what should the inputs support? Uploading images? Food for thought.

Implementation options

We discussed briefly the options brought up last week. The two options

• “Add manipulation stage(s) to the ingest data flow”
• “Rework the ingest data flow to have fine grained reactions to every NRCS update”

seems to be the most feasible to continue thinking about. The other options where discarded.

It was noted that there is a high likelihood that the implementation affects the implementation planned in #1126 . Care should be taken to coordinate implementation efforts.

Technical workshop

We concluded that the next step is to have an in-depth technical workshop on designing the architectural structures.

Questions to be answered by the technical workshop:

• Does the proposed solution support all of the above mentioned use cases?
• Can the proposed solution be implented in stages?
• How small can an initial implementation be for a PoC or MVP?

The technical workshop will be held this week, joined by @nytamin, @PeterC89, @Julusian, @mint-dewit and @jstarpl .

Next workshop

Another workshop is planned for ~Monday, February 5th at 14:00 CET (13:00 GMT)~ Thursday, February 8th at 14:00 CET (13:00 GMT).

Agenda:

1. Presentation from the technical workshop
2. Future / implementation plan?

If anyone else wishes to join the series of workshops, please email me at johan.nyman@nrk.no

[nytamin]

Note: The next workshop is moved to Thursday, February 8th at 14:00 CET (13:00 GMT), to give the technical workshop group time to finish up.

If anyone else wishes to join the series of workshops, please email me at johan.nyman@nrk.no

[nytamin]

Meeting notes, Workshop February 8th

Presentation from the technical workshop group

A series of technical workshops have been held during the past week, which have resulted in the following proposal (presented by @mint-dewit and @Julusian on behalf of BBC).

How Sofie works right now

1. INGEST STAGE a. Data from NRCS is sent into Sofie Core via MOS Gateway (or another ingest gateway) b. Sofie Core maintains a copy of the (NRCS) ingestRundown and applies MOS upates onto it. c. Blueprints transforms the ingestRundown into Segments, Parts & Pieces.
2. PLAYOUT STAGE a. Sofie Core: Play Parts and Pieces, using PartInstances & PieceInstances & generate the Timeline b. Playout Gateway: Executes the Timeline

Our proposal

We propose to add a second stage on the ingest side, and add a new Blueprint method which can selectively choose how to update the SourceData, using the NRCS IngestRundown and user-edit-action as input.

1. INGEST STAGE a. Data from NRCS is sent into Sofie Core via MOS Gateway (or another ingest gateway) b. Sofie Core maintains a copy of the (NRCS) ingestRundown and applies MOS upates onto it. c. Sofie Core maintains SourceData in the DB d. Blueprints updates the SourceData selectively, using the ingestRundown and user-edit-action results as input.
2. CONTENT STAGE a. Blueprints transforms the SourceData into Segments, Parts & Pieces.
3. PLAYOUT STAGE a. Sofie Core: Play Parts and Pieces, using PartInstances & PieceInstances & generate the Timeline b. Playout Gateway: Executes the Timeline

Note: The SourceData has the same type definition as the previous IngestRundown, so it'll be backwards compatible with Blueprints.

With this proposal, the Blueprints will now have full control of the rundown data and have the ability to e.g. split stories into segments, reject updates or insert data freely.

This change will be backwards compatible, as there can be implemented a simple fallback blueprint method with the functionality of simply "accept all data from the NRCS, and split stories into segments using the ";" delimiter".

Blueprint helpers

In order to make it easier to write blueprints, we propose that we eventually add a few "helper function", to be used by the blueprints for common operations such as "basic editing of properties", "move a Part", "invert change/undo/redo". Not needed initially though.

User Actions

Blueprints will now be able to define user-edit-action manifests on Parts and Pieces, allowing the Sofie GUI to provide the user with editing capabilities to match.

The exact types of user-edit-actions and their UX is not handled in this RFC, but some potental ones are:

• Edit propterties in a web form (defined by a schema)
• Display a MOS Plugin to edit something
• Display drag-handlers to be able to drag Pieces, or Parts
• Drop MOS objects into the GUI to insert Parts/Pieces

When a user makes an user-edit-action, the result is sent into 1.d step above, and so the Part will be regenerated downstream.

Future

• After this change, we could potentially replace the blueprint.getSegment with a blueprint.getPart instead. Allowing for a finer update granularity, resulting in better performance.

Questions

• Q: Does this mean that we could potentially have more than one data source to generate the Rundowns from? A: Yes, that could be a natural continuation after this has been implemented.
• Q: So how would the default helper functions work? How do we decide what is a default functionality? A: We'll begin by implementing it all in blueprints-only to begin with. Then, when we've gathered knowledge and proven the concepts, we'll start looking into moving the some of the blueprint code into Sofie Core functionality and exposing it as "helper functions".
• Q: How would Mos active/inactive (and reloading NRCS data) work? A: It would be possible to distinguish between changed data vs just a reload (for example by using modified timestamp), so the blueprints would be able to intelligently handle this situation.
• Q: Should the blueprints be able to reject a "removed rundown" input? A: Yeah, probably. That would be a good use case to take into account.

Next steps

We consider the discussions on this RFC to be concluded for now.

If anyone else have any opinions or questions regarding this, feel free to post questions in this thread, or contact me at johan.nyman@nrk.no.