Edit view UX logic

[dylman123]

UX improvements to make the Edit screen easier to use:

There should be 4 modes within the Edit view: a) Start, b) Playback, c) Select, and d) Edit mode.

In Start mode (similar to Select mode but where no caption is selected):

1. The review screen should open in Start mode (video is paused).
2. Caption speaker names to be initialised as blank.
3. Clicking "play" should enter Playback mode.
4. Clicking on a caption should enter Select mode.

In Playback mode:

5. Video plays, relevant captions are highlighted as the captions list is automatically scrolled.
6. Clicking either "pause" should pause the video in Playback mode.
7. Clicking on a caption should enter Edit mode (video is paused, relevant caption is highlighted).

In Select mode:

8. Caption should remain highlighted in list view and become highlighted in timeline view.
9. Clicking on a caption (in the list OR timeline) should play the video segment which relates to that particular caption segment (with 0.5 secs or so buffer played back before the segment) and end playback at the end of that segment.
10. Clicking on the currently selected caption should enter Edit mode.

In Edit mode:

11. Edit buttons to become visible (Add, Edit, Delete, Split, Split, Merge).
12. Caption speaker name should be signalled to be filled in if blank.
13. Caption speaker names should be colour-coded according to the colours assigned in Import view.
14. Entering caption speaker names should implement auto-complete (of the text field).
15. The initial entering of any unique speaker name should auto-fill all other captions which are linked (based on identical speaker tags).

General:

16. Keyboard shortcuts should enable the user to do the above actions with ease. A keyboard shortcuts help box may be provided.

[dylman123]

State machine diagram

[dylman123]

How to override keypresses: https://www.hackingwithswift.com/example-code/uikit/how-to-detect-keyboard-input-using-pressesbegan-and-pressesended

[dylman123]

Currently there is an issue with how the TextField view and the List selector interact. When text is typed into the TextField, often the TextField is exited unintentionally (bug).

[dylman123]

Bugs that currently exist in edit view:

• TextField exits prematurely
• "+" and "-" buttons are not clickable
• Keyboard presses are dodgy in general
• Captions sometimes do not render correctly (for items further down the list)

[dylman123]

Me thinks some (if not all) of the above are memory related issues. Using the Leaks instrument, I detected 50+ leaks. App is using 40-60MB of memory on a simple view. This doesn't seem right. Need to work out what is causing the leaks.

The following vids may help:

• https://www.hackingwithswift.com/articles/210/how-to-fix-slow-list-updates-in-swiftui
• https://www.hackingwithswift.com/sixty/8/6/deinitializers
• https://www.hackingwithswift.com/read/30/6/fixing-the-bugs-running-out-of-memory

[dylman123]

Maybe there is a bug using List in SwiftUI? https://stackoverflow.com/questions/58965282/problems-with-layout-of-some-rows-in-swiftui-list

• Can try using ScrollView instead?

[dylman123]

Maybe there is an issue with how I am passing @State or @EnvironmentObject throughout views? https://forums.developer.apple.com/thread/126890

My latest thinking is that maybe all the functions should be decoupled from the UserData class. Maybe the UI should call the generateCaptions() function directly, and this shouldn’t sit inside UserData.

What might be happening is some sort of interference between the execution of these functions and the UserData class which updates the UI (by “publishing”) whenever a change is sensed.

Although this tutorial seems like I’m on the right track! https://youtu.be/1IlUBHvgY8Q

[dylman123]

Swapping List for ScrollView fixed a lot of the bugs! There are still some memory & CPU issues when the list gets long.

[dylman123]

Learn how to use NSEvents such as capturing key and mouse events:

• https://stackoverflow.com/questions/61153562/how-to-detect-keyboard-events-in-swiftui-on-macos
• https://www.youtube.com/watch?v=ursEgzA79Q0
• https://www.youtube.com/watch?v=mshI3UZmiGI
• https://code.bitbebop.com/macos-game-keyboard-input/
• https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/EventOverview/HandlingKeyEvents/HandlingKeyEvents.html
• https://developer.apple.com/documentation/appkit/nsevent

[dylman123]

Look at 30:00 for keyboard events: https://www.youtube.com/watch?v=FE4ys3tW1VI

[dylman123]

Use this reference to add more keyCodes: https://eastmanreference.com/complete-list-of-applescript-key-codes

[dylman123]

There is a bug that occurs: A double delete when deleting caption at index 0. Selected caption goes to -1 position.

Next elements to build:

• ~Opening a textbox in edit mode~
• ~Closing the textbox when back in select mode~
• ~Spacebar enters edit mode when in select mode~
• ~Mouse clicking on captions triggers actions~
• Signal a text input when in Edit mode
• ~Replace TextField in stepper with Text~
• ~Move logic out of views and into functions!~

[dylman123]

Fixed bug where double delete was occurring at index 0!

[dylman123]

Need ScrollView to scroll when selected caption reaches end of visible list.

[dylman123]

I can get the list to scroll, but the UI doesn’t refresh instantly. It seems to refresh only when certain variables are updated.

[dylman123]

Solved state sync and UI bugs!

[dylman123]

New requirements

~1. Tagging system to quickly assign font settings to each caption with a keyboard press~

2. A way to select and "tag" multiple captions using SHIFT + mouse click, or SHIFT + up/down arrows
3. Undo function with CTRL + Z ~4. In pause/edit modes, clicking on a caption should trigger the video to play that segment.~
4. An error should be displayed if Final Cut Pro is not found/installed when exporting captions
5. ColorPicker currently has a workaround in order to fix the binding issue (this is not ideal UX). This should be fixed.

[dylman123]

Look at using TextField / TextEditor (Swift 5.3) in combination with keyboard shortcuts - will need a refactor...

[dylman123]

~Merge edit & pause states?~ ~This would mean that 'edit' state is when the TextField is active.~

Will not work on this, I am happy with the decoupling of pause and edit states for now.