sillsdev / appbuilder-pwa

Progress Web App template for Scripture App Builder
https://software.sil.org/scriptureappbuilder
MIT License
8 stars 26 forks source link

Storybook with single audio and timings #597

Open davidmoore1 opened 3 months ago

davidmoore1 commented 3 months ago

Prerequisite: Issue #596 Basic Storybook

A storybook may be defined to have a single audio file that plays for the entire time of the storybook. A timing file can be specified for the audio. This behaves in a similar manner to how timing files and audios work with the scripture passages. In portrait mode, as the audio plays, each phrase on the page is highlighted as it is played. Since there is one audio for the entire storybook, once the timing file indicates that the new timing is on the next page, the page is advanced and the first phrase there is highlighted and so on through the storybook. The audio bar at the bottom controls play of the audio in the standard way: Simulator Screenshot - iPhone 15 - 2024-06-10 at 12 11 34

In landscape mode, a slightly opaque subtitle bar displays at the bottom, showing the text of the current phrase. An audio play/pause bar with the progress of the audio and a timing displays if the user single taps on the screen. After about 3 seconds, if the user does not interact with the bar, it hides. If the user pauses the audio, the bar remains on the screen. Simulator Screenshot - iPhone 15 - 2024-06-10 at 12 10 36

If the user scrolls backward or forward, the audio file is positioned to the start of the new page, as indicated by the timing file.

The following is a sample of part of a timing file:

\id 001
\c 1
\level phrase
\separators .?!:;,
1.000   9.880   1a
9.880   11.000  1b
11.000  11.680  1c
11.680  15.680  1d
15.680  17.720  1e
17.720  18.480  1f
18.480  19.560  1g
19.560  21.440  1h
21.440  22.320  1i
22.320  25.740  1j
25.740  28.020  1k
28.020  29.900  2a
29.900  34.700  2b
34.700  39.820  2c
39.820  44.520  3a
44.520  45.580  3b
45.580  51.780  3c
51.780  56.680  4a
56.680  58.440  4b
58.440  61.180  4c
61.180  63.480  4d
63.480  68.140  4e
68.140  72.200  5a
72.200  76.320  5b
76.320  79.000  5c
79.000  83.100  5d

The first two columns are the start and end time of the phrase and the third column is the phrase name, which is the page number followed by the phrase letter. A sample of the html for a storybook page would be: Screenshot 2024-06-10 at 11 55 01 AM

Configuration for a storybook with a single file. (This also has motion and background audios defined, which will be covered in other issues)

        <book id="001" type="story">
            <n>Unmerciful Servant</n>
            <v>US</v>
            <ct c="9" />
            <cn value="1-9" />
            <font-choice type="book-collection" />
            <f>Unmerciful Servant.usfm</f>
            <audio-story-book mode="single-file" />
            <page num="1">
                <audio background="play">
                    <f src="a1" len="145842" size="2328449">unmercifulservant.mp3</f>
                    <y>C01-01-001-01-timing.txt</y>
                    <background-filename src="a1" len="140356" size="2305216" volume="25">
                        remotebackground.mp3</background-filename>
                </audio>
                <image-filename>Unmerciful Servant-image1.jpeg</image-filename>
                <image-motion start="0.003 0 0.669 0.737" end="0 0 0.988 0.986" />
            </page>
            <page num="2">
                <audio background="continue"></audio>
                <image-filename>Unmerciful Servant-image2.jpeg</image-filename>
                <image-motion start="0.350 0.408 0.550 0.547" end="0.006 0.028 0.994 0.961" />
            </page>
            <page num="3">
                <audio background="continue"></audio>
                <image-filename>Unmerciful Servant-image3.jpeg</image-filename>
                <image-motion start="0.006 0 0.547 1" end="0.322 0.011 0.669 0.989" />
            </page>
            <page num="4">
                <audio background="continue"></audio>
                <image-filename>Unmerciful Servant-image4.jpeg</image-filename>
                <image-motion start="0 0 1 1" end="0.125 0.168 0.700 0.698" />
            </page>
            <page num="5">
                <audio background="continue"></audio>
                <image-filename>Unmerciful Servant-image5.jpeg</image-filename>
            </page>
            <page num="6">
                <audio background="continue"></audio>
                <image-filename>Unmerciful Servant-image6.jpeg</image-filename>
                <image-motion start="0.297 0.028 0.550 0.547" end="0 0 1 1" />
            </page>
            <page num="7">
                <audio background="continue"></audio>
                <image-filename>Unmerciful Servant-image7.jpeg</image-filename>
                <image-motion start="0.028 0.823 0.790 0.170" end="0.369 0.752 0.619 0.248" />
            </page>
            <page num="8">
                <audio background="continue"></audio>
                <image-filename>Unmerciful Servant-image8.jpeg</image-filename>
                <image-motion start="0 0.022 0.650 0.620" end="0.300 0.302 0.700 0.698" />
            </page>
            <page num="9">
                <audio background="continue"></audio>
                <image-filename>Unmerciful Servant-image9.jpeg</image-filename>
                <image-motion start="0 0 1 1" end="0.500 0.201 0.500 0.799" />
            </page>
            <features type="book">
                <e name="show-chapter-numbers" value="no" />
                <e name="lock-orientation" value="none" />
                <e name="show-border" value="inherit" />
                <e name="audio-goto-next-chapter" value="inherit" />
                <e name="story-image-max-height" value="44" />
            </features>
            <images type="illustration">
                <image width="410" height="230">image1.jpeg</image>
                <image width="410" height="230">image6.jpeg</image>
                <image width="410" height="230">image7.jpeg</image>
                <image width="410" height="230">image8.jpeg</image>
                <image width="410" height="230">image4.jpeg</image>
                <image width="410" height="230">image5.jpeg</image>
                <image width="410" height="230">image9.jpeg</image>
                <image width="410" height="230">image2.jpeg</image>
                <image width="410" height="230">image3.jpeg</image>
            </images>
        </book>

The text is broken up into phrases in the same manner as Scripture text is in ScriptureViewSophria