search

w3c / publishingcg

Repository of the Publishing Community Group
https://www.w3.org/community/publishingcg/
Other
19 stars 8 forks source link

Narrated comic books in epub #20

Open m-abs opened 3 years ago

m-abs commented 3 years ago

Hi,

I'm working as a developer for Nota (the Danish Library and Expertise Center for people with print disabilities). We provide accessibility reading material for Danes, who cannot read ordinary printed text.

We''re in progress of moving from DAISY 2.02 to EPUB 3.

We have a number of narrated comic books in our collection, in our own custom DAISY 2.02 format (I can provide some examples if needed), but we want to use a standard format to distribute them in the future.

I've been looking at EPUB Region Based Narration and the EPUB 3.3 draft which includes comics.

But I don't see how to combine it with media-overlays.

Is Region Based Narration + Media Overlays the way to go? If so how are they combined? If not, is there a better way to do it? What about reading system support?

OriIdan commented 3 years ago

I think that if you have the narration files (in MP3 format or other audio format) the best way is Media overlay. As for reading application support, unfortunately as much as I know many reading applications does not support Media overlay. We at Helicon Books developed a reading application that fully support Media Overlay including in reflowable books. Contact me in private for more details.

-- Ori Idan CEO Helicon Books http://www.heliconbooks.com

On Wed, Feb 24, 2021 at 4:57 PM Morten Sjøgren notifications@github.com wrote:

Hi,

I'm working as a developer for Nota https://nota.dk/om-nota/english (the Danish Library and Expertise Center for people with print disabilities). We provide accessibility reading material for Danes, who cannot read ordinary printed text.

We''re in progress of moving from DAISY 2.02 to EPUB 3.

We have a number of narrated comic books in our collection, in our own custom DAISY 2.02 format (I can provide some examples if needed), but we want to use a standard format to distribute them in the future.

I've been looking at EPUB Region Based Narration http://idpf.org/epub/renditions/region-nav/ and the EPUB 3.3 draft https://www.w3.org/TR/epub-33/ which includes comics https://www.w3.org/TR/epub-33/#comics.

But I don't see how to combine it with media-overlays https://www.w3.org/TR/epub-33/#sec-media-overlays.

Is Region Based Narration + Media Overlays the way to go? If so how are they combined? If not, is there a better way to do it? What about reading system support?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/w3c/epub-specs/issues/1522, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAB43QGT64TVIXMAQ5P4CYLTAUHV7ANCNFSM4YEVE5VQ .

llemeurfr commented 3 years ago

I've seen a sample of comics designed by Nota. Narration is associated with panel-to-panel navigation. A "simple" media-overlay on fixed layout EPUB does not solve the issue.

This is why Morten is asking about the future of EPUB region based navigation in EPUB 3.3.

m-abs commented 3 years ago

I think I better show the examples @llemeurfr mentioned. You can see a free example here: https://m.nota.dk/embedded/39619?guest=true

It runs as a guest user, so you might need to open the URL a couple of times before it works (guest sessions are a bit broken at the moment).

If you go to "Side 3" (page 3) in the right-side burger menu, you'll see the panel-to-panel navigation with synchronized narration.

Our DAISY comic books are very simple, each comic book page looks like this:

<?xml version="1.0" encoding="Windows-1252"?>

<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <meta http-equiv="Content-type" content="text/html; charset=windows-1252" />
    <title>Gravræverne</title>
    <style type="text/css" xml:space="preserve">
      div.page {
        position: relative;
        top: 0px;
        left: 0px;
        zoom: 50%;
      }
      div.area {
        position: absolute;
        border: dotted;
        border-width: 1px;
        margin: 0px;
        padding: 0px;
      }
      div.page img {
        position: absolute;
        top: 0px;
        left: 0px;
        z-index: 2;
        filter: alpha(opacity=70);
        opacity: 0.7;
      }
    </style>
    <meta name="dc:identifier" content="dk-nota-39619" />
    <meta name="ncc:charset" content="windows-1252" />
  </head>
  <body>
    <h2 id="eifw_0003"><a href="eifw0007.smil#eifw_0003">Side 3</a></h2>
    <div class="page" id="image0003" style="height: 2898px; width: 2048px;">
      <img style="height: 2898px; width: 2048px;" src="image0003.jpg" alt="image" />
      <div class="area" id="xeif_0009" style="left: 145px; top: 127px; width: 1813px; height: 749px;"></div>
      <div class="area" id="xeif_000a" style="left: 145px; top: 870px; width: 516px; height: 497px;"></div>
      <div class="area" id="xeif_000b" style="left: 661px; top: 870px; width: 345px; height: 497px;"></div>
      <div class="area" id="xeif_000c" style="left: 1006px; top: 870px; width: 952px; height: 497px;"></div>
      <div class="area" id="xeif_000d" style="left: 142px; top: 1364px; width: 1816px; height: 300px;"></div>
      <div class="area" id="xeif_000e" style="left: 142px; top: 1661px; width: 746px; height: 515px;"></div>
      <div class="area" id="xeif_000f" style="left: 888px; top: 1664px; width: 527px; height: 512px;"></div>
      <div class="area" id="xeif_0010" style="left: 1415px; top: 1664px; width: 543px; height: 512px;"></div>
      <div class="area" id="xeif_0011" style="left: 142px; top: 2173px; width: 906px; height: 503px;"></div>
      <div class="area" id="xeif_0012" style="left: 1048px; top: 2173px; width: 419px; height: 503px;"></div>
      <div class="area" id="xeif_0013" style="left: 1467px; top: 2173px; width: 491px; height: 506px;"></div>
    </div>
  </body>
</html>

The smil-files in our DAISY books points to the div.area#<id>, just like it would for any text element.

What we're looking for is a more standardized way of doing the same in EPUB 3. :)

mattgarrish commented 3 years ago

Is Region Based Narration + Media Overlays the way to go?

Region-based navigation wasn't designed with media overlays playback in scope. So even if there were support for the navigation in reading systems (pretty sure there isn't as only aquafadas was working on it that I know of and they aren't around anymore) I highly suspect the regions would not auto-focus with playback.

You'd have to know what text is in what region to make them work together, and I'm not sure how that would work in practice (id the region in the smil doc with a par?).

m-abs commented 3 years ago

Region-based navigation wasn't designed with media overlays playback in scope.

I feared that was the case.

So even if there were support for the navigation in reading systems....

I'm hoping since it is used with EPUB 3.3 comics in the draft, it will become more supported in the future :)

...I highly suspect the regions would not auto-focus with playback.

Agreed.

You'd have to know what text is in what region to make them work together, and I'm not sure how that would work in practice (id the region in the smil doc with a par?).

I don't think linking the ID in the file smil to the region defined by the media fragment in the region-based navigation would probably not work in practice.

Maybe if we could share the media fragment between those two, it would solve it but I don't think that is supported by relative IRI reference used in the smil.

mattgarrish commented 3 years ago

I'm hoping since it is used with EPUB 3.3 comics in the draft, it will become more supported in the future :)

Well, I'd hate to disappoint you, but... those semantics are only in the 3.3 spec because the structure vocabulary was integrated into it, not because there's any movement to take up region navigation at this time.

It's probably one to incubate in the community group first, especially this problem of interaction with media overlays.

m-abs commented 3 years ago

Ahh :(

Thank you for the clarification.

wareid commented 3 years ago

@m-abs Thank you for posting this issue! It's a really interesting challenge, and while others have pointed out the challenges here, I think this aligns really well with work we want to do around FXL accessibility, and I've also been curious about the use of region-based navigation to accomplish this. While it may not be currently possible to do this, I think it's an interesting idea to explore as a future feature.

iherman commented 3 years ago

@wareid should we label this (and similar) issues as deferred?

iherman commented 3 years ago

@Jeffxz @mteixeira-wwn Is this issue for the CG? If yes, it may have to moved there...

mteixeira-wwn commented 3 years ago

Yes, worth pulling into the CG. It would be great to get an intro to the problem and examples shared in our calls.

dalerrogers commented 3 years ago

I work in multimedia. Watching.

Best Regards,

Dale Rogers, M.Ed., CIW Assistant Professor, Web and Multimedia Design Nashville State Community College Instructional/eLearning Developer @. @. http://dalerogers.me/ https://www.linkedin.com/in/dalerrogers/

From my iPhone. Pardon my thumbs.

From: Morten Sjøgren @.> Sent: Monday, July 12, 2021 9:13:37 AM To: w3c/publishingcg @.> Cc: Subscribed @.***> Subject: [w3c/publishingcg] Narrated comic books in epub (#20)

Hi,

I'm working as a developer for Notahttps://nota.dk/om-nota/english (the Danish Library and Expertise Center for people with print disabilities). We provide accessibility reading material for Danes, who cannot read ordinary printed text.

We''re in progress of moving from DAISY 2.02 to EPUB 3.

We have a number of narrated comic books in our collection, in our own custom DAISY 2.02 format (I can provide some examples if needed), but we want to use a standard format to distribute them in the future.

I've been looking at EPUB Region Based Narrationhttp://idpf.org/epub/renditions/region-nav/ and the EPUB 3.3 drafthttps://www.w3.org/TR/epub-33/ which includes comicshttps://www.w3.org/TR/epub-33/#comics.

But I don't see how to combine it with media-overlayshttps://www.w3.org/TR/epub-33/#sec-media-overlays.

Is Region Based Narration + Media Overlays the way to go? If so how are they combined? If not, is there a better way to do it? What about reading system support?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/w3c/publishingcg/issues/20, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAOCKESF4UCBXE6XFNSHJCLTXL2BDANCNFSM5AG57LGA.

m-abs commented 2 years ago

Could it be as simple as using <image> in the SyncMedia 1.0?

Something like this:

<smil xmlns="http://www.w3.org/ns/SMIL">
    <body>
        <par>
            <audio src="chapter01.mp3" clipBegin="30" clipEnd="40"/>
            <image src="chapter01.jpeg#xywh=percent:5,5,15,15"/>
        </par>
        <par>
            <audio src="chapter01.mp3" clipBegin="40" clipEnd="50"/>
            <image src="chapter01.jpeg#xywh=percent:20,20,25,25"/>
        </par>
        <par>
            <audio src="chapter01.mp3" clipBegin="50" clipEnd="60"/>
            <image src="chapter01.jpeg#xywh=percent:5,45,30,30"/>
        </par>
    </body>
</smil>

This probably doesn't solve it for more complex comic books, but it could serve as a temporary solution for our needs.

Jeffxz commented 2 years ago

Hi @m-abs, do you have some "real" example of a few images, region defined by "chapter01.jpeg#xywh=percent:20,20,25,25" or "left: 145px; top: 127px; width: 1813px; height: 749px;" and audio clip? I probably can help experiment something? I think this could also relate to potential annotation for comic.

Jeffxz commented 2 years ago

I also created a discussion related to this https://github.com/w3c/publishingcg/discussions/29

m-abs commented 2 years ago

Hi @m-abs, do you have some "real" example of a few images, region defined by "chapter01.jpeg#xywh=percent:20,20,25,25" or "left: 145px; top: 127px; width: 1813px; height: 749px;" and audio clip? I probably can help experiment something? I think this could also relate to potential annotation for comic.

Hi @Jeffxz

"chapter01.jpeg#xywh=percent:20,20,25,25" was just a suggestion, we don't have a real life example of it.

"left: 145px; top: 127px; width: 1813px; height: 749px;" is how we make the comics in our DAISY 2.0.2 books. You can download one of our free comicbookshere select "Daisy" in the dropdown to the left of "Download"