gregoriopellegrino commented 2 years ago

Knowledge base references:

Export setup

In the EPUB export wizard, in the Text tab we set the footnotes as as popup (this in our opinion should be the default for InDesign, when the user chooses to export an EPUB 3):

Issue

With these export settings, which is the most semantically rich (and therefore the most accessible) InDesign exports a blank footnote backlink:

 <a class="_idFootnoteAnchor _idGenColorInherit" href="0-source.xhtml#footnote-000-backlink"></a>

How to fix it

Add the note number in the backlink so the user can return to the main text.

Other improvements

To improve the accessibility of footnotes, you can enrich the semantics by using DPUB Aria roles in the HTML role attribute. In the file 3-enriched-semantics.epub we have made a proposal for enriched semantics.

LauraB7 commented 1 year ago

User story: "As an assistive technology user I can read the text of a footnote and return to the correct position in the text."

awkawk commented 1 year ago

@gregoriopellegrino Can you clarify which control you are changing (e.g., the dropdown labeled "punti elenco" in the screenshot?) and what the english value that you are selecting says?

LauraB7 commented 1 year ago

Just to clarify: it’s not about a change in the GUI, but that the export adds the note number in the backlink so the user can return to the main text. This screenshot from an English edition of InDesign should help. PopUpFootnotes

raman211 commented 8 months ago

@LauraB7 @awkawk @gregoriopellegrino @ways2read Could you please share your thoughts on the following code snippet from InDesign for the footer notes There are no issues reported by ACE

This one is for the end notes

jonaslil commented 8 months ago

Note references

The code snippets don't display the mark up for the note references in the body text. The file 3-enriched-semantics.epub contains suggestions for them, too. The suggested improvements to note reference semantics are applicable to endnotes as well.

Footnotes

The backlink should have the attribute role="doc-backlink".
The class _idGenColorInherit and its associated style is harmful, because it prevents the links being displayed with the default blue colour people are used to. It is applied both to the note reference and the backlink, in footnotes as well as endnotes.

On a more general note, the original issue addresses only one of the three export options for footnotes, and specifically the backlink problem. Indesign has three different options for footnotes which all create their own mark up, and a fourth way that is used for endnotes.

Endnotes

The backlink <a> element should contain only the number, not the entire note text.
The backlink <a> element should have the attribute role="doc-backlink".
The id attribute that is the target of the note reference is in a <span> that contains only the note number, e.g. <span id="endnote-001">1</span>. It needs to be placed in an element that contains the entire note text. Explanation: Some publishers have endnotes in their Indesign files, but will want to display them in a pop up in the epub. This can be attained by adding epub:type="noteref" and/or role="doc-noteref" to the note reference. But with the mark up Indesign produces for endnotes, this results in a pop up that contains only the note number, since reading systems tend to populate the pop up with the content of the element whose id the note reference points to. In the example, moving the id to the <p> element would solve the problem.
The note reference link should have epub:type="noteref" and role="doc-noteref" by default.
Having a <p> element as the container for an endnote is not ideal, since an endnote may contain several paragraphs. Adding a new paragraph to an endnote in Indesign will lead to the note number and its id being repeated for each paragraph in the epub, which leads to a validation error. The examples in the Daisy Knowledge Base use list items or divs as containers for endnotes.
The container for all the endnotes should, if possible, be a <section> (not a <div>) and it should have epub:type="endnotes" and role="doc-endnotes".
It is not quite clear what semantic attribute should be used for a single endnote. I have used epub:type="endnote".

gregoriopellegrino commented 7 months ago

Thank you @jonaslil for your very comprehensive feedback, it is indeed what we wanted to point out to @raman211

raman211 commented 7 months ago

Thank you @jonaslil for your valuable feedback

@awkawk @gregoriopellegrino @ways2read @LauraB7 Could you please share your thoughts on the footnotes of the EPUB fixed layout?

Currently, we are treating them as the normal paragraphs in the fixed layout as shown below

Thanks, Ramanjulu

gregoriopellegrino commented 7 months ago

Thank you. We had a discussion in the expert group. These are our considerations for EPUB Fixed Layout footnotes.

We recommend using the same semantic structure used for reflowable EPUBs:

Note reference: <a epub:type="noteref" role="doc-noteref" href="test.xhtml#footnote-000">1</a>
Note in an aside: <aside id="footnote-000" epub:type="footnote" role="doc-footnote">
Note text in a <p> tag
Backlink at the beginning of the footnote content: <a href="test.xhtml#footnote-000-backlink" role="doc-backlink">1</a>

Thanks

raman211 commented 6 months ago

@gregoriopellegrino @jonaslil @awkawk @ways2read @LauraB7 Thank You, so much for your valuable feedback

The expected semantics for all three below-mentioned options are the same right?

And it should follow

Note reference: <a epub:type="noteref" role="doc-noteref" href="test.xhtml#footnote-000">1</a>
Note in an aside: <aside id="footnote-000" epub:type="footnote" role="doc-footnote">
Note text in a <p> tag
Backlink at the beginning of the footnote content: <a href="test.xhtml#footnote-000-backlink" role="doc-backlink">1</a>

Please share your thoughts on it?

Thanks, Ramanjulu

jonaslil commented 6 months ago

@raman211, thank you for raising this question. We had a discussion in the expert group and this is what we think:

The note reference should have the same semantics in all three options. It follows the pattern in your code example.

As for the notes, the second and third option should use the same markup, as described in your post and in https://github.com/ways2read/InDesignA11Y/issues/3#issuecomment-1961316723 .

The first option, End of section (Endnotes), should have endnote semantics. We propose the following markup:

<section role="doc-endnotes" epub:type="endnotes">
    <ol>
        <li id="endnote1">
            <p>...</p>
            <p><a href="#ref1" role="doc-backlink">Go to note reference 1.</a></p>
        </li>
        <li id="endnote2">
            <p>...</p>
            <p><a href="#ref1" role="doc-backlink">Go to note reference 2.</a></p>
        </li>
        ...
    </ol>
</section>

Note that the backlink string needs to be in the language of the publication, and should ideally be customizable by the user. For this reason, we propose adding a field in the export settings where the user can enter the desired backlink string. The position of the number in the string could be indicated with a special character.

If this backlink model is not possible to achieve, we propose using the same backlink markup as in the other options: note number as backlink at the beginning of the note text. In this case, the automatically generated list numbers need to be hidden with CSS.

We also think that a more appropriate label for the third option would be "End of section (Footnotes)". The position of the notes after export will be the same as in the first option, only with different semantics.

raman211 commented 6 months ago

Hi @gregoriopellegrino @ways2read @LauraB7 @CircularKen @jonaslil ,

The feature is available (currently done it for the reflowable workflow only) in the latest InDesign PreRelease of 19.4 The complete one (including the fixed layout one ) will be available in few days time (this week in another PreRelease build)

Looking forward to your valuable feedback, before releasing it in 19.4 Please share your feedback on it ASAP, so that we can accommodate the changes for the 19.4 release.

Thanks, Ramanjulu.

gregoriopellegrino commented 6 months ago

Thank you @raman211 here is our feedback.

For footnotes:

we suggest changing the labels of the third option in the export dialog: a more appropriate label for the third option would be "End of section (Footnotes)"
there is an extra span tag in the footnote reference
in footnotes there is missing a space between the close tag of the backlink </a> and the beginning of the text
with the "End of section" export the notes are exported inside aside tags with footnote role (different from what we suggested in comment https://github.com/ways2read/InDesignA11Y/issues/3#issuecomment-2016404259)

For endnotes:

the id attribute that is the target of the note reference is still in the wrong place (as mentioned here https://github.com/ways2read/InDesignA11Y/issues/3#issuecomment-1931961662)
the container for individual notes is still a p element, instead of the structure suggested in https://github.com/ways2read/InDesignA11Y/issues/3#issuecomment-2016404259

raman211 commented 6 months ago

Hi @gregoriopellegrino, Thank you so much for your valuable feedback.

We have addressed the following items already, but unfortunately those were not part of the builds shared, as these were fixed later The items we have addressed already from our end.

with the "End of section"
the container for individual notes is still a p element, instead of the structure suggested in
there is an extra span tag in the footnote reference, it was added to fix one bug earlier (The behaviour is the same in ID19.3 also ), need to see if this can be changed.

The items we are planning to address before the release.

in footnotes there is missing a space between the close tag of the backlink and the beginning of the text
the id attribute that is the target of the note reference is still in the wrong place

The items we need to move to the next release due to the timelines.

"End of section (Footnotes)"

We are already past the UI / UX freeze by the time the request came, hence it is not feasible now to add the string

CC: @ways2read @LauraB7 @jonaslil @Racmathu @mattgarrish

gregoriopellegrino commented 6 months ago

Thank you @raman211. When can we test the new developments, before the official release?

raman211 commented 6 months ago

@gregoriopellegrino Another PreRelease build ID19.4#48 is available now (Contains all the fixes except the two mentioned above, these will be part of the main Release) The items we are planning to address before the release.

in footnotes there is missing a space between the close tag of the backlink and the beginning of the text the id attribute that is the target of the note reference is still in the wrong place

Could you please share your feedback ASAP?

Thanks, Ramanjulu.

CC: @ways2read @LauraB7 @jonaslil @Racmathu @mattgarrish

Racmathu commented 6 months ago

Please file a JIRA ticket in case you find a bug in our implementation OR you need some enhancements/improvements. That way, you'll have visibility into the latest updates on that issue.

gregoriopellegrino commented 6 months ago

Dear @raman211, thank you. Here is our feedback on footnotes and endnotes in the latest version of the Pre Release.

Footnotes:

in all cases the footnote reference has an extra "unnecessary" span
export as endnotes: everything looks okay, except that the value attribute of the <li> element is incorrect when the document is split by style export flag. For example, the note is number two, but you find <li value="1"> instead of <li value="2"> (test file: notes-test-footnotes-as-endnotes.epub.zip)
export after paragraph: everything looks okay
export to popup: except that it needs to change the name in the export dialag, it is missing the space between the note reference and the note text (that is present when you export the footnotes with the other options), otherwise it is ok
we noted that in Fixed Layout footnotes, the id attribute is in the backlink element, not in the aside

Endnotes:

there are the problems that @jonaslil already reported: the heading is not exported and the id is inserted on the wrong element (https://github.com/ways2read/InDesignA11Y/issues/3#issuecomment-1931961662)

raman211 commented 5 months ago

Hi @gregoriopellegrino We have passed the timelines of ID19.4 to address the issues mentioned in the above comment We will be addressing these as part of the InDesign next release i.e ID19.5.

With the noteref syntax like <a epub:type="noteref" role="doc-noteref" href="test.xhtml#footnote-000">1</a> We have seen issues (like always coming inside the popup and clicking on the link not working) with apps like Books and Thorium, when Epub Type and Role are applied on the noteref tag. These apps are frequently used EBOOK Readers, hence we have made last-minute changes to the noteref to not to have the above properties.

Could you please share your thoughts on this to finalize the syntax of noteref tag for different footnote placements like 1) InSide Popup 2) After Paragraph 3) End of the Seectuion etc, and also for the End Notes cases?

Thanks, Ramanjulu.

CC: @ways2read @LauraB7 @jonaslil @Racmathu @mattgarrish

gregoriopellegrino commented 5 months ago

Dear @raman211, do you mean the backlink that is shown inside the popup that shows the note text?

Will some of the developments with respect to this issue be released in version 19.4? Or is everything postponed to the next version?

Thanks, Gregorio

raman211 commented 5 months ago

Hi @gregoriopellegrino, Yes, we are talking about the backlink that is shown inside the popup that shows the note text. We have removed at few of the places of adding the epub:type="noteref" role="doc-noteref" especially when the footnote placement is chosen as the end of the paragraph.

The changes (which would go to the ID19.4 release) are available in the latest pre-release build of ID19.4#62 build. Please have a look at the changes there and share your thoughts on it.

raman211 commented 5 months ago

Hi @gregoriopellegrino @ways2read @LauraB7 @awkawk @jonaslil

Please find the changes in the ID19.4 release builds, which are available for the public now. CC: @Racmathu

jonaslil commented 5 months ago

Dear @raman211 , here's our feedback to your questions about noteref markup in https://github.com/ways2read/InDesignA11Y/issues/3#issuecomment-2063572083:

We don't see the backlink inside the popup as a problem, or at least not a reason to change the markup we have proposed. The backlink is needed for reading applications that do not support popup display. (One of these apps is Adobe Digital Editions.) Reading systems could implement hiding the backlink in the popup based on the semantic backlink markup.
We noticed that epub:type="noteref" has been removed in the End of the section and After paragraph options, and only role="doc-noteref" is present. We think it is better to keep both attributes for all the export options, for the following reasons:

These semantic attributes do not prescribe any particular way of displaying footnotes, they just identify the link as a note reference. For example, Thorium displays notes in a popup by default when either role="doc-noteref" or epub:type="noteref" is present, but you can disable this behaviour by checking Display > Disable footnotes. Then the note references work as regular links. So removing semantic attributes is not necessary to achieve this.

Regarding Apple Books (iOS), it will hide the footnote text from view if it is in an aside with epub:type="footnote". The footnote will then only be available in popup display through the noteref link. But for this to work in Books, you need epub:type="noteref". So removing epub:type="noteref" may make the note text impossible to access in Books.

jonaslil commented 5 months ago

Adding some more detailed feedback on footnotes and endnotes. The testing was done mostly in the last 19.4 prerelease.

Footnote markup:

In the After paragraph export option, footnotes are now in a div instead of an aside. We proposed aside and still think it's better, especially in this option which places the note in the body text flow.

Footnotes, issues related to note numbering:

If you set Restart numbering Every page in the Document footnote options, the numbering will still be sequential (not restart) in the epub note references and backlinks.
In the End of section option, the value attribute does not always reflect the actual note numbering. If you split the content at export, you will have one note list at the end of each file. The note numbering may be continuous, but the value attribute sequence starts at 1 in each list. See sample file Footnotes_prerelease_19.4_end_of_section.zip
The value attribute does not seem necessary here since the automatic list numbering is hidden. If it is used, it should reflect the note numbering. It is only needed for the first list item, when the number of this item is something else than 1.

Endnotes:

Endnotes are much better now. The id is in the right place and the heading is exported as it should. Some remaining issues:

Empty span around the backlink
If you set the numbering mode to Restart every story, but collect the endnotes in a single place (Positioning Options, Scope: Document) you will get duplicate id:s when the numbering restarts: id="endnote-001" etc will be used again. The “value” attributes also do not reflect the restart. See sample file Endnotes_prerelease_19.4_restart_every_story.zip
The best way to handle this would be to create a new ol for each restart. Typically this setting would be used when the endnote numbering restarts for each chapter, but the endnotes are collected at the end of the book. The publisher may then want to insert subheadings for different chapters in the endnote section. This is not possible if the endnotes are one single list.
If you set the endnote numbering to start from something else than 1, the value attribute sequence will still start from 1. See comment above about the value attribute in footnotes exported at the end of section.

jonaslil commented 4 months ago

Hello @raman211, here's a sample epub file showing the empty span around endnote backlinks: Endnotes_19.4.zip

raman211 commented 4 months ago

@jonaslil Could you please clarify this one

If you set Restart numbering Every page in the Document footnote options, the numbering will still be sequential (not restart) in the epub note references and backlinks. When we choose the option of Restart numbering Every page. Are you talking about the ID of the element we are inserting which always starts with 0 or 1 As we need to keep the ID's of the inserted elements unique, and say there are two pages where we have the footnotes placed, then no way we can match the footnote number with the ID of the footnote element. As it is just part of the code and we believe it is not going to read out by the screen readers. Is it mandate to match the footnote number with the ID of the element?

jonaslil commented 4 months ago

Hi @raman211. Sorry for being unclear about this. I was referring to the visible note numbers, not the element's id. You are right: the id does not need to match the footnote number. Screen readers will ignore this attribute.

gregoriopellegrino commented 3 months ago

Dear @raman211, to prepare ourselves for the call scheduled on Monday, June 17th: are there any new updates or releases related to this issue (specifically https://github.com/ways2read/InDesignA11Y/issues/3#issuecomment-2074954495)?

Currently, we are running the InDesign Pre-Release 19.5 (Mainline CI #f95bd09) version on our machines.

Thanks

NawneetG commented 3 months ago

Dear @gregoriopellegrino @ways2read @LauraB7 , we have incorporated all the feedbacks received and the changes are part of 19.5.0.58 PR build. Please test and let us know your valuable feedback. Happy to hear from you @raman211 @Racmathu

jonaslil commented 3 months ago

We were happy to note that much of our feedback has indeed been incorporated in the latest prerelease build.

We see two things that need to be addressed before we can close this issue:

Footnotes

A more appropriate label for the third footnote export option, currently labeled "Inside a popup (EPUB3)". We suggest a label similar to "End of the section (Endnotes)", replacing the word "Endnotes" with "Footnotes".

Endnotes

Remove the empty <span> around endnote numbers (inside the backlink <a> element).

For the other remaining endnote issues, I have submitted bug reports on the prerelease page.

jonaslil commented 3 weeks ago

In the latest InDesign 20 prerelease builds, the first placement option for footnotes has been relabeled.

Previous label: End of the section (Endnotes) Current label: End of the section (Footnotes)

This is misleading, as the first placement option actually creates endnote semantics for the footnotes. The original label was accurate and should be restored.

We have proposed changing the last option, currently labeled "Inside a popup (EPUB3)", to "End of the section (Footnotes)" since this option also places the notes at the end, but uses footnote semantics.

ways2read / InDesignA11Y

Footnotes and Endnotes & Links to Reference Marks #3

Export setup

Issue

How to fix it

Other improvements