export fails if ID links are missing - even with: #+OPTIONS: broken-links:mark

[hpgisler]

For the following .org file, I get this error: OpenDocument export failed: FIXME: Unable to resolve "todos.org"

+OPTIONS: broken-links:mark

+SELECT_TAGS: drm

• test linking to todos.org file :drm: [[id:452b58ab-0881-43c4-ad0d-5d2d8df83370][linked id inexistant]]

[kjambunathan]

For the following .org file, I get this error: OpenDocument export failed: FIXME: Unable to resolve "todos.org"

+OPTIONS: broken-links:mark

+SELECT_TAGS: drm

* test linking to todos.org file                                                                  :drm:
  [[id:452b58ab-0881-43c4-ad0d-5d2d8df83370][linked id inexistant]]

I tried all values for `broken-links'. Above snippet exports along expected lines.

Broken link handling is done by the underlying export library ox and not ox-odt, and was part of this commit.

commit 59761024b17b3b71e70b6fde4f8580eb586d2f7b
Author: Nicolas Goaziou <mail@nicolasgoaziou.fr>
Date:   Sat Oct 10 16:03:05 2015 +0200

    ox: Add an option to ignore broken links

If you have recently upgraded your org or emacs ensure that you are NOT using the old ox. M-x load-library ox and M-x list-load-path-shadows are your friends.

[hpgisler]

Hmm, still no luck, I even updated to newest packages: [...] /home/hp/.emacs.d/elpa/org-20200713/ox hides /usr/share/emacs/26.3/lisp/org/ox [...] /home/hp/.emacs.d/elpa/ox-odt-9.3.7.319/ox-odt hides /home/hp/.emacs.d/elpa/org-20200713/ox-odt [...] /home/hp/.emacs.d/elpa/org-20200713/org hides /usr/share/emacs/26.3/lisp/org/org

Note: I have (org-reload) at the end of my .emacs - because some mess on dependencies inside. Could this be the problem?

[kjambunathan]

For the following .org file, I get this error: OpenDocument export failed: FIXME: Unable to resolve "todos.org"

+OPTIONS: broken-links:mark

+SELECT_TAGS: drm

* test linking to todos.org file                                                                  :drm:
  [[id:452b58ab-0881-43c4-ad0d-5d2d8df83370][linked id inexistant]]

Your snippet is missing a vital piece of information. I could infer what the missing piece is from the error string. Your snippet should have been this:

#+OPTIONS: broken-links:mark
#+SELECT_TAGS: drm

* Heading
  :PROPERTIES:
  :ID:       00ca6414-187b-491a-b56d-140cca83ae5a
  :END:

* test linking to todos.org file                                        :drm:

[[id:00ca6414-187b-491a-b56d-140cca83ae5a][Heading]]

i.e., for the bug to be triggered, the linked ID should exist, but the corresponding headline should be excluded from export.

There are two issues: one in ox.el and another in ox-odt.el

Issue with ox.el

This pertains to how org-export-resolve-id-link behaves

(defun org-export-resolve-id-link (link info)
  "Return headline referenced as LINK destination.

INFO is a plist used as a communication channel.

Return value can be the headline element matched in current parse
tree or a file name.  Assume LINK type is either \"id\" or
\"custom-id\".  Throw an error if no match is found."
  (let ((id (org-element-property :path link)))
    ;; First check if id is within the current parse tree.
    (or (org-element-map (plist-get info :parse-tree) 'headline
      (lambda (headline)
        (message "headline: %S" headline)
        (when (or (equal (org-element-property :ID headline) id)
              (equal (org-element-property :CUSTOM_ID headline) id))
          headline))
      info 'first-match)
    ;; Otherwise, look for external files.
    (cdr (assoc id (plist-get info :id-alist)))
    (signal 'org-link-broken (list id)))))

The above function tries to resolve an ID to a local headline. When no such headline exists, it resolves it to an "external" file that has this ID.

In the example case, the ID-ed headline doesn't exist. This is because it is tagged as "noexport" i.e., it is NOT selected for export. But a file containing the ID exists--and that file happens to be right this file. So, org export library says link to this file.

In your case, you will argue that link is broken. Because when you click on the link it would lead you nowhere. To appreciate this, look at the output from HTML engine

  <div id="outline-container-org751e1cc" class="outline-2">
    <h2 id="org751e1cc">
      <span class="section-number-2">1
      </span> test linking to todos.org file&#xa0;&#xa0;&#xa0;
      <span class="tag">
    <span class="drm">drm
    </span>
      </span>
    </h2>
    <div class="outline-text-2" id="text-1">
      <p>
    <a href="z.html#ID-00ca6414-187b-491a-b56d-140cca83ae5a">Heading
    </a>
      </p>
    </div>
  </div>

But the ox.el author seems to think that the behaviour you expect, though right in this case, may not carry over well broadly. i.e., He may argue that pruning IDs reported by org-id-find based on the export tags is not practical. He may feel that there is no fool-proof way of handling such IDs and that much would depend on the state of publishing system.

I don't use org publish much, so, at your discretion, you can debate your specific scenario with the org maintainers and arrive at a possible solution. (See Feedback - The Org Manual).

Issue with ox-odt.el

org-odt-link when resolving a ID link is NOT HANDLING the case where it is of type plaintext i.e., of type external file.

Summary

Issue#1 is beyond my control. I can fix Issue#2.

Additional note

If you are interested in taking up issue#1 with org maintainers... it is best to present the problem as a problem with HTML engine and NOT as one with the ODT exporter. This is because there is NO ONE in the mailing list who has interest / expertise in the ODT export engine.

[kjambunathan]

Hmm, still no luck, I even updated to newest packages: [...] /home/hp/.emacs.d/elpa/org-20200713/ox hides /usr/share/emacs/26.3/lisp/org/ox [...] /home/hp/.emacs.d/elpa/ox-odt-9.3.7.319/ox-odt hides /home/hp/.emacs.d/elpa/org-20200713/ox-odt [...] /home/hp/.emacs.d/elpa/org-20200713/org hides /usr/share/emacs/26.3/lisp/org/org

This seems OK.

Note: I have (org-reload) at the end of my .emacs - because some mess on dependencies inside. Could this be the problem?

Don't do org-reload.

org-reload seems to load the "features" from it's own directory. i.e., if you do org-reload the ox-odt that will be used is will be the one from the org ELPA dir and not from this repository's ELPA dir.

To confirm what org-reload does you can add the following advice

(advice-add #'load :before
        (defun my-load-advice (&rest args)
          (message "%S" args)))

and then do a M-x org-reload.

The above advice prints the arguments that org-reload passes to load.

When I do this on my machine, I get something like

("/home/kjambunathan/src/org-mode-ox-odt/lisp/ox-odt" noerror nil nil mustsuffix)
Loading /home/kjambunathan/src/org-mode-ox-odt/lisp/ox-odt (compiled; note, source file is newer)...done

Loading /home/kjambunathan/src/org-mode-ox-odt/lisp/org-version.el (source)...done
("/home/kjambunathan/src/org-mode-ox-odt/lisp/org" noerror nil nil mustsuffix)

In my case, I load both org and ox-odt from a github checkout my repo. So, the dir paths match. In your case, you will see org is re-loaded from org-20200713 (which is what you want) and ox-odt also from org-20200713 (which is what you DO NOT want). You want your ox-odt from ox-odt-9.3.7.319.

That log message should answer what is happening beneath the hood.

[kjambunathan]

Btw, org-reload is meant for use by developers. Unless you are hacking the org codebase, you most definitely DO NOT WANT that.

[hpgisler]

Wow, thank you for these detailed explenations: true: org-reload loaded again the ox-org from org-20200713! When removing org-reload however (i.e. correct ox-odt-9.3.7 is used) then I - yet - have an other problem:

Messages:

ox-odt: Content template file is /home/hp/.emacs.d/elpa/ox-odt-9.3.7.319/etc/styles/OrgOdtContentTemplate.xml
Using schema ~/.emacs.d/elpa/ox-odt-9.3.7.319/etc/schema/od-schema.rnc
Saving file /tmp/odt-zXXwTP/content.xml...
Wrote /tmp/odt-zXXwTP/content.xml
org-odt-template: OpenDocument export failed with error: ‘Cannot read styles file: /path/to/file/("/path/to/file/template_todo.odt" ("styles.xml" "Pictures/100000000000008C00000039914151311286766F.png"))’

(this obviously worked with org-20200713) - or has the format changed somehow? What I use is this:

#+ODT_STYLES_FILE: ("/path/to/file/template_todo.odt" ("styles.xml" "Pictures/100000000000008C00000039914151311286766F.png"))

[kjambunathan]

Just a quick update ...

When you export MY test snippet with MY repo, export to ODT goes fine.

But you will see an output like this

ODT output

ODT output after Tools->Update->Update All

When you export MY test snippet with 9.3.7 version of org

You will see the following error

OpenDocument export failed: FIXME: Unable to resolve "z.org"

My analysis of the bug stands. (I was switching between org repo and my own repo, and got a little bit disoriented when typing out my previous comment)

Btw, one good side-effect of this bug is that you can find out which ODT exporter you are using. If you are seeing the error message, then you are using org's ODT exporter (and NOT this exporter)

If you actually produce a ODT document (that looks like the one you see above), you are using this ODT exporter (and NOT org's ODT exporter)

After this comment ... I have NO DOUBT that it is the org-reload in your init file that overrides MY ODT exporter with org's own. So, remove org-reload, if you want to use MY ODT exporter.

[hpgisler]

Yes, I confirm: org-reload does load 9.3.7 version of org - which is not what I want. I removed org-reload from my .emacs However, as stated above, now this fails:

#+ODT_STYLES_FILE: ("/path/to/file/template_todo.odt")

with error: org-odt-template: OpenDocument export failed with error: ‘Cannot read styles file: /path/to/file/("/path/to/file/template_todo.odt" ....

[kjambunathan]

What I use is this:

#+ODT_STYLES_FILE: ("/path/to/file/template_todo.odt" ("styles.xml" "Pictures/100000000000008C00000039914151311286766F.png"))

I have introduced #+ODT_EXTRA_IMAGES for this purpose.

This is documented in the manual at https://kjambunathan.github.io/org-mode-ox-odt/Applying-custom-styles.html#ODT_005fMASTER_005fSTYLES-and-ODT_005fEXTRA_005fAUTOMATIC_005fSTYLES. The XML snippet in this page may confuse you. So, scroll down the HTML page until you see an image to get an idea of what the example is trying to do.

Essentially, you need something like

#+ODT_STYLES_FILE: /path/to/file/template_todo.odt
#+ODT_EXTRA_IMAGES: Pictures/100000000000008C00000039914151311286766F.png

You can see more examples in these links

1. https://github.com/kjambunathan/org-mode-ox-odt/commit/44a720dca7b647352eedcecf9f1120818058139e. The commit message has an example snippet.

2. https://github.com/kjambunathan/org-mode-ox-odt/issues/53#issuecomment-465620470. The original issue that motivated this feature.

[kjambunathan]

2. https://github.com/kjambunathan/org-mode-ox-odt/issues/53#issuecomment-465620470. The original issue that motivated this feature.

Unzip the file and goto xxx/test_page_styles_fixed. Locate the page_backgroud_test.org. It will look like below. The important thing to note is that instead of using a single ott file, or inlining of XML styles, all new styles.xml and content.xml are defined. Essentially, there are 3 flavours/ways of specifying the styles/template files.

#+ODT_EXTRA_IMAGES: backgrounds/Cover.jpg
#+ODT_EXTRA_IMAGES: backgrounds/Standard.jpg
#+ODT_EXTRA_IMAGES: backgrounds/Epilogue.jpg

#+odt_extra_styles: <draw:fill-image draw:name="OrgTitlePage"
#+odt_extra_styles: xlink:href="backgrounds/Cover.jpg"/>
#+odt_extra_styles: <draw:fill-image draw:name="Standard"
#+odt_extra_styles: xlink:href="backgrounds/Standard.jpg"/>
#+odt_extra_styles: <draw:fill-image draw:name="EpiloguePage"
#+odt_extra_styles: xlink:href="backgrounds/Epilogue.jpg"/>
#+options:  toc:nil author:nil tex:t latex:t :H 10
#+language: zh-CN
#+odt_display_outline_level:  0
#+odt_styles_file: "user/presentation_cn_big.xml"
#+odt_content_template_file: "user/default_template_big.xml"

#+attr_odt: :page-style "OrgTitlePage" :page-break t
org title page

#+attr_odt: :page-style "StandardPage" :page-break t
normal page

#+attr_odt: :page-style "EpiloguePage" :page-break t
last page

[kjambunathan]

+ODT_EXTRA_IMAGES: backgrounds/Cover.jpg

+ODT_EXTRA_IMAGES: backgrounds/Standard.jpg

+ODT_EXTRA_IMAGES: backgrounds/Epilogue.jpg

You can have more than one background image. Just include a line for each of the background images.

If the image is from your ott file (instead of from the file path as in this example), then you unzip the ott file and copy the image path AS IT IS in the ott file.

Let me know if you have confusion.

---

There is an ODT info manual in your ELPA download dir.

Just do something like C-u C-h i /home/kjambunathan/.emacs.d/elpa/ox-odt-9.3.6.315/docs/org-odt and browse to whatever node you want.

[kjambunathan]

Essentially, you need something like

#+ODT_STYLES_FILE: /path/to/file/template_todo.odt
#+ODT_EXTRA_IMAGES: Pictures/100000000000008C00000039914151311286766F.png

If the image is in the odt (or ott) file already, you only have to specify the image path. The ODT exporter is smart enough to unzip the template file, and locate the image base on the path information, and copy it over to the newly exported file.

What I am saying is this: You don't have to have a physical copy of the image in your filesystem. It suffices if it is within the ODT/OTT file.

[hpgisler]

Thank's a LOT.

This works for me now:

#+ODT_STYLES_FILE: /path/to/file/template_todo.odt
#+ODT_EXTRA_IMAGES: Pictures/100000000000008C00000039914151311286766F.png