kjambunathan / org-mode-ox-odt

The Authoritative fork of Org mode's ODT exporter
GNU General Public License v3.0
45 stars 9 forks source link

The UpdateAll macro freezes for some large files #53

Closed QiangF closed 5 years ago

QiangF commented 5 years ago

I still need to find a good example. But for some files, the exported odt freezes libreoffice when opened. I need to do a conversion with unoconv to make the file openable:

unoconv -f odt -o output.odt input.odt

Maybe the macro can be replaced with a python script with the uno interface.

kjambunathan commented 5 years ago

Please use LibreOffice/OrgModeUtilities.oxt from ox-odt-9.1.14.169

unoconv -f odt -o output.odt input.odt

Here is the link to code that updates Indices: https://github.com/dagwieers/unoconv/blame/6413cb8378c278fb18b70eabae157de277a602f6/unoconv#L1044

Maybe the macro can be replaced with a python script with the uno interface.

I don't want to create an additional dependency on unoconv, for now.

QiangF commented 5 years ago

The new macro works perfect so far!

QiangF commented 5 years ago

Unoconv saves my day once more. Recently I have found that Libreoffice > 6.1 can't show the background image of my export, and unoconv conversion fix that. I use backgroud image to insert complex powerpoint page styles as images. These images are inserted in the beginning of the document:

#+odt_extra_styles: <draw:fill-image draw:name="OrgTitlePage"
#+odt_extra_styles: xlink:href="file:///home/q/.dotfiles/emacs/org/ti1/Cover.png"/>
#+odt_extra_styles: <draw:fill-image draw:name="Standard"
#+odt_extra_styles: xlink:href="file:///home/q/.dotfiles/emacs/org/ti1/Standard.png"/>
#+odt_extra_styles: <draw:fill-image draw:name="EpiloguePage"
#+odt_extra_styles: xlink:href="file:///home/q/.dotfiles/emacs/org/ti1/Epilogue.png"/>

The in the style file, I create a page layout style like this.

<!-- OrgTitlePage -->
<style:page-layout style:name="Mpm2">
    <style:page-layout-properties fo:page-width="25.4cm"
                                    fo:page-height="19.0cm"
                                    style:num-format="1"
                                    style:print-orientation="landscape"
                                    fo:margin-top="0cm" fo:margin-bottom="0cm"
                                    fo:margin-left="0cm" fo:margin-right="0cm"
                                    fo:border="0.02pt solid #FFFFFF"
                                    fo:padding="0.7cm"
                                    style:shadow="none"
                                    style:writing-mode="lr-tb"
                                    draw:fill="bitmap" draw:fill-image-name="OrgTitlePage"
                                    draw:fill-image-width="100%"
                                    draw:fill-image-height="100%"
                                    style:footnote-max-height="0cm">
    </style:page-layout-properties>
    <style:columns fo:column-count="1" fo:column-gap="0cm"/>
    <style:header-style/>
    <style:footer-style/>
</style:page-layout>

and the corresponding master page style.

<style:master-page style:name="OrgTitlePage" style:next-style-name="StandardPage" style:page-layout-name="Mpm2"/>

Since the definition of styles in the later part of the styles.xml overwrite previous definition. I have some duplication of style definitions (I keep style snippets in different files, and make combinations using a script). Unoconv seems to serve as a xml cleaning tool.

kjambunathan commented 5 years ago

First export the below snippet using ox-odt-9.2.1.180.tar. You should get a document whose first page will have a yellow background. This is accomplised with two new options #+ODT_EXTRA_AUTOMATIC_STYLES: ... and #+ODT_MASTER_STYLES: ...: The first option creates new page layout ,while the second option creates a new page style.

#+OPTIONS: ':nil *:t -:t ::t <:t H:3 \n:nil ^:t arch:headline
#+OPTIONS: author:nil broken-links:nil c:nil creator:nil
#+OPTIONS: d:(not "LOGBOOK") date:nil e:t email:nil f:t inline:t num:t
#+OPTIONS: p:nil pri:nil prop:nil stat:t tags:t tasks:t tex:t
#+OPTIONS: timestamp:t title:nil toc:nil todo:t |:t
#+TITLE: How to customize page styles and layouts
#+AUTHOR: Jambunathan K

#+ODT_EXTRA_AUTOMATIC_STYLES: <style:page-layout style:name="MyFirstPageLayout">
#+ODT_EXTRA_AUTOMATIC_STYLES:  <style:page-layout-properties fo:page-width="21.001cm" fo:page-height="29.7cm" style:num-format="1" style:print-orientation="portrait" fo:margin-top="2cm" fo:margin-bottom="2cm" fo:margin-left="2cm" fo:margin-right="2cm" fo:background-color="#ffff00" style:writing-mode="lr-tb" style:layout-grid-color="#c0c0c0" style:layout-grid-lines="20" style:layout-grid-base-height="0.706cm" style:layout-grid-ruby-height="0.353cm" style:layout-grid-mode="none" style:layout-grid-ruby-below="false" style:layout-grid-print="false" style:layout-grid-display="false" draw:fill="solid" draw:fill-color="#ffff00" style:footnote-max-height="0cm">
#+ODT_EXTRA_AUTOMATIC_STYLES:   <style:footnote-sep style:width="0.018cm" style:distance-before-sep="0.101cm" style:distance-after-sep="0.101cm" style:line-style="solid" style:adjustment="left" style:rel-width="25%" style:color="#000000"/>
#+ODT_EXTRA_AUTOMATIC_STYLES:  </style:page-layout-properties>
#+ODT_EXTRA_AUTOMATIC_STYLES:  <style:header-style/>
#+ODT_EXTRA_AUTOMATIC_STYLES:  <style:footer-style/>
#+ODT_EXTRA_AUTOMATIC_STYLES: </style:page-layout>

#+ODT_MASTER_STYLES: <style:master-page style:name="MyFirstPage" style:page-layout-name="MyFirstPageLayout" style:next-style-name="OrgFrontMatterPage"/>

#+ATTR_ODT: :style "OrgCenter" :page-style "MyFirstPage" :page-break t
This page has yellow background.
kjambunathan commented 5 years ago

I use backgroud image in page styles

The recommended way to achieve this is documented in The Org Manual: Applying custom styles, see item (2). The item (2) in that node lacks details and is of little use to anyone. The item (2) was added in response to a request I received in the Org-mode mailing list around March 2011 (see [O] Re: org-odt (Applying custom templates) and is one of the earlier features I added to the ODT exporter. What I am saying is this: The feature you are requesting has been in the exporter for good 8 years now. Unfortunately, it is poorly documented.

This is what you need to do:

Download the fileset from issue53-attachment-1.zip This link has 3 files (1) template.ott (2) pagestyles.org and (3) pagestyles.odt. (1) & (2) are input files. (3) is the output file.

  1. template.ott was created by hand. The OrgTitlePage of this template uses a tiled image as it's background.

  2. The file pagestyles.org uses the template.ott as the styles file. Since the style relies on an image, the #+ODT_STYLES_FILE specifies this image as well.

This is how pagestyles.org looks like:

#+OPTIONS: ':nil *:t -:t ::t <:t H:3 \n:nil ^:t arch:headline
#+OPTIONS: author:nil broken-links:nil c:nil creator:nil
#+OPTIONS: d:(not "LOGBOOK") date:nil e:t email:nil f:t inline:t num:t
#+OPTIONS: p:nil pri:nil prop:nil stat:t tags:t tasks:t tex:t
#+OPTIONS: timestamp:t title:nil toc:nil todo:t |:t
#+TITLE: How to customize page styles and layouts
#+AUTHOR: Jambunathan K

#+ODT_STYLES_FILE: ("template.ott" ("styles.xml" "Pictures/10000201000000A2000000B0B55B7ABDC4CED431.png"))

#+ATTR_ODT: :style "OrgCenter" :page-style "OrgTitlePage" :page-break t
Phasellus purus.  Nunc eleifend leo vitae magna.  Suspendisse potenti.
Vivamus id enim.  Suspendisse potenti.
  1. pagestyle.odt: This is the file you get when you export the above pagestyles.org. Note that this file inherit the styles in template.ott. Specifically, it will have a title page that uses a tiled image as a background (just like the the template.ott file).
kjambunathan commented 5 years ago

Unoconv saves my day once more. Recently I have found that Libreoffice > 6.1 can't show the background image of my export, and unoconv conversion fix that.

I cannot validate this on my side

I created a MWE (minimal work example) to validate your claim.

The files are available at issue53-attachment-2.zip. The fileset has two files issue53.zip and readme.txt. Start with readme.txt.

Contents of readme.txt

Summary: I am not able to validate your claim that `unoconv` does the
right thing.

I am using these packages (I run Debian Unstable)

$ dpkg -l | grep writer
ii  libreoffice-writer 1:6.1.5-1 amd64        office productivity suite -- word processor

$ dpkg -l | grep unoconv 
ii  unoconv 0.7-1.1 all converter between LibreOffice document formats

1. Unzip issue53.zip

2. In the input file `pagestyles-1.org`, fix the path
   "/home/kjambunathan/Downloads/issue53/Pictures/10000201000000A2000000B0B55B7ABDC4CED431.png"
   as appropriate.

   Note that the #+ODT_*_STYLES in this file are verbatim copies of
   XML found in the `template.ott` file attached in previous comment
   (see https://github.com/kjambunathan/org-mode-ox-odt/issues/53).
   (Only the style names are changed so that they don't conflict with
   existing styles)

3. Export it
4. `pagestyles-1.odt` is the output file.  (As you say it doesn't show the backgroud image)
5. Run unoconv -f odt -o pagestyles-1-output.odt pagestyles-1.odt 
6. `pagestyles-1-output.odt` still doesn't show the background image.
   (i.e., I cannot reproduce what you claim about `unoconv` at your
   end.)
7. Inflate `pagestyles-1-output.odt`. You will see that the
   `Pictures/10000201000000A2000000B0B55B7ABDC4CED431.png` is absent
   in this `*.odt` file.  i.e., There is *no* ground to expect that
   `pagestyles-1-output.odt` will display a background image.  (Even
   if LibreOffice picks up the file from your local system, and
   displays the background image in *your* computer, the receiver will
   still be unable to see the background image on *his* computer.)

Contents of pagestyles-1.org


#+OPTIONS: ':nil *:t -:t ::t <:t H:3 \n:nil ^:t arch:headline
#+OPTIONS: author:nil broken-links:nil c:nil creator:nil
#+OPTIONS: d:(not "LOGBOOK") date:nil e:t email:nil f:t inline:t num:t
#+OPTIONS: p:nil pri:nil prop:nil stat:t tags:t tasks:t tex:t
#+OPTIONS: timestamp:t title:nil toc:nil todo:t |:t
#+TITLE: How to customize page styles and layouts
#+AUTHOR: Jambunathan K

#+OPTIONS: tex:t
#+ODT_FILE_EXTENSION: odt | odm
#+ODT_STYLES_FILE:
#+ODT_EXTRA_STYLES:
#+ODT_EXTRA_AUTOMATIC_STYLES:
#+ODT_MASTER_STYLES:
#+ODT_CONTENT_TEMPLATE_FILE:
#+ODT_AUTOMATIC_STYLES:
#+ODT_DISPLAY_OUTLINE_LEVEL: 2
#+BIB_FILE:

#+ODT_EXTRA_STYLES: <draw:fill-image draw:name="org-mode-unicorn" xlink:href="/home/kjambunathan/Downloads/issue53/Pictures/10000201000000A2000000B0B55B7ABDC4CED431.png" xlink:type="simple" xlink:show="embed" xlink:actuate="onLoad"/>

#+ODT_EXTRA_AUTOMATIC_STYLES: <style:page-layout style:name="MyTitlePageLayout">
#+ODT_EXTRA_AUTOMATIC_STYLES:  <style:page-layout-properties fo:page-width="21.001cm" fo:page-height="29.7cm" style:num-format="1" style:print-orientation="portrait" fo:margin-top="2cm" fo:margin-bottom="2cm" fo:margin-left="2cm" fo:margin-right="2cm" style:writing-mode="lr-tb" style:layout-grid-color="#c0c0c0" style:layout-grid-lines="20" style:layout-grid-base-height="0.706cm" style:layout-grid-ruby-height="0.353cm" style:layout-grid-mode="none" style:layout-grid-ruby-below="false" style:layout-grid-print="false" style:layout-grid-display="false" draw:fill="bitmap" draw:fill-image-name="org-mode-unicorn" draw:fill-image-width="0cm" draw:fill-image-height="0cm" style:repeat="repeat" draw:fill-image-ref-point-x="0%" draw:fill-image-ref-point-y="0%" draw:fill-image-ref-point="center" draw:tile-repeat-offset="0% vertical" style:footnote-max-height="0cm">
#+ODT_EXTRA_AUTOMATIC_STYLES:   <style:background-image xlink:href="Pictures/10000201000000A2000000B0B55B7ABDC4CED431.png" xlink:type="simple" xlink:actuate="onLoad"/>
#+ODT_EXTRA_AUTOMATIC_STYLES:   <style:footnote-sep style:width="0.018cm" style:distance-before-sep="0.101cm" style:distance-after-sep="0.101cm" style:line-style="solid" style:adjustment="left" style:rel-width="25%" style:color="#000000"/>
#+ODT_EXTRA_AUTOMATIC_STYLES:  </style:page-layout-properties>
#+ODT_EXTRA_AUTOMATIC_STYLES:  <style:header-style/>
#+ODT_EXTRA_AUTOMATIC_STYLES:  <style:footer-style/>
#+ODT_EXTRA_AUTOMATIC_STYLES: </style:page-layout>

#+ODT_MASTER_STYLES:  <style:master-page style:name="MyTitlePage" style:page-layout-name="MyTitlePageLayout" style:next-style-name="OrgFrontMatterPage"/>

#+ATTR_ODT: :style "OrgCenter" :page-style "MyTitlePage" :page-break t
Phasellus purus.  Nunc eleifend leo vitae magna.  Suspendisse potenti.
Vivamus id enim.  Suspendisse potenti.
kjambunathan commented 5 years ago

If I haen't understood the issue properly, please share sample test files (and instructions) so that I can understand what you say. I have done my best with whatever instructions you have given me. Unfortunately, I am unable to reproduce the issue.

kjambunathan commented 5 years ago

Btw, the XML that LibreOffice emits for in page layout is NOT ODF-1.2 compliant. i.e., LibreOffice uses it's own extension on top ODF-1.2 spec. In essence, the ODT document you create can potentially break if it is opened in a different* application (say Microsoft, or AbiWord).

Here is a screenshot of what nxml-mode's tells me:

screenshot from 2019-02-18 21-15-31

QiangF commented 5 years ago

I have uploaded an example at test_page_styles.zip Please extract the folder to /tmp, the odt file is the exported file of page_backgroud_test.org, you can see the different rendering effect in these two pdf files. update_styles.py is the script I use to make style files. The draw back of using template.ott as style file is the human eye lacks precision, so what you see probably is not what you want. Also it lacks reusability, say if you want to copy a style from one ott file. So I make some reusable units in snippets folder, and the templates makes combinations of these units, the generated style files are in the user folder.

To make slide using the odt exporter like beamer in latex, I miss the most is a frame environment, an environment that can contain all types (heading, paragraph, table, frame, text box ...). The minimum attribute a frame enviroment should have are width, anchoring position, masterpage specification, page-break. Another feature that is nice to have is a sub figure environment, like in latex, a captioned figure with captioned subfigures.

Thank you for making this exporter even better!

kjambunathan commented 5 years ago

I have uploaded an example, http://www.fileconvoy.com/dfl.php?id=g5fcb4d8095dd954e10001505424bfeb94f0af02471

Github doesn't allow uploading of OpenDocument files. But it allows uploading of zip files. So, if we want to share OpenDocument files in the future, we can zip it and attach it to github itself. This way we don't have to bother about come-today-leave-tomorrow file upload sites.

I have put your files under test_page_styles.zip. Github allows me to edit your replies. So, I have replaced the link to fileconvoy with this link.

kjambunathan commented 5 years ago

Test case for the above commit how-to-use-odt-extra-images.zip

kjambunathan commented 5 years ago

I have uploaded xxx.zip. The directory test_page_styles_notok contains the files that you shared with me. The directorytest_page_styles_fixed contains my own changes to the above files that accomplishes what you have in mind. The changes are:

  1. Replace absolute paths with relative paths
  2. Embed the background images to the exported file.

That is all.


$ diff -u -r test_page_styles_notok test_page_styles_fixed/
Binary files test_page_styles_notok/page_backgroud_test.odt and test_page_styles_fixed/page_backgroud_test.odt differ
diff -u -r test_page_styles_notok/page_backgroud_test.org test_page_styles_fixed/page_backgroud_test.org
--- test_page_styles_notok/page_backgroud_test.org  2019-02-20 06:39:55.000000000 +0530
+++ test_page_styles_fixed/page_backgroud_test.org  2019-02-20 20:34:16.449399524 +0530
@@ -1,14 +1,18 @@
+#+ODT_EXTRA_IMAGES: backgrounds/Cover.jpg
+#+ODT_EXTRA_IMAGES: backgrounds/Standard.jpg
+#+ODT_EXTRA_IMAGES: backgrounds/Epilogue.jpg
+
 #+odt_extra_styles: <draw:fill-image draw:name="OrgTitlePage"
-#+odt_extra_styles: xlink:href="file:///tmp/test_page_styles/backgrounds/Cover.jpg"/>
+#+odt_extra_styles: xlink:href="backgrounds/Cover.jpg"/>
 #+odt_extra_styles: <draw:fill-image draw:name="Standard"
-#+odt_extra_styles: xlink:href="file:///tmp/test_page_styles/backgrounds/Standard.jpg"/>
+#+odt_extra_styles: xlink:href="backgrounds/Standard.jpg"/>
 #+odt_extra_styles: <draw:fill-image draw:name="EpiloguePage"
-#+odt_extra_styles: xlink:href="file:///tmp/test_page_styles/backgrounds/Epilogue.jpg"/>
+#+odt_extra_styles: xlink:href="backgrounds/Epilogue.jpg"/>
 #+options:  toc:nil author:nil tex:t latex:t :H 10
 #+language: zh-CN
 #+odt_display_outline_level:  0
-#+odt_styles_file: "/tmp/test_page_styles/user/presentation_cn_big.xml"
-#+odt_content_template_file: "/tmp/test_page_styles/user/default_template_big.xml"
+#+odt_styles_file: "user/presentation_cn_big.xml"
+#+odt_content_template_file: "user/default_template_big.xml"

 #+attr_odt: :page-style "OrgTitlePage" :page-break t
 org title page
kjambunathan commented 5 years ago

file:///tmp/test_page_styles/backgrounds/Cover.jpg

Why are using absolute paths? What if you mail the ODT file to someone else and he opens it? Obviously, it won't work. (From what you say, it looks like you are not interested in ODT files themeselves, but only the PDF file produced from the ODT files. If this the case i.e., if you have never shared your ODT file with others, you would have never noticed this problem.

Btw, I cannot confirm if there is a change in behaviour of LibreOffice between 6.0 and 6.2. Even if there is a change, it should pertain to how LO handles absolute image paths in styles.xml.

kjambunathan commented 5 years ago

Do you still think that there unoconv does some thing which my own Basic macros doesn't do? If yes, I would need concrete examples to investigate it.

kjambunathan commented 5 years ago

Please use ox-odt-9.2.1.182.tar: Embed additional images with ODT_EXTRA_IMAGES

kjambunathan commented 5 years ago

To make slide using the odt exporter like beamer in latex, I miss the most is a frame environment, an environment that can contain all types (heading, paragraph, table, frame, text box ...). The minimum attribute a frame enviroment should have are width, anchoring position, masterpage specification, page-break.

Another feature that is nice to have is a sub figure environment, like in latex, a captioned figure with captioned subfigures.

Could you please give me sample ODT files for the above two cases. I am confident that these can be achieved with minimal efforts.

QiangF commented 5 years ago

The example files is inside suggestions.zip Please read readme.org first. Thank you!

kjambunathan commented 5 years ago

Since you mention Beamer, I have a feeling that you aren't interested in Writer / ODT document but Impress / Presentation documents. Is my understanding right?

QiangF commented 5 years ago

No, impress is overkill for me. For this exporter, I think most people don't need more than a subfigure environment.

QiangF commented 5 years ago

Sorry, the table cell format is caused by the style file. So what I suggest is:

  1. Solve the issue: the figure anchor is the figure frame anchor, the image is not anchored as char.
  2. A wishlist: a subfigure environment using a table.
QiangF commented 5 years ago

It will be very helpful to use absolute path in

+ODT_EXTRA_IMAGES:, otherwise I need to copy my backgrounds

image to the current folder every time. Why not put all the images inside a folder named "odt-extra-images" inside the current odt file. I have make a change in ox-odt.el and it works

(dolist (path (plist-get info :odt-extra-images))
    (let* ((input-dir (file-name-directory (plist-get info :input-file)))
        (full-path (if (file-name-absolute-p path) path
                        (expand-file-name path input-dir)))
        (target-path (concat (directory-file-name "odt-extra-images") "/" (file-name-nondirectory path))))
    (org-odt--copy-image-file full-path target-path)))
kjambunathan commented 5 years ago

ox-odt-9.2.1.187.tar: Support for multiple captioned images in a para…

Sample Snippet

#+ATTR_ODT: :style "OrgCenter"
#+begin_paragraph
    #+ATTR_ODT: :width 5 :anchor "as-char"
    #+CAPTION: First Figure
    [[./org-mode-unicorn.png]]

    #+ATTR_ODT: :width 5 :anchor "as-char"
    #+CAPTION: Second Figure
    [[./org-mode-unicorn.png]]

    #+ATTR_ODT: :width 10 :anchor "as-char"
    #+CAPTION: Third Figure
    [[./org-mode-unicorn.png]]
#+end_paragraph

Corresponding LibreOffice Output

screenshot from 2019-02-26 15-11-43

kjambunathan commented 5 years ago

ox-odt-9.2.1.187.tar: Support for multiple captioned images in a para…

I have introduce an incompatible change.

From now on DO NOT double-quote the style files. i.e.,

Wrong

#+odt_styles_file:      "user/presentation_cn_big.xml"
#+odt_content_template_file:    "user/default_template_big.xml"

Right

#+odt_styles_file:       user/presentation_cn_big.xml
#+odt_content_template_file:     user/default_template_big.xml

(I believe) this is what the upstream `Org' exporter does.

kjambunathan commented 5 years ago

I have added support for subfigures.

In the example below, pay particular attention to these things:

If you want more details look at the commit log and comments in file ox-odt.el.

As a side-note, to support the above feature I had to rework almost all parts of the code that deals with captions and caption numbers. Because of these elaborate changes, I might have introduced bugs. So, please keep an extra eye on captions & numbers in the output, and report any unexpected results.

Org Snippet

#+TOC: figures
#+TOC: tables

#+NAME: table
#+CAPTION: Animals
#+ATTR_ODT: :category "figure"
#+ATTR_ODT: :list-table t
-
  -
    #+NAME: dog
    #+CAPTION: A Dog
    [[./org-mode-unicorn.png]]
  -
    #+NAME: goat
    #+CAPTION: A Goat
    [[./org-mode-unicorn.png]]

#+CAPTION: A Unicorn
[[./org-mode-unicorn.png]]

LibreOffice Output

screenshot from 2019-03-03 17-43-04

kjambunathan commented 5 years ago

Btw, use ox-odt-9.2.1.205.tar: Support for subfigures etc.