kjambunathan / org-mode-ox-odt

The Authoritative fork of Org mode's ODT exporter
GNU General Public License v3.0
45 stars 9 forks source link

Filename encoding issues in org-odt-zip on windows 11 chinese language #263

Closed QiangF closed 8 months ago

QiangF commented 8 months ago

This might be an issue for all windows that has a non-english system language. The solution is just remove (coding-system-for-write 'no-conversion) in org-odt-zip.

kjambunathan commented 8 months ago

This might be an issue for all windows that has a non-english system language. The solution is just remove (coding-system-for-write 'no-conversion) in org-odt-zip.

The zip uses call-process , and if I understand correctly it is very much like running a shell command but without any shell quoting. So, I believe that coding-sytem-for-write may affect what gets displayed in *Messages* buffer, but not the ODT file itself.

Could you please attach a OK zip file, and a not OK zip file ....

I remember reading somewhere that the FILE NAMEs themselves uses a specific encoding, do a M-x apropos-variable file-name-coding-system ... I believe the files themselves may be encoded right (using utf-8) but the file names within the zip file, and the name of the zip file itself, uses an improper coding system for file name, and that screws up things.

Try enclosing the problem zip file, and give the values of various coding systems that you use on the system.

If you are adventurous, You can also add a debug before the call-process, and when the Emacs is suspended get in to the CLI shell and do the following two commands to zip up the file as odt. This way you can see if zipping up with call-process gives different results from zipping up with manually running commands.

     (cmds `(("zip" "-mX0" ,target-name "mimetype")
         ("zip" "-rmTq" ,target-name ".")))

I need more information, before I can understand what the problem is. My gut feeling is the coding-system-for-write may not be a problem, but your file name coding system could be a problem.

kjambunathan commented 8 months ago

I just filtered on variables in Emacs manual that have the string coding in them

(
 buffer-file-coding-system
 coding-system-for-read
 coding-system-for-write
 coding-system-require-warning
 default-process-coding-system
 file-coding-system-alist
 file-name-coding-system
 last-coding-system-used
 locale-coding-system
 mode-line-coding-system-map
 network-coding-system-alist
 process-coding-system-alist
 save-buffer-coding-system
 select-safe-coding-system-accept-default-p
 select-safe-coding-system-function
 selection-coding-system
 )

I suggest that you focus on values of coding system vars which have file-name or process in them. I think coding-system-for-write is the wrong coding system to look at

I also suggest that you do describe-coding-system in Windows machines that have problems and do not have problems. Just run a diff on these outputs.

file-name-coding-system is a variable defined in ‘C source code’.

Its value is nil

Coding system for encoding file names.
If it is nil, ‘default-file-name-coding-system’ (which see) is used.

On MS-Windows, the value of this variable is largely ignored if
‘w32-unicode-filenames’ (which see) is non-nil.  Emacs on Windows
behaves as if file names were encoded in ‘utf-8’.

  Probably introduced at or before Emacs version 20.1.

[back]

See if the value of w32-unicode-filenames has any effect

It is important which process is displaying the output of the zip file. If LO is reporting corrupted file, then the zip is invoked with wrong process coding system (may be)

If its emacs archive-mode that zips and unzips the file, then all the coding systems may come in to play.

In other words, experiment and see if you can get better handle on the problem.

kjambunathan commented 8 months ago

Your issue itself says it is issue with file names, and NOT the issue with encoding of the file-s themselves. So, focus on file name encoding and not file encoding. There are good Chinese hackers; before you start describing the problem they will know where the fix may lie. Just put the question or describe your problem on Emacs China may be.

(I don't have Windows machines, and even if I change the system locale of my debian machine to Chinese, and I would have no clue to make a head or tail ... So I would appreciate the Chinese speakers are better equipped to experiment and figure where the problem is and how the fix may look like.

QiangF commented 8 months ago

I don't have the windows computer on hand. I already have this setting.

(set-language-environment "UTF-8")
(setq w32-unicode-filenames '())
(set-file-name-coding-system 'gbk-dos)
(modify-coding-system-alist 'process "*" '(utf-8 . gbk-dos))

Exactly the issue is the filename. If I keep (coding-system-for-write 'no-conversion) in org-odt-zip, file name is screwed up. And the exportor complains that the file can't be found. Actually it is there but a name that not in utf-8 encoding.

kjambunathan commented 8 months ago

I want to collect as much info as possible ...

  1. What is the version of zip you are using.
  2. What is system-type and system-configuration
  3. The output of M-x describe-coding-system
  4. The output of M-x describe-language-environment
  5. The different coding-system specific values that you set explicitly in your init file
    (set-language-environment "UTF-8")
    (setq w32-unicode-filenames '())
    (set-file-name-coding-system 'gbk-dos)
    (modify-coding-system-alist 'process "*" '(utf-8 . gbk-dos))

See if changing the file name with recode-file-name helps

recode-file-name is an interactive byte-compiled Lisp function in files.el.


(recode-file-name FILE CODING NEW-CODING &optional
OK-IF-ALREADY-EXISTS)

Change the encoding of FILE’s name from CODING to NEW-CODING. The value is a new name of FILE. Signals a file-already-exists error if a file of the new name already exists unless optional fourth argument OK-IF-ALREADY-EXISTS is non-nil. A number as fourth arg means request confirmation if the new name already exists. This is what happens in interactive use with M-x.

Probably introduced at or before Emacs version 22.1.

[back]


As a Chinese user, you can experiment with following complex scenarios,

  1. Give "problematic" Chinese names to ODT_STYLES_FILE, ODT_CONTENT_TEMPLATE_FILE , and ODT_EXTRA_IMAGES. Note that the ODT_STYLES_FILE can specify header/footer images file names to be extracted out of a odt file. So give "complicated" chinese names to the image file names that needs to be pulled away from an existing ott file.
  2. unzip a odt file created by exporter using dired and see if browsing the file names within the odt archive creates issues.
  3. Open the zip file created by odt exporter in LibreOffice.

IIUC, the problem is with the name of the odt file created with call-process on zip. (I have reasons to believe that coding-system-for-write may influence how the arguments to the zip program (note that, one of the arguments is the name of the odt file) could be encoded on its way to the zip program.

Now that we know that coding-system-for-write can include file name encoding / decoding, we know that there are many places--particularly where the "standard" content.xml, styles.xml are produced the coding-system-for-write is set to utf-8. I hope these make-shift bindings doesn't clobber up the file name themselves.


Since this is a file name encoding / decoding issue, there are multiple file names involved ....

  1. The file name of the odt file
  2. The file names that are WITHIN the odt file, like content.xm, meta.xml etc etc. I don't know if these ASCII file names content.xml, meta.xml, style.xml, manifest.xml, mimetype etc create issues with non-ASCII systems like Chinese. _(The only case where files with probematic names may end up in the odt zip archive is when ODT_EXTRAIMAGES get used)
  3. The behaviour of zip and unzip program (as invoked by archive-mode) when they pack and unpack odt files using dired commands.
  4. The other place where file name comes in to picture is as part of ODT_STYLES_FILE (this can specify header images to be extracted from a template file, and be included in the odt file created by the exporter) and ODT_CONTENT_TEMPLATE_FILE


Not a solution but some remarks for future reference ...

I believe there are too many moving parts ... I was wondering how coding-system-for-write affects file name encodings. This comment by Eli and a cursory look at the C-code for call-process suggests that the value of coding-system-for-write can influence the value that goes in to argument-coding which gets subsequently used as part of encode-coding-string. The presence of [find-operation-coding-system for call-process] suggests that the enter for the process (zip in our case) and (git in the afore-mentioned issue with magit particularly this remark about why rg has "strange" process encodings) will also influence how args to call-process gets encoded.

find-operation-coding-system is a built-in function in src/coding.c.


(find-operation-coding-system OPERATION ARGUMENTS...)

Choose a coding system for an operation based on the target name. The value names a pair of coding systems: (DECODING-SYSTEM . ENCODING-SYSTEM). DECODING-SYSTEM is the coding system to use for decoding (in case OPERATION does decoding), and ENCODING-SYSTEM is the coding system for encoding (in case OPERATION does encoding).

The first argument OPERATION specifies an I/O primitive: For file I/O, insert-file-contents or write-region. For process I/O, call-process, call-process-region, or start-process. For network I/O, open-network-stream.

The remaining arguments should be the same arguments that were passed to the primitive. Depending on which primitive, one of those arguments is selected as the TARGET. For example, if OPERATION does file I/O, whichever argument specifies the file name is TARGET.

TARGET has a meaning which depends on OPERATION: For file I/O, TARGET is a file name (except for the special case below). For process I/O, TARGET is a process name. For network I/O, TARGET is a service name or a port number.

This function looks up what is specified for TARGET in file-coding-system-alist, process-coding-system-alist, or network-coding-system-alist depending on OPERATION. They may specify a coding system, a cons of coding systems, or a function symbol to call. In the last case, we call the function with one argument, which is a list of all the arguments given to this function. If the function can’t decide a coding system, it can return undecided so that the normal code-detection is performed.

If OPERATION is insert-file-contents, the argument corresponding to TARGET may be a cons (FILENAME . BUFFER). In that case, FILENAME is a file name to look up, and BUFFER is a buffer that contains the file’s contents (not yet decoded). If file-coding-system-alist specifies a function to call for FILENAME, that function should examine the contents of BUFFER instead of reading the file.

Probably introduced at or before Emacs version 22.1.

[back]


M-x man on zip has the following remarks

Unicode.  Though  the zip standard  requires storing paths  in an
archive using  a specific  character set,  in practice  zips have
stored paths in archives in  whatever the local character set is.
This creates problems when an archive  is created or updated on a
system  using one  character set  and then  extracted on  another
system  using  a different  character  set.   When compiled  with
Unicode   support  enabled   on  platforms   that  support   wide
characters, zip  now stores,  in addition  to the  standard local
path  for backward  compatibility, the  UTF-8 translation  of the
path.  This provides a common universal character set for storing
paths  that allows  these paths  to be  fully extracted  on other
systems that support Unicode and to match as close as possible on
systems that don't.

On Win32 systems where paths are internally stored as Unicode but
represented in the  local character set, it's  possible that some
paths  will be  skipped during  a local  character set  directory
scan.   zip with  Unicode support  now can  read and  store these
paths.  Note that Win 9x systems and FAT file systems don't fully
support Unicode.

Be aware  that console  windows on Win32  and Unix,  for example,
sometimes don't  accurately show all  characters due to  how each
operating  system   switches  in  character  sets   for  display.
However, directory navigation tools should show the correct paths
if the needed fonts are loaded.

This issue Proper encoding for file names in zip archives created in Windows and unpacked in linux - Super User suggests that there different zip programs behave differently when it comes to encoding file name within the archive. There is also bug#52480: 28.0.60; Emacs Zip-Archive open inside image file does not di which talk about how zip programs encode file names within the zip archive.


kjambunathan commented 8 months ago

I know for a fact that @PierreTechoueyres at one point in time, in his own for of org-mode-ox-odt had a non-submitted patch for issues with FILE-NAME-ENCODING. Pierre never brought it to my attention; But as an author who was curious about what various people who fork my repo do, I noted this local change. Unfortunately, I don't see his change any longer. And I believe he is working around this issue by choosing a ASCII-like file name for his ODT files.

I will be happy if @PierreTechoueyres is kind enough to comment on what his original problem was, and his experiences navigating it.

kjambunathan commented 8 months ago

@QiangF, rememeber to upload screenshots and the problematic and good odt file. This way we will have some notes to consult if we run in to this or similar issue in distant future.

I wonder whether the ODT exporter has any problems reading / writing the linked files -- which could be Image Files, or any other file link inserted with org link.

What am I saying is this ...The current problem might only be a tip of the iceberg, and we can use these issue as an excuse for testing / confirming if other FILE-NAME encoding things aren't broken.

Needless to say, we don't have much expertise to understand happens at Emacs or OS layers. Atleast we can cook up some unit test cases to ensure that there aren't more problems hidden elsewhere.

QiangF commented 8 months ago

Thank you sir. I will send you a detailed report when I get back to my friends computer. After reading your analysis, I think Emacs lisp has good support for all these coding systems, the problem is mainly with zip and unzip. Actually, I have learn from emacs-china.org that there is a function to adjust the encoding and decoding setting for a process by the executable name, I will try this:

(modify-coding-system-alist 'process "zip" '(gbk-dos . gbk-dos))
(modify-coding-system-alist 'process "unzip" '(gbk-dos . gbk-dos))

In case it works, I think overriding the (coding-system-for-write 'no-conversion) in org-odt-zip is still problematic. I think @PierreTechoueyres has his change here: https://github.com/PierreTechoueyres/org-mode-ox-odt/commit/da57da77dfc22d343aeeee1e88d737a5afb7a9e5

I don't use Emacs on windows, but if we can solve this problem, many non-english users of windows will benefit. I will get back to you later.

PierreTechoueyres commented 8 months ago

@kjambunathan I still update my branch from time to time. I don't know if all files linked in odt have issues or not, BUT what I am sure of is that when an org file has accented characters in its path, I fail exporting it . This is the reason for my branch and commit 6a5616b7 Commits above this are convenience ones. One for the version keyword (and I admit I didn't search hard if there is another way to achieve it) and the last for a warning with Org 9.6.

I think the minimal reproducer should be something like : 1) Create an temp file in c:/Somewhere/AvecDesCaractèresAccentués/myFile.org 2) wrote anything inside it 3) try to export and open (this part is important) it.

And I know this doesn't happen on GNU/Linux.

If you need I could try to create a more detailed reproducer.

I could also try something with the idea from @QiangF comments (but I'm not confident this will help as my patch work with shell command an not with processes).

QiangF commented 8 months ago

I think there could be many config combinations, a one solution for all probably doesn't exist. To proper address the unicode char in file names, we should let the user decide. We should not override coding-system-for-write, during setting the zip file name in org-odt-zip. When the name is save in one coding format and fed into zip with another coding format, file does not exist behaviour is unavoidable. Without that line I have no issues what ever I put Chinese character in the file path, either odt filename, or style file name or in a dir name. The relevant information is:

  1. What is the version of zip you are using:

    zip -h Copyright (c) 1990-2006 Info-ZIP - Type 'zip "-L"' for software license. Zip 2.32 (June 19th 2006).

  2. What is system-type and system-configuration:

    system-type value is ‘windows-nt’

    system-configuration is "x86_64-w64-mingw32"

  3. The output of M-x describe-current-coding-system:

    Coding system for saving this buffer: Not set locally, use the default. Default coding system (for new files): c -- chinese-gbk-dos (alias: gbk-dos cp936-dos windows-936-dos)

    Coding system for keyboard input: U -- utf-8-unix (alias: mule-utf-8-unix cp65001-unix)

    Coding system for terminal output: U -- utf-8 (alias: mule-utf-8 cp65001)

    Coding system for inter-client cut and paste: U -- utf-16le-dos

    Defaults for subprocess I/O: decoding: c -- chinese-gbk-dos (alias: gbk-dos cp936-dos windows-936-dos)

    encoding: c -- chinese-gbk-unix (alias: gbk-unix cp936-unix windows-936-unix)

    Priority order for recognizing coding systems when reading files:

    1. chinese-gbk (alias: gbk cp936 windows-936)
    2. iso-2022-cn (alias: chinese-iso-7bit)
    3. chinese-big5 (alias: big5 cn-big5 cp950)
    4. chinese-iso-8bit (alias: cn-gb-2312 euc-china euc-cn cn-gb gb2312)
    5. utf-8 (alias: mule-utf-8 cp65001)
    6. iso-2022-7bit
    7. iso-2022-8bit-ss2
    8. emacs-mule
    9. raw-text
    10. iso-2022-jp (alias: junet)
    11. in-is13194-devanagari (alias: devanagari)
    12. utf-8-auto
    13. utf-8-with-signature
    14. utf-16
    15. utf-16be-with-signature (alias: utf-16-be)
    16. utf-16le-with-signature (alias: utf-16-le)
    17. utf-16be
    18. utf-16le
    19. japanese-shift-jis (alias: shift_jis sjis)
    20. undecided

      Other coding systems cannot be distinguished automatically from these, and therefore cannot be recognized automatically with the present coding system priorities.

    Particular coding systems specified for certain file names:

    OPERATION TARGET PATTERN CODING SYSTEM(s)


    File I/O "\.\(arc\|zip\|lzh\|lha\|zoo\|[jew]ar\|xpi\|rar\|7z\|squashfs\|ARC\|ZIP\|LZH\|LHA\|ZOO\|[JEW]AR\|XPI\|RAR\|7Z\|SQUASHFS\)\'" no-conversion-multibyte "\.\(exe\|EXE\)\'" no-conversion "\.\(sx[dmicw]\|odt\|tar\|t[bg]z\)\'" no-conversion "\.\(gz\|Z\|bz\|bz2\|xz\|gpg\)\'" no-conversion "\.\(jpe?g\|png\|gif\|tiff?\|p[bpgn]m\)\'" no-conversion "\.pdf\'" no-conversion "/#[^/]+#\'" utf-8-emacs-unix "\.tzst\'" (no-conversion . no-conversion) "\.zst\'" (no-conversion . no-conversion) "\.dz\'" (no-conversion . no-conversion) "\.txz\'" (no-conversion . no-conversion) "\.xz\'" (no-conversion . no-conversion) "\.lzma\'" (no-conversion . no-conversion) "\.lz\'" (no-conversion . no-conversion) "\.g?z\'" (no-conversion . no-conversion) "\.\(?:tgz\|svgz\|sifz\)\'" (no-conversion . no-conversion) "\.tbz2?\'" (no-conversion . no-conversion) "\.bz2\'" (no-conversion . no-conversion) "\.Z\'" (no-conversion . no-conversion) "\.elc\'" utf-8-emacs "\.el\'" prefer-utf-8 "\.utf\(-8\)?\'" utf-8 "\.xml\'" xml-find-file-coding-system "\(\`\|/\)loaddefs.el\'" (raw-text . raw-text-unix) "\.tar\'" (no-conversion . no-conversion) "\.po[tx]?\'\|\.po\." po-find-file-coding-system "\.\(tex\|ltx\|dtx\|drv\)\'" latexenc-find-file-coding-system "" (undecided) Process I/O "*" (utf-8 . gbk-dos) "[pP][lL][iI][nN][kK]" (undecided-dos . undecided-dos) "[cC][mM][dD][pP][rR][oO][xX][yY]" (undecided-dos . undecided-dos) Network I/O nothing specified

  4. The output of M-x describe-language-environment:

    Chinese-GBK language environment

    Support for Chinese GBK character set.

    Sample text: Chinese (中文,普通话,汉语) 妳好

    Input methods (default chinese-py-punct) chinese-py-punct ("拼符" in mode line)

    Character sets: chinese-gbk: GBK Chinese simplified.

    Coding systems: chinese-gbk (‘c’ in mode line): GBK encoding for Chinese (MIME:GBK). (alias: chinese-gbk gbk cp936 windows-936)

  5. The different coding-system specific values that you set explicitly in your init file

    (set-language-environment "Chinese-GBK") (setq w32-unicode-filenames '()) (set-file-name-coding-system 'gbk-dos) (modify-coding-system-alist 'process "zip" '(utf-8 . gbk-dos))

kjambunathan commented 8 months ago

We should not override coding-system-for-write, during setting the zip file name in org-odt-zip

I have made this commit. Just to be sure, check the output file format to docx so that there are known issues with how LO'ssoffice is invoked.


@PierreTechoueyres could you please check if commit https://github.com/kjambunathan/org-mode-ox-odt/commit/a40d2acad2339ff85cc44dccfda95d96570a6cbc fixes your original issue.

I have a gut feeling that Emacs already does the right thing for argument encoding ... I think it was the spurious coding-system-for-write being forced to no-conversion which was causing an issue in your case as well.

For informational purpose, please dump these values on your problem machine ....

I want to collect as much info as possible ...

What is the version of zip you are using.
What is system-type and system-configuration
The output of M-x describe-coding-system
The output of M-x describe-language-environment
The different coding-system specific values that you set explicitly in your init file
kjambunathan commented 8 months ago

@PierreTechoueyres could you please check if commit a40d2ac fixes your original issue.

Ensure that you unapply your local changes before testing this. (This is because Emacs already has built-in heuristics and does argument encoding by default)

kjambunathan commented 8 months ago

5. (setq w32-unicode-filenames '())

When I look at an overview of how the Windows build supports file names that cannot be encoded by the current system codepage, there is a remark @L1583 which says

The UTF-8 encoded file names cannot be passed to system APIs, as Windows does not support that. Therefore, they are converted either to UTF-16 or to the ANSI codepage, depending on the value of w32-unicode-filenames, before calling any system APIs or CRT library functions. The default value of that variable is determined by the OS on which Emacs runs: nil on Windows 9X and t otherwise, but the user can change that default (although I don't see why would she want to)

So, my question to is why are you overriding the recommended value of w32-unicode-filenames

(modify-coding-system-alist 'process "zip" '(utf-8 . gbk-dos))

Dump the value of default value of process encoding system too .... It is a good idea to experiment if the value influences argument encoding in any way

File I/O ".(arc|zip|lzh|lha|zoo|[jew]ar|xpi|rar|7z|squashfs|ARC|ZIP|LZH|LHA|ZOO|[JEW]AR|XPI|RAR|7Z|SQUASHFS)\'" no-conversion-multibyte

How does the above entry for zip related to the below config entry

(modify-coding-system-alist 'process "zip" '(utf-8 . gbk-dos))

Make sure that the ODT export works for emacs -q as well


It looks like M-x describe-coding-system doesn't seem to report about file name coding ....

(I don't understand the issue well, so the above questions are meant to get more understanding of the issue)

Try executing the execute-this block in the attached org file ... it collects all variables that has coding in them. I don't know if it is important, but no harm in collecting it for both with emacs -Q and emacs.

coding-system.txt

``` ;;; -*- mode: org; -*- #+name:vars | archive-file-name-coding-system | lv | nil | | | archive-member-coding-system | lv | nil | Coding-system of archive member. | | buffer-file-coding-system | Lv | undecided-unix | Coding system to be used for encoding the buffer contents on saving. | | buffer-file-coding-system-explicit | lv | nil | The file coding system explicitly specified for the current buffer. | | coding-system-alist | v | (("georgian-academy") ("georgian-aca… | Alist of coding system names. | | coding-system-for-read | v | nil | Specify the coding system for read operations. | | coding-system-for-write | v | nil | Specify the coding system for write operations. | | coding-system-history | v | ("gbk" "windows-936" "cp936" "nil" "… | | | coding-system-iso-2022-flags | v | (long-form ascii-at-eol ascii-at-cnt… | List of symbols that control ISO-2022 encoder/decoder. | | coding-system-list | fv | (georgian-academy georgian-academy-m… | List of coding systems. | | default-file-name-coding-system | v | utf-8-unix | Default coding system for encoding file names. | | default-keyboard-coding-system | v | utf-8-unix | Default value of the keyboard coding system. | | default-process-coding-system | v | (utf-8-unix . utf-8-unix) | Cons of coding systems used for process I/O by default. | | default-terminal-coding-system | v | iso-latin-1-unix | Default value for the terminal coding system. | | file-coding-system-alist | u | (("\\.tzst\\'" . #1=(no-conversion .… | Alist to decide a coding system to use for a file I/O operation. | | file-name-coding-system | v | nil | Coding system for encoding file names. | | inherit-process-coding-system | v | nil | Non-nil means process buffer inherits coding system of process output. | | jka-compr-added-to-file-coding-system-alist | v | (("\\.tzst\\'" . #1=(no-conversion .… | | | keyboard-coding-system | PU | iso-latin-1-unix | Specify coding system for keyboard input. | | last-coding-system-specified | v | nil | Most recent coding system explicitly specified by the user when asked. | | last-coding-system-used | v | utf-8-unix | Coding system used in the latest file or process I/O. | | last-next-selection-coding-system | v | nil | | | locale-coding-system | v | iso-latin-1-unix | Coding system to use with system messages. | | locale-preferred-coding-systems | v | ((".*8859[-_]?1\\>" . iso-8859-1) ("… | List of pairs of locale regexps and preferred coding systems. | | mule-cmds--prefixed-command-last-coding-system | v | nil | | | mule-cmds--prefixed-command-next-coding-system | v | nil | | | network-coding-system-alist | v | nil | Alist to decide a coding system to use for a network I/O operation. | | next-selection-coding-system | v | nil | Coding system for the next communication with other programs. | | process-coding-system-alist | v | nil | Alist to decide a coding system to use for a process I/O operation. | | save-buffer-coding-system | lv | nil | If non-nil, use this coding system for saving the buffer. | | select-safe-coding-system-accept-default-p | v | nil | If non-nil, a function to control the behavior of coding system selection. | | select-safe-coding-system-function | v | #'select-safe-coding-system | Function to call to select safe coding system for encoding a text. | | selection-coding-system | u | nil | Coding system for communicating with other programs. | | set-coding-system-map | v | # | | | x-input-coding-system | v | nil | Coding system used for input from X input methods. | #+begin_src emacs-lisp :exports results :results value pp :wrap src elisp :var table=vars (thread-last table (seq-mapcat (lambda (it) (thread-last it car intern (funcall (lambda (it) (list (intern (format ":%S" it)) it)))))) (cons 'list)) #+end_src #+name: execute-this #+begin_src elisp :exports results :results value pp :wrap src elisp (list :archive-file-name-coding-system archive-file-name-coding-system :archive-member-coding-system archive-member-coding-system :buffer-file-coding-system buffer-file-coding-system :buffer-file-coding-system-explicit buffer-file-coding-system-explicit :coding-system-for-read coding-system-for-read :coding-system-for-write coding-system-for-write :coding-system-history coding-system-history :coding-system-iso-2022-flags coding-system-iso-2022-flags ;; :coding-system-list coding-system-list :default-file-name-coding-system default-file-name-coding-system :default-keyboard-coding-system default-keyboard-coding-system :default-process-coding-system default-process-coding-system :default-terminal-coding-system default-terminal-coding-system :file-coding-system-alist file-coding-system-alist :file-name-coding-system file-name-coding-system :inherit-process-coding-system inherit-process-coding-system :jka-compr-added-to-file-coding-system-alist jka-compr-added-to-file-coding-system-alist :keyboard-coding-system keyboard-coding-system :last-coding-system-specified last-coding-system-specified :last-coding-system-used last-coding-system-used :last-next-selection-coding-system last-next-selection-coding-system :locale-coding-system locale-coding-system :locale-preferred-coding-systems locale-preferred-coding-systems :mule-cmds--prefixed-command-last-coding-system mule-cmds--prefixed-command-last-coding-system :mule-cmds--prefixed-command-next-coding-system mule-cmds--prefixed-command-next-coding-system :network-coding-system-alist network-coding-system-alist :next-selection-coding-system next-selection-coding-system :process-coding-system-alist process-coding-system-alist :save-buffer-coding-system save-buffer-coding-system :select-safe-coding-system-accept-default-p select-safe-coding-system-accept-default-p :select-safe-coding-system-function select-safe-coding-system-function :selection-coding-system selection-coding-system :set-coding-system-map set-coding-system-map :x-input-coding-system x-input-coding-system :coding-system-alist coding-system-alist ) #+end_src #+RESULTS: #+begin_src elisp ( :archive-file-name-coding-system nil :archive-member-coding-system nil :buffer-file-coding-system utf-8-unix :buffer-file-coding-system-explicit nil :coding-system-for-read nil :coding-system-for-write nil :coding-system-history ("gbk" "windows-936" "cp936" "nil" "utf-8-dos" "dos utf-8" "dos" "utf-8" "raw-text" "utf-8 dos" "utf-8-unix" "u" "binary" "bin") :coding-system-iso-2022-flags ( long-form ascii-at-eol ascii-at-cntl 7-bit locking-shift single-shift designation revision direction init-at-bol designate-at-bol safe latin-extra composition euc-tw-shift use-roman use-oldjis 8-bit-level-4 ) :default-file-name-coding-system utf-8-unix :default-keyboard-coding-system utf-8-unix :default-process-coding-system (utf-8-unix . utf-8-unix) :default-terminal-coding-system iso-latin-1-unix :file-coding-system-alist ( #2= ("\\.tzst\\'" . #1= (no-conversion . no-conversion)) #3= ("\\.zst\\'" . #1#) #4= ("\\.dz\\'" . #1#) #5= ("\\.txz\\'" . #1#) #6= ("\\.xz\\'" . #1#) #7= ("\\.lzma\\'" . #1#) #8= ("\\.lz\\'" . #1#) #9= ("\\.g?z\\'" . #1#) #10= ("\\.\\(?:tgz\\|svgz\\|sifz\\)\\'" . #1#) #11= ("\\.tbz2?\\'" . #1#) #12= ("\\.bz2\\'" . #1#) #13= ("\\.Z\\'" . #1#) ("\\.elc\\'" . utf-8-emacs) ("\\.el\\'" . prefer-utf-8) ("\\.utf\\(-8\\)?\\'" . utf-8) ("\\.xml\\'" . xml-find-file-coding-system) ("\\(\\`\\|/\\)loaddefs.el\\'" raw-text . raw-text-unix) ("\\.tar\\'" no-conversion . no-conversion) ("\\.po[tx]?\\'\\|\\.po\\." . po-find-file-coding-system) ("\\.\\(tex\\|ltx\\|dtx\\|drv\\)\\'" . latexenc-find-file-coding-system) ("" undecided) ) :file-name-coding-system nil :inherit-process-coding-system nil :jka-compr-added-to-file-coding-system-alist (#2# #3# #4# #5# #6# #7# #8# #9# #10# #11# #12# #13#) :keyboard-coding-system iso-latin-1-unix :last-coding-system-specified nil :last-coding-system-used utf-8-unix :last-next-selection-coding-system nil :locale-coding-system iso-latin-1-unix :locale-preferred-coding-systems ((".*8859[-_]?1\\>" . iso-8859-1) (".*8859[-_]?2\\>" . iso-8859-2) (".*8859[-_]?3\\>" . iso-8859-3) (".*8859[-_]?4\\>" . iso-8859-4) (".*8859[-_]?9\\>" . iso-8859-9) (".*8859[-_]?14\\>" . iso-8859-14) (".*8859[-_]?15\\>" . iso-8859-15) (".*utf\\(?:-?8\\)?" . utf-8) (".*@euro" . iso-8859-15) ("koi8-?r" . koi8-r) ("koi8-?u" . koi8-u) ("tcvn" . tcvn) ("big5[-_]?hkscs" . big5-hkscs) ("big5" . big5) ("euc-?tw" . euc-tw) ("euc-?cn" . euc-cn) ("gb2312" . gb2312) ("gbk" . gbk) ("gb18030" . gb18030) ("ja.*[._]euc" . japanese-iso-8bit) ("ja.*[._]jis7" . iso-2022-jp) ("ja.*[._]pck" . japanese-shift-jis) ("ja.*[._]sjis" . japanese-shift-jis) ("jpn" . japanese-shift-jis)) :mule-cmds--prefixed-command-last-coding-system nil :mule-cmds--prefixed-command-next-coding-system nil :network-coding-system-alist nil :next-selection-coding-system nil :process-coding-system-alist nil :save-buffer-coding-system nil :select-safe-coding-system-accept-default-p nil :select-safe-coding-system-function select-safe-coding-system :selection-coding-system nil :set-coding-system-map (keymap (universal-coding-system-argument menu-item "For Next Command" universal-coding-system-argument :help "Coding system to be used by next command") (separator-1 #14="--") (set-buffer-file-coding-system menu-item "For Saving This Buffer" set-buffer-file-coding-system :help "How to encode this buffer when saved") (revert-buffer-with-coding-system menu-item "For Reverting This File Now" revert-buffer-with-coding-system :enable buffer-file-name :help "Revisit this file immediately using specified coding system") (set-file-name-coding-system menu-item "For File Name" set-file-name-coding-system :help "How to decode/encode file names") (separator-2 #14#) (set-keyboard-coding-system menu-item "For Keyboard" set-keyboard-coding-system :help "How to decode keyboard input") (set-terminal-coding-system menu-item "For Terminal" set-terminal-coding-system :enable (null (memq initial-window-system '(x w32 ns haiku pgtk android))) :help "How to encode terminal output") (separator-3 #14#) (set-selection-coding-system menu-item "For X Selections/Clipboard" set-selection-coding-system :visible #15= (display-selections-p) :help "How to en/decode data to/from selection/clipboard") (set-next-selection-coding-system menu-item "For Next X Selection" set-next-selection-coding-system :visible #15# :help "How to en/decode next selection/clipboard operation") (set-buffer-process-coding-system menu-item "For I/O with Subprocess" set-buffer-process-coding-system :visible (fboundp 'make-process) :enable (get-buffer-process (current-buffer)) :help "How to en/decode I/O from/to subprocess connected to this buffer") "Set Coding System") :x-input-coding-system nil :coding-system-alist ( ("georgian-academy") ("georgian-academy-mac") ("georgian-academy-dos") ("georgian-academy-unix") ("georgian-ps") ("georgian-ps-mac") ("georgian-ps-dos") ("georgian-ps-unix") ("cp1256") ("cp1256-mac") ("cp1256-dos") ("cp1256-unix") ("windows-1256") ("windows-1256-mac") ("windows-1256-dos") ("windows-1256-unix") ("iso-8859-6") ("iso-8859-6-mac") ("iso-8859-6-dos") ("iso-8859-6-unix") ("cp1258") ("cp1258-mac") ("cp1258-dos") ("cp1258-unix") ("windows-1258") ("windows-1258-mac") ("windows-1258-dos") ("windows-1258-unix") ("viqr") ("viqr-mac") ("viqr-dos") ("viqr-unix") ("vietnamese-viqr") ("vietnamese-viqr-mac") ("vietnamese-viqr-dos") ("vietnamese-viqr-unix") ("tcvn-5712") ("tcvn-5712-mac") ("tcvn-5712-dos") ("tcvn-5712-unix") ("tcvn") ("tcvn-mac") ("tcvn-dos") ("tcvn-unix") ("vietnamese-tcvn") ("vietnamese-tcvn-mac") ("vietnamese-tcvn-dos") ("vietnamese-tcvn-unix") ("vscii") ("vscii-mac") ("vscii-dos") ("vscii-unix") ("vietnamese-vscii") ("vietnamese-vscii-mac") ("vietnamese-vscii-dos") ("vietnamese-vscii-unix") ("viscii") ("viscii-mac") ("viscii-dos") ("viscii-unix") ("vietnamese-viscii") ("vietnamese-viscii-mac") ("vietnamese-viscii-dos") ("vietnamese-viscii-unix") ("tibetan") ("tibetan-mac") ("tibetan-dos") ("tibetan-unix") ("tibetan-iso-8bit") ("tibetan-iso-8bit-mac") ("tibetan-iso-8bit-dos") ("tibetan-iso-8bit-unix") ("iso-8859-11") ("iso-8859-11-mac") ("iso-8859-11-dos") ("iso-8859-11-unix") ("ibm874") ("ibm874-mac") ("ibm874-dos") ("ibm874-unix") ("cp874") ("cp874-mac") ("cp874-dos") ("cp874-unix") ("tis-620") ("tis-620-mac") ("tis-620-dos") ("tis-620-unix") ("tis620") ("tis620-mac") ("tis620-dos") ("tis620-unix") ("th-tis620") ("th-tis620-mac") ("th-tis620-dos") ("th-tis620-unix") ("thai-tis620") ("thai-tis620-mac") ("thai-tis620-dos") ("thai-tis620-unix") ("lao") ("lao-mac") ("lao-dos") ("lao-unix") ("cp949") ("cp949-mac") ("cp949-dos") ("cp949-unix") ("korean-cp949") ("korean-cp949-mac") ("korean-cp949-dos") ("korean-cp949-unix") ("korean-iso-7bit-lock") ("korean-iso-7bit-lock-mac") ("korean-iso-7bit-lock-dos") ("korean-iso-7bit-lock-unix") ("iso-2022-kr") ("iso-2022-kr-mac") ("iso-2022-kr-dos") ("iso-2022-kr-unix") ("ks_c_5601-1987") ("ks_c_5601-1987-mac") ("ks_c_5601-1987-dos") ("ks_c_5601-1987-unix") ("euc-korea") ("euc-korea-mac") ("euc-korea-dos") ("euc-korea-unix") ("euc-kr") ("euc-kr-mac") ("euc-kr-dos") ("euc-kr-unix") ("korean-iso-8bit") ("korean-iso-8bit-mac") ("korean-iso-8bit-dos") ("korean-iso-8bit-unix") ("cp290") ("cp290-mac") ("cp290-dos") ("cp290-unix") ("ebcdic-jp-kana") ("ebcdic-jp-kana-mac") ("ebcdic-jp-kana-dos") ("ebcdic-jp-kana-unix") ("ibm290") ("ibm290-mac") ("ibm290-dos") ("ibm290-unix") ("cp281") ("cp281-mac") ("cp281-dos") ("cp281-unix") ("ebcdic-jp-e") ("ebcdic-jp-e-mac") ("ebcdic-jp-e-dos") ("ebcdic-jp-e-unix") ("ibm281") ("ibm281-mac") ("ibm281-dos") ("ibm281-unix") ("shift_jis-2004") ("shift_jis-2004-mac") ("shift_jis-2004-dos") ("shift_jis-2004-unix") ("japanese-shift-jis-2004") ("japanese-shift-jis-2004-mac") ("japanese-shift-jis-2004-dos") ("japanese-shift-jis-2004-unix") ("euc-jisx0213") ("euc-jisx0213-mac") ("euc-jisx0213-dos") ("euc-jisx0213-unix") ("euc-jis-2004") ("euc-jis-2004-mac") ("euc-jis-2004-dos") ("euc-jis-2004-unix") ("iso-2022-jp-3") ("iso-2022-jp-3-mac") ("iso-2022-jp-3-dos") ("iso-2022-jp-3-unix") ("iso-2022-jp-2004") ("iso-2022-jp-2004-mac") ("iso-2022-jp-2004-dos") ("iso-2022-jp-2004-unix") ("eucjp-ms") ("eucjp-ms-mac") ("eucjp-ms-dos") ("eucjp-ms-unix") ("euc-jp") ("euc-jp-mac") ("euc-jp-dos") ("euc-jp-unix") ("euc-japan") ("euc-japan-mac") ("euc-japan-dos") ("euc-japan-unix") ("euc-japan-1990") ("euc-japan-1990-mac") ("euc-japan-1990-dos") ("euc-japan-1990-unix") ("japanese-iso-8bit") ("japanese-iso-8bit-mac") ("japanese-iso-8bit-dos") ("japanese-iso-8bit-unix") ("old-jis") ("old-jis-mac") ("old-jis-dos") ("old-jis-unix") ("iso-2022-jp-1978-irv") ("iso-2022-jp-1978-irv-mac") ("iso-2022-jp-1978-irv-dos") ("iso-2022-jp-1978-irv-unix") ("japanese-iso-7bit-1978-irv") ("japanese-iso-7bit-1978-irv-mac") ("japanese-iso-7bit-1978-irv-dos") ("japanese-iso-7bit-1978-irv-unix") ("cp932") ("cp932-mac") ("cp932-dos") ("cp932-unix") ("japanese-cp932") ("japanese-cp932-mac") ("japanese-cp932-dos") ("japanese-cp932-unix") ("sjis") ("sjis-mac") ("sjis-dos") ("sjis-unix") ("shift_jis") ("shift_jis-mac") ("shift_jis-dos") ("shift_jis-unix") ("japanese-shift-jis") ("japanese-shift-jis-mac") ("japanese-shift-jis-dos") ("japanese-shift-jis-unix") ("iso-2022-jp-2") ("iso-2022-jp-2-mac") ("iso-2022-jp-2-dos") ("iso-2022-jp-2-unix") ("junet") ("junet-mac") ("junet-dos") ("junet-unix") ("iso-2022-jp") ("iso-2022-jp-mac") ("iso-2022-jp-dos") ("iso-2022-jp-unix") ("ibm862") ("ibm862-mac") ("ibm862-dos") ("ibm862-unix") ("cp862") ("cp862-mac") ("cp862-dos") ("cp862-unix") ("cp1255") ("cp1255-mac") ("cp1255-dos") ("cp1255-unix") ("windows-1255") ("windows-1255-mac") ("windows-1255-dos") ("windows-1255-unix") ("iso-8859-8-i") ("iso-8859-8-i-mac") ("iso-8859-8-i-dos") ("iso-8859-8-i-unix") ("iso-8859-8-e") ("iso-8859-8-e-mac") ("iso-8859-8-e-dos") ("iso-8859-8-e-unix") ("iso-8859-8") ("iso-8859-8-mac") ("iso-8859-8-dos") ("iso-8859-8-unix") ("hebrew-iso-8bit") ("hebrew-iso-8bit-mac") ("hebrew-iso-8bit-dos") ("hebrew-iso-8bit-unix") ("ibm869") ("ibm869-mac") ("ibm869-dos") ("ibm869-unix") ("cp869") ("cp869-mac") ("cp869-dos") ("cp869-unix") ("ibm851") ("ibm851-mac") ("ibm851-dos") ("ibm851-unix") ("cp851") ("cp851-mac") ("cp851-dos") ("cp851-unix") ("cp737") ("cp737-mac") ("cp737-dos") ("cp737-unix") ("cp1253") ("cp1253-mac") ("cp1253-dos") ("cp1253-unix") ("windows-1253") ("windows-1253-mac") ("windows-1253-dos") ("windows-1253-unix") ("iso-8859-7") ("iso-8859-7-mac") ("iso-8859-7-dos") ("iso-8859-7-unix") ("greek-iso-8bit") ("greek-iso-8bit-mac") ("greek-iso-8bit-dos") ("greek-iso-8bit-unix") ("latin-10") ("latin-10-mac") ("latin-10-dos") ("latin-10-unix") ("iso-8859-16") ("iso-8859-16-mac") ("iso-8859-16-dos") ("iso-8859-16-unix") ("iso-latin-10") ("iso-latin-10-mac") ("iso-latin-10-dos") ("iso-latin-10-unix") ("adobe-standard-encoding") ("adobe-standard-encoding-mac") ("adobe-standard-encoding-dos") ("adobe-standard-encoding-unix") ("roman8") ("roman8-mac") ("roman8-dos") ("roman8-unix") ("hp-roman8") ("hp-roman8-mac") ("hp-roman8-dos") ("hp-roman8-unix") ("next") ("next-mac") ("next-dos") ("next-unix") ("macintosh") ("macintosh-mac") ("macintosh-dos") ("macintosh-unix") ("mac-roman") ("mac-roman-mac") ("mac-roman-dos") ("mac-roman-unix") ("ibm437") ("ibm437-mac") ("ibm437-dos") ("ibm437-unix") ("cp437") ("cp437-mac") ("cp437-dos") ("cp437-unix") ("ibm865") ("ibm865-mac") ("ibm865-dos") ("ibm865-unix") ("cp865") ("cp865-mac") ("cp865-dos") ("cp865-unix") ("ibm863") ("ibm863-mac") ("ibm863-dos") ("ibm863-unix") ("cp863") ("cp863-mac") ("cp863-dos") ("cp863-unix") ("ibm861") ("ibm861-mac") ("ibm861-dos") ("ibm861-unix") ("cp861") ("cp861-mac") ("cp861-dos") ("cp861-unix") ("ibm860") ("ibm860-mac") ("ibm860-dos") ("ibm860-unix") ("cp860") ("cp860-mac") ("cp860-dos") ("cp860-unix") ("cp858") ("cp858-mac") ("cp858-dos") ("cp858-unix") ("ibm857") ("ibm857-mac") ("ibm857-dos") ("ibm857-unix") ("cp857") ("cp857-mac") ("cp857-dos") ("cp857-unix") ("ibm852") ("ibm852-mac") ("ibm852-dos") ("ibm852-unix") ("cp852") ("cp852-mac") ("cp852-dos") ("cp852-unix") ("ibm850") ("ibm850-mac") ("ibm850-dos") ("ibm850-unix") ("cp850") ("cp850-mac") ("cp850-dos") ("cp850-unix") ("ibm775") ("ibm775-mac") ("ibm775-dos") ("ibm775-unix") ("cp775") ("cp775-mac") ("cp775-dos") ("cp775-unix") ("cp297") ("cp297-mac") ("cp297-dos") ("cp297-unix") ("ebcdic-cp-fr") ("ebcdic-cp-fr-mac") ("ebcdic-cp-fr-dos") ("ebcdic-cp-fr-unix") ("ibm297") ("ibm297-mac") ("ibm297-dos") ("ibm297-unix") ("cp285") ("cp285-mac") ("cp285-dos") ("cp285-unix") ("ebcdic-cp-gb") ("ebcdic-cp-gb-mac") ("ebcdic-cp-gb-dos") ("ebcdic-cp-gb-unix") ("ibm285") ("ibm285-mac") ("ibm285-dos") ("ibm285-unix") ("cp284") ("cp284-mac") ("cp284-dos") ("cp284-unix") ("ebcdic-cp-es") ("ebcdic-cp-es-mac") ("ebcdic-cp-es-dos") ("ebcdic-cp-es-unix") ("ibm284") ("ibm284-mac") ("ibm284-dos") ("ibm284-unix") ("cp280") ("cp280-mac") ("cp280-dos") ("cp280-unix") ("ebcdic-cp-it") ("ebcdic-cp-it-mac") ("ebcdic-cp-it-dos") ("ebcdic-cp-it-unix") ("ibm280") ("ibm280-mac") ("ibm280-dos") ("ibm280-unix") ("cp278") ("cp278-mac") ("cp278-dos") ("cp278-unix") ("ebcdic-cp-se") ("ebcdic-cp-se-mac") ("ebcdic-cp-se-dos") ("ebcdic-cp-se-unix") ("ebcdic-cp-fi") ("ebcdic-cp-fi-mac") ("ebcdic-cp-fi-dos") ("ebcdic-cp-fi-unix") ("ibm278") ("ibm278-mac") ("ibm278-dos") ("ibm278-unix") ("cp277") ("cp277-mac") ("cp277-dos") ("cp277-unix") ("ebcdic-cp-no") ("ebcdic-cp-no-mac") ("ebcdic-cp-no-dos") ("ebcdic-cp-no-unix") ("ebcdic-cp-dk") ("ebcdic-cp-dk-mac") ("ebcdic-cp-dk-dos") ("ebcdic-cp-dk-unix") ("ibm277") ("ibm277-mac") ("ibm277-dos") ("ibm277-unix") ("cp275") ("cp275-mac") ("cp275-dos") ("cp275-unix") ("ebcdic-br") ("ebcdic-br-mac") ("ebcdic-br-dos") ("ebcdic-br-unix") ("ibm275") ("ibm275-mac") ("ibm275-dos") ("ibm275-unix") ("cp274") ("cp274-mac") ("cp274-dos") ("cp274-unix") ("ebcdic-be") ("ebcdic-be-mac") ("ebcdic-be-dos") ("ebcdic-be-unix") ("ibm274") ("ibm274-mac") ("ibm274-dos") ("ibm274-unix") ("cp273") ("cp273-mac") ("cp273-dos") ("cp273-unix") ("ibm273") ("ibm273-mac") ("ibm273-dos") ("ibm273-unix") ("cp256") ("cp256-mac") ("cp256-dos") ("cp256-unix") ("ebcdic-int1") ("ebcdic-int1-mac") ("ebcdic-int1-dos") ("ebcdic-int1-unix") ("ibm256") ("ibm256-mac") ("ibm256-dos") ("ibm256-unix") ("cp1257") ("cp1257-mac") ("cp1257-dos") ("cp1257-unix") ("windows-1257") ("windows-1257-mac") ("windows-1257-dos") ("windows-1257-unix") ("cp1254") ("cp1254-mac") ("cp1254-dos") ("cp1254-unix") ("windows-1254") ("windows-1254-mac") ("windows-1254-dos") ("windows-1254-unix") ("cp1252") ("cp1252-mac") ("cp1252-dos") ("cp1252-unix") ("windows-1252") ("windows-1252-mac") ("windows-1252-dos") ("windows-1252-unix") ("cp1250") ("cp1250-mac") ("cp1250-dos") ("cp1250-unix") ("windows-1250") ("windows-1250-mac") ("windows-1250-dos") ("windows-1250-unix") ("latin-0") ("latin-0-mac") ("latin-0-dos") ("latin-0-unix") ("latin-9") ("latin-9-mac") ("latin-9-dos") ("latin-9-unix") ("iso-8859-15") ("iso-8859-15-mac") ("iso-8859-15-dos") ("iso-8859-15-unix") ("iso-latin-9") ("iso-latin-9-mac") ("iso-latin-9-dos") ("iso-latin-9-unix") ("latin-8") ("latin-8-mac") ("latin-8-dos") ("latin-8-unix") ("iso-8859-14") ("iso-8859-14-mac") ("iso-8859-14-dos") ("iso-8859-14-unix") ("iso-latin-8") ("iso-latin-8-mac") ("iso-latin-8-dos") ("iso-latin-8-unix") ("latin-7") ("latin-7-mac") ("latin-7-dos") ("latin-7-unix") ("iso-8859-13") ("iso-8859-13-mac") ("iso-8859-13-dos") ("iso-8859-13-unix") ("iso-latin-7") ("iso-latin-7-mac") ("iso-latin-7-dos") ("iso-latin-7-unix") ("latin-6") ("latin-6-mac") ("latin-6-dos") ("latin-6-unix") ("iso-8859-10") ("iso-8859-10-mac") ("iso-8859-10-dos") ("iso-8859-10-unix") ("iso-latin-6") ("iso-latin-6-mac") ("iso-latin-6-dos") ("iso-latin-6-unix") ("latin-5") ("latin-5-mac") ("latin-5-dos") ("latin-5-unix") ("iso-8859-9") ("iso-8859-9-mac") ("iso-8859-9-dos") ("iso-8859-9-unix") ("iso-latin-5") ("iso-latin-5-mac") ("iso-latin-5-dos") ("iso-latin-5-unix") ("latin-4") ("latin-4-mac") ("latin-4-dos") ("latin-4-unix") ("iso-8859-4") ("iso-8859-4-mac") ("iso-8859-4-dos") ("iso-8859-4-unix") ("iso-latin-4") ("iso-latin-4-mac") ("iso-latin-4-dos") ("iso-latin-4-unix") ("latin-3") ("latin-3-mac") ("latin-3-dos") ("latin-3-unix") ("iso-8859-3") ("iso-8859-3-mac") ("iso-8859-3-dos") ("iso-8859-3-unix") ("iso-latin-3") ("iso-latin-3-mac") ("iso-latin-3-dos") ("iso-latin-3-unix") ("latin-2") ("latin-2-mac") ("latin-2-dos") ("latin-2-unix") ("iso-8859-2") ("iso-8859-2-mac") ("iso-8859-2-dos") ("iso-8859-2-unix") ("iso-latin-2") ("iso-latin-2-mac") ("iso-latin-2-dos") ("iso-latin-2-unix") ("cp038") ("cp038-mac") ("cp038-dos") ("cp038-unix") ("ebcdic-int") ("ebcdic-int-mac") ("ebcdic-int-dos") ("ebcdic-int-unix") ("ibm038") ("ibm038-mac") ("ibm038-dos") ("ibm038-unix") ("cp1047") ("cp1047-mac") ("cp1047-dos") ("cp1047-unix") ("ibm1047") ("ibm1047-mac") ("ibm1047-dos") ("ibm1047-unix") ("ebcdic-uk") ("ebcdic-uk-mac") ("ebcdic-uk-dos") ("ebcdic-uk-unix") ("ebcdic-us") ("ebcdic-us-mac") ("ebcdic-us-dos") ("ebcdic-us-unix") ("devanagari") ("devanagari-mac") ("devanagari-dos") ("devanagari-unix") ("in-is13194-devanagari") ("in-is13194-devanagari-mac") ("in-is13194-devanagari-dos") ("in-is13194-devanagari-unix") ("pt154") ("pt154-mac") ("pt154-dos") ("pt154-unix") ("mik") ("mik-mac") ("mik-dos") ("mik-unix") ("ibm855") ("ibm855-mac") ("ibm855-dos") ("ibm855-unix") ("cp855") ("cp855-mac") ("cp855-dos") ("cp855-unix") ("cp866u") ("cp866u-mac") ("cp866u-dos") ("cp866u-unix") ("ruscii") ("ruscii-mac") ("ruscii-dos") ("ruscii-unix") ("cp1125") ("cp1125-mac") ("cp1125-dos") ("cp1125-unix") ("cp1251") ("cp1251-mac") ("cp1251-dos") ("cp1251-unix") ("windows-1251") ("windows-1251-mac") ("windows-1251-dos") ("windows-1251-unix") ("koi8-t") ("koi8-t-mac") ("koi8-t-dos") ("koi8-t-unix") ("cp866") ("cp866-mac") ("cp866-dos") ("cp866-unix") ("alternativnyj") ("alternativnyj-mac") ("alternativnyj-dos") ("alternativnyj-unix") ("cyrillic-alternativnyj") ("cyrillic-alternativnyj-mac") ("cyrillic-alternativnyj-dos") ("cyrillic-alternativnyj-unix") ("koi8-u") ("koi8-u-mac") ("koi8-u-dos") ("koi8-u-unix") ("cp878") ("cp878-mac") ("cp878-dos") ("cp878-unix") ("koi8") ("koi8-mac") ("koi8-dos") ("koi8-unix") ("koi8-r") ("koi8-r-mac") ("koi8-r-dos") ("koi8-r-unix") ("cyrillic-koi8") ("cyrillic-koi8-mac") ("cyrillic-koi8-dos") ("cyrillic-koi8-unix") ("iso-8859-5") ("iso-8859-5-mac") ("iso-8859-5-dos") ("iso-8859-5-unix") ("cyrillic-iso-8bit") ("cyrillic-iso-8bit-mac") ("cyrillic-iso-8bit-dos") ("cyrillic-iso-8bit-unix") ("gb18030") ("gb18030-mac") ("gb18030-dos") ("gb18030-unix") ("chinese-gb18030") ("chinese-gb18030-mac") ("chinese-gb18030-dos") ("chinese-gb18030-unix") ("windows-936") ("windows-936-mac") ("windows-936-dos") ("windows-936-unix") ("cp936") ("cp936-mac") ("cp936-dos") ("cp936-unix") ("gbk") ("gbk-mac") ("gbk-dos") ("gbk-unix") ("chinese-gbk") ("chinese-gbk-mac") ("chinese-gbk-dos") ("chinese-gbk-unix") ("euc-taiwan") ("euc-taiwan-mac") ("euc-taiwan-dos") ("euc-taiwan-unix") ("euc-tw") ("euc-tw-mac") ("euc-tw-dos") ("euc-tw-unix") ("cn-big5-hkscs") ("cn-big5-hkscs-mac") ("cn-big5-hkscs-dos") ("cn-big5-hkscs-unix") ("big5-hkscs") ("big5-hkscs-mac") ("big5-hkscs-dos") ("big5-hkscs-unix") ("chinese-big5-hkscs") ("chinese-big5-hkscs-mac") ("chinese-big5-hkscs-dos") ("chinese-big5-hkscs-unix") ("cp950") ("cp950-mac") ("cp950-dos") ("cp950-unix") ("cn-big5") ("cn-big5-mac") ("cn-big5-dos") ("cn-big5-unix") ("big5") ("big5-mac") ("big5-dos") ("big5-unix") ("chinese-big5") ("chinese-big5-mac") ("chinese-big5-dos") ("chinese-big5-unix") ("hz") ("hz-mac") ("hz-dos") ("hz-unix") ("hz-gb-2312") ("hz-gb-2312-mac") ("hz-gb-2312-dos") ("hz-gb-2312-unix") ("chinese-hz") ("chinese-hz-mac") ("chinese-hz-dos") ("chinese-hz-unix") ("gb2312") ("gb2312-mac") ("gb2312-dos") ("gb2312-unix") ("cn-gb") ("cn-gb-mac") ("cn-gb-dos") ("cn-gb-unix") ("euc-cn") ("euc-cn-mac") ("euc-cn-dos") ("euc-cn-unix") ("euc-china") ("euc-china-mac") ("euc-china-dos") ("euc-china-unix") ("cn-gb-2312") ("cn-gb-2312-mac") ("cn-gb-2312-dos") ("cn-gb-2312-unix") ("chinese-iso-8bit") ("chinese-iso-8bit-mac") ("chinese-iso-8bit-dos") ("chinese-iso-8bit-unix") ("iso-2022-cn-ext") ("iso-2022-cn-ext-mac") ("iso-2022-cn-ext-dos") ("iso-2022-cn-ext-unix") ("chinese-iso-7bit") ("chinese-iso-7bit-mac") ("chinese-iso-7bit-dos") ("chinese-iso-7bit-unix") ("iso-2022-cn") ("iso-2022-cn-mac") ("iso-2022-cn-dos") ("iso-2022-cn-unix") ("utf-7-imap") ("utf-7-imap-mac") ("utf-7-imap-dos") ("utf-7-imap-unix") ("utf-7") ("utf-7-mac") ("utf-7-dos") ("utf-7-unix") ("ascii") ("ascii-mac") ("ascii-dos") ("ascii-unix") ("iso-safe") ("iso-safe-mac") ("iso-safe-dos") ("iso-safe-unix") ("us-ascii") ("us-ascii-mac") ("us-ascii-dos") ("us-ascii-unix") ("ctext-with-extensions") ("ctext-with-extensions-mac") ("ctext-with-extensions-dos") ("ctext-with-extensions-unix") ("x-ctext-with-extensions") ("x-ctext-with-extensions-mac") ("x-ctext-with-extensions-dos") ("x-ctext-with-extensions-unix") ("compound-text-with-extensions") ("compound-text-with-extensions-mac") ("compound-text-with-extensions-dos") ("compound-text-with-extensions-unix") ("ctext-no-compositions") ("ctext-no-compositions-mac") ("ctext-no-compositions-dos") ("ctext-no-compositions-unix") ("ctext") ("ctext-mac") ("ctext-dos") ("ctext-unix") ("x-ctext") ("x-ctext-mac") ("x-ctext-dos") ("x-ctext-unix") ("compound-text") ("compound-text-mac") ("compound-text-dos") ("compound-text-unix") ("iso-2022-8bit-ss2") ("iso-2022-8bit-ss2-mac") ("iso-2022-8bit-ss2-dos") ("iso-2022-8bit-ss2-unix") ("iso-2022-cjk") ("iso-2022-cjk-mac") ("iso-2022-cjk-dos") ("iso-2022-cjk-unix") ("iso-2022-7bit-lock-ss2") ("iso-2022-7bit-lock-ss2-mac") ("iso-2022-7bit-lock-ss2-dos") ("iso-2022-7bit-lock-ss2-unix") ("iso-2022-int-1") ("iso-2022-int-1-mac") ("iso-2022-int-1-dos") ("iso-2022-int-1-unix") ("iso-2022-7bit-lock") ("iso-2022-7bit-lock-mac") ("iso-2022-7bit-lock-dos") ("iso-2022-7bit-lock-unix") ("iso-2022-7bit-ss2") ("iso-2022-7bit-ss2-mac") ("iso-2022-7bit-ss2-dos") ("iso-2022-7bit-ss2-unix") ("iso-2022-7bit") ("iso-2022-7bit-mac") ("iso-2022-7bit-dos") ("iso-2022-7bit-unix") ("utf-16-be") ("utf-16-be-mac") ("utf-16-be-dos") ("utf-16-be-unix") ("utf-16-le") ("utf-16-le-mac") ("utf-16-le-dos") ("utf-16-le-unix") ("utf-16") ("utf-16-mac") ("utf-16-dos") ("utf-16-unix") ("utf-16be-with-signature") ("utf-16be-with-signature-mac") ("utf-16be-with-signature-dos") ("utf-16be-with-signature-unix") ("utf-16le-with-signature") ("utf-16le-with-signature-mac") ("utf-16le-with-signature-dos") ("utf-16le-with-signature-unix") ("utf-16be") ("utf-16be-mac") ("utf-16be-dos") ("utf-16be-unix") ("utf-16le") ("utf-16le-mac") ("utf-16le-dos") ("utf-16le-unix") ("emacs-internal") ("utf-8-emacs") ("utf-8-emacs-mac") ("utf-8-emacs-dos") ("utf-8-emacs-unix") ("cp65001") ("cp65001-mac") ("cp65001-dos") ("cp65001-unix") ("mule-utf-8") ("mule-utf-8-mac") ("mule-utf-8-dos") ("mule-utf-8-unix") ("utf-8-auto") ("utf-8-auto-mac") ("utf-8-auto-dos") ("utf-8-auto-unix") ("utf-8-with-signature") ("utf-8-with-signature-mac") ("utf-8-with-signature-dos") ("utf-8-with-signature-unix") ("utf-8") ("utf-8-mac") ("utf-8-dos") ("utf-8-unix") ("emacs-mule") ("emacs-mule-mac") ("emacs-mule-dos") ("emacs-mule-unix") ("latin-1") ("latin-1-mac") ("latin-1-dos") ("latin-1-unix") ("iso-8859-1") ("iso-8859-1-mac") ("iso-8859-1-dos") ("iso-8859-1-unix") ("iso-latin-1") ("iso-latin-1-mac") ("iso-latin-1-dos") ("iso-latin-1-unix") ("no-conversion-multibyte") ("raw-text") ("raw-text-mac") ("raw-text-dos") ("raw-text-unix") ("prefer-utf-8") ("prefer-utf-8-mac") ("prefer-utf-8-dos") ("prefer-utf-8-unix") ("mac") ("dos") ("unix") ("binary") ("undecided") ("undecided-mac") ("undecided-dos") ("undecided-unix") ("no-conversion") )) #+end_src ```
PierreTechoueyres commented 8 months ago

@kjambunathan With your latest fix I'm now unable to reproduce the problem with accentuated files. Everything work as expected.

FYI I've used the following file named c:\Temp\accentué.org with the following content:

# accentué.org -- -*- coding: utf-8 -*-
#+setupfile: ~/.emacs.d/themes/betomorrow-theme/theme-readtheorg.setup
#+title:accentué
#+version: 1.0
#+LANGUAGE: fr
#+OPTIONS: tex:nil ^:{} html-postamble:nil ':t toc:4
#+STARTUP: align nolatexpreview nofold hideblocks
#+PAGEBREAK:

* Setup

#+begin_src elisp
(w32-register-hot-key [h-])

(let ((libre-office-path "C:/Program Files/LibreOffice/program"))
  (setenv "PATH" (concat libre-office-path ";" (getenv "PATH")))
  (add-to-list 'exec-path libre-office-path))

(custom-set-variables
 '(w32-apps-modifier 'super)
 '(w32-pass-apps-to-system nil)
 '(package-archives '(("gnu"       . "https://elpa.gnu.org/packages/")
              ("nongnu"    . "https://elpa.nongnu.org/nongnu/")
              ("ox-odt"    . "https://kjambunathan.github.io/elpa/")))
 '(org-odt-convert-process "LibreOffice")
 '(org-odt-preferred-output-format "docx")
 '(org-odt-transform-processes
   '(("Optimize Column Width of all Tables"
      "soffice" "--norestore" "--invisible" "--headless"
      "macro:///OrgMode.Utilities.OptimizeColumnWidth(%I)")
     ("Update All"
      "soffice" "--norestore" "--invisible" "--headless"
      "macro:///OrgMode.Utilities.UpdateAll(%I)")
     ("Reload"
      "soffice" "--norestore" "--invisible" "--headless"
      "macro:///OrgMode.Utilities.Reload(%I)")))
 (set-fontset-font t 'emoji "Segoe UI Emoji" nil 'prepend)

 )

(set-fontset-font t 'emoji "Segoe UI Emoji" nil 'prepend)

(prefer-coding-system 'utf-8)
(set-default-coding-systems 'utf-8-unix)

(package-refresh-contents)
(package-install 'ox-odt)
(windmove-default-keybindings 'super)
#+end_src

#+RESULTS:

* Object
Just test with accents and emoji

Pierre Téchoueyres

🥳🧁🍰🎁🎂🎈🎺🎉🎊📧〽️🧿🌶️🔋😂❤️😍🤣😊🥺🙏💕😭😘👍

# Fin du fichier accentué

Some part aren't necessary for the current problem BUT I've tried with a setup as close as possible as my one.

P.S. Would you mind to make a new release of ox-odt with the latest change ? Extra-bonus : would you mind to include the two other commits from my branch ?

kjambunathan commented 8 months ago

Everything work as expected.

I think this means ALL the 3 things below

  1. zip works
  2. soffice converstion to docx works
  3. soffice macro for Optimizing Column Width of tables etc works

    would you mind to include the two other commits from my branch ?

Do you mean

  1. Add version keyword and
  2. fix-org-9.6-deprecation

(1) is already supported, see Add macro keyword-to-documentproperty and keyword ODT_DOCUMENT_PROPERTIES. Just define version a document property, and you will be all set. If that feature is incomplete, head over to #117.

(2) is probably minor. The warnings are annoying for me to. But I have left it alone, just in case there are people interested in using my exporter with old Emacs-en or old Org.

Would you mind to make a new release of ox-odt with the latest change ?

No issues ... This once you confirm all the 3 zip and soffice issues are resolved.

PierreTechoueyres commented 8 months ago

Would you mind to make a new release of ox-odt with the latest change ?

No issues ... This once you confirm all the 3 zip and soffice issues are resolved. I confirm, for my usage, that everything is working as expected. Thanks in advance

PierreTechoueyres commented 8 months ago

@kjambunathan Sorry I missed this when I try to byte compile the file

Compiling file ~/Projects/Divers/org-mode-ox-odt/lisp/ox-odt.el at Thu Dec 28 15:05:34 2023 ox-odt.el:2759:20: Error: Wrong number of arguments: (2 . 2), 1

kjambunathan commented 8 months ago

Compiling file ~/Projects/Divers/org-mode-ox-odt/lisp/ox-odt.el at Thu Dec 28 15:05:34 2023 ox-odt.el:2759:20: Error: Wrong number of arguments: (2 . 2), 1

I think that is the line

(pcase-let* (((odt-map :soffice-executable-signature) org-odt-custom-desktop-file-options)) and the odt-map is defined in odt.el and pcase-defmacro is auto-loaded

Ofcourse, I run Emacs from master and I updated only very recently

~/src1/org-mode-ox-odt-pristine$ emacs --fingerprint
3b2a84f2f1842767edbd58e6151aba3a1c3a6b934274adbc7bdfd2d0104623f7
~/src1/org-mode-ox-odt-pristine$ emacs --version
GNU Emacs 30.0.50
Development version 8f571769e155 on master branch; build date 2023-12-28.
Copyright (C) 2023 Free Software Foundation, Inc.
GNU Emacs comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GNU Emacs
under the terms of the GNU General Public License.
For more information about these matters, see the file named COPYING.

May be pcase-defmacro is not loaded in your case ... I am running it out of ideas Try to narrow it further. Btw, for compiling ox-ods, peg may be needed. You can download it from GNU ELPA. And I compile ox-ods with

ELPA_DIR_FOR_ODT=~/src/nongnu-elpa/packages make

and ~/src/nongnu-elpa/packages is essentially my elpa dir and has a peg subdir.

So, I a anticipate some issues with ox-ods but not ox-odt.

For me compilation goes through fine at the following commit ...

commit 2f824e71882d7d18dd4a3fabb94b6e3d38df9510 (HEAD -> master, origin/master, origin/HEAD)
Merge: a40d2acad 58c91cbf9
Author: Jambunathan K <kjambunathan@gmail.com>
Date:   Thu Dec 28 17:44:07 2023 +0530

    Merge tag 'release_9.6.14'

    Release 9.6.14
PierreTechoueyres commented 8 months ago

@kjambunathan

With the following modifications on file odt.el I was able to compile everything :

@@ -26,9 +26,20 @@
 ;;; Code:

 (require 'dom)
+(require 'map)

 ;;;; Misc. Helpers

+(unless (fboundp 'map--pcase-map-elt)
+  (defmacro map--pcase-map-elt (key default map)
+    "A macro to make MAP the last argument to `map-elt'.
+
+This allows using default values for `map-elt', which can't be
+done using `pcase--flip'.
+
+KEY is the key sought in the map.  DEFAULT is the default value."
+    `(map-elt ,map ,key ,default)))
+
 (defun odt-map--make-pcase-bindings (args)
   (thread-last args
           (seq-map

I've stolen the defmacro on map.el wich is provided on master branch of Emacs.

kjambunathan commented 8 months ago

(require 'map)

This may be the only thing required if you could upgrade map using GNU ELPA. This require is only needed only if somehow pcase doesn't load map somewhere else ...

Would you like to try the upgrade?

I've stolen the defmacro on map.el wich is provided on master branch of Emacs.

Instead of doing this, can you install map.el from GNU ELPA, and the version there seems to offer map--make-pcase-bindings

PierreTechoueyres commented 8 months ago

(require 'map)

This may be the only thing required if you could upgrade map using GNU ELPA. This require is only needed only if somehow pcase doesn't load map somewhere else ...

This is essentially the native compiler which complained about missing function definitions. This also remind me that the ox-odt-pkg.el file need some upgrades:

Would you like to try the upgrade?

I tried with no result. See bellow.

I've stolen the defmacro on map.el wich is provided on master branch of Emacs.

Instead of doing this, can you install map.el from GNU ELPA, and the version there seems to offer map--make-pcase-bindings

The version of map available on GNU ELPA is the same that's shipped with my Emacs 29.1 and for now the one in the git repository is only available in master.

Package Version Status Archive Description
map 3.3.1 available gnu Map manipulation functions
map 3.3.1 built-in Map manipulation functions
kjambunathan commented 8 months ago

I've stolen the defmacro on map.el wich is provided on master branch of Emacs.

I have introduced pcase--flip-3ary. When I introduced as cl-macrolet', I couldn't get the compilation to pass. So, I have introduced a global macro. (I need to think about whycl-macrolet` wouldn't work)

Instead of doing this, can you install map.el from GNU ELPA, and the version there seems to offer map--make-pcase-bindings

The version of map available on GNU ELPA is the same that's shipped with my Emacs 29.1 and for now the one in the git repository is only available in master.

Use gnu-devel ... if I set package-archives to

`(("gnu-devel" . "https://elpa.gnu.org/devel/")
 ("gnu" . "https://elpa.gnu.org/packages/")
 ("nongnu" . "https://elpa.nongnu.org/nongnu/")
 ("melpa" . "https://melpa.org/packages/")
 ("ox-odt" . "https://kjambunathan.github.io/elpa/"))

I see this

  map                            3.3.1          available    gnu      Map manipulation functions      
  map                            3.3.1.0.20230… available    gnu-dev… Map manipulation functions      

So, installing from gnu-devel is a good idea. I hope the 4th decimals in the version are significant.

This is essentially the native compiler which complained about missing function definitions. This also remind me that the ox-odt-pkg.el file need some upgrades:

add requirements for peg, fix the local variable block to avoid byte compilation maybe add informations like commit, author and url

The folks that regularly who use the exporter can be counted in fingers of my hand. They are comfortable building from git checkout. I make releases only when someone requests it. The earlier release was almost an year ago, things have moved quite a bit both on ODT and ODS side. I will ensure that the items you are flagged addressed (whenever I happen to make the release)

Please pull and let me know what you find.

PierreTechoueyres commented 8 months ago

I've stolen the defmacro on map.el wich is provided on master branch of Emacs.

I have introduced pcase--flip-3ary. When I introduced as cl-macrolet', I couldn't get the compilation to pass. So, I have introduced a global macro. (I need to think about whycl-macrolet` wouldn't work)

It works !

… Use gnu-devel ... if I set package-archives to … I see this

  map                            3.3.1          available    gnu      Map manipulation functions      
  map                            3.3.1.0.20230… available    gnu-dev… Map manipulation functions      

So, installing from gnu-devel is a good idea. I hope the 4th decimals in the version are significant.

I didn't need it to compile successfully.

… The folks that regularly who use the exporter can be counted in fingers of my hand. They are comfortable building from git checkout. I make releases only when someone requests it. The earlier release was almost an year ago, things have moved quite a bit both on ODT and ODS side. I will ensure that the items you are flagged addressed (whenever I happen to make the release)

I can do that too, but I'm lazy in following your developments, so if you publish a release from times to times it will help me keep up with you.

Anyway, thanks for your GREAT work on this exporter. This really makes my life easier (as in the past).

Please pull and let me know what you find.

For now everything I'm using is working.