msgfmt tries to open file named --xml when building AppStream or Desktop Entry files

[awilfox]

Environment

• HexChat 2.14.1
• Meson 0.47.0
• gettext-tiny master

Steps to reproduce

mkdir _build
cd _build
meson -Dprefix=/usr -Dwith-lua=false
ninja

Actual results

[32/92] Generating io.github.Hexchat.desktop_merge with a custom command.
FAILED: data/misc/io.github.Hexchat.desktop 
/usr/bin/python3 /usr/bin/meson --internal msgfmthelper data/misc/io.github.Hexchat.desktop.in data/misc/io.github.Hexchat.desktop desktop ../data/misc/../../po
fopen: No such file or directory

Using strace we see the issue:

execve("/usr/bin/msgfmt", ["msgfmt", "--xml", "-d", "../data/misc/../../po", "--template", "../data/misc/io.github.Hexchat.a"..., "-o", "data/misc/io.github.Hexchat.appd"...], 0x3fffd39336a0 /* 46 vars */) = 0
open("--xml", O_RDONLY|O_LARGEFILE)     = -1 ENOENT (No such file or directory)

Notes

Even if --xml support was added (or just ignored), there's no support for --template which is used for Desktop Entry stuff in GNU msgfmt.

I don't have any experience with this stuff at all. I've always written my own Desktop Entry files using the embedded translation syntax, and I have no experience with AppStream. In fact this is the first time I've even heard of it.

[rofl0r]

@xhebox and i are discussing the issue right now in chat... meanwhile may i suggest my own fork of xchat2 ( https://github.com/rofl0r/ixchat ), which doesn't use meson nor XML stuff ;)

[xhebox]

@awilfox

I made a discussion with @rofl0r . We did not heard of appstream, too, but we agreed that AppStream is too new a thing. And only translating two or four files in one program is not worth for us coding a full support. So coding a stub is just fine.

The result of discussion is simple copy should make --template works to fix the transformation of .desktop. And ignore --xml should fix the transformation of .appdata.xml. I will do the work later.

Though this should fix the build, but we think hexchat should not generate appdata files by default:

if get_option('with-gtk')
  hexchat_appdata = i18n.merge_file(
    input: 'io.github.Hexchat.appdata.xml.in',
    output: 'io.github.Hexchat.appdata.xml',
    po_dir: '../../po',
    install: true,
    install_dir: join_paths(get_option('datadir'), 'metainfo')
  )

Adding a new option to dis/enable the generation of appdata files would be nice. I would suggest you make an issue for hexchat.

By the way, welcome if you're interested in maintaining this project in long term. You could get a write access.

[awilfox]

@rofl0r – I personally prefer WeeChat; HexChat was specifically requested by a number of our users (actually, I think it was almost a dozen; out of ~50, that's a significant chunk unfortunately). Also, the last time I investigated ixchat, the Python plugin required Python 2, which we do not support in Adélie. It does not look like that has changed in the meantime.

@xhebox – Yes, I think AppStream should be an option as well. I have filed a bug with HexChat that mentions it.

At this time I am not interested in gaining push/write access to gettext-tiny, but I am flattered at the offer, and will consider it seriously when I have more time to dedicate to finding and fixing actual issues with it.

[TingPing]

meanwhile may i suggest my own fork of xchat2 ( https://github.com/rofl0r/ixchat ), which doesn't use meson nor XML stuff ;)

Yay security bugs \o/ (Yes I see you backported a few of my patches, being the author of them I can assure you that is nowhere near enough)

[rofl0r]

@TingPing

Yay security bugs \o/

heh, do you have any specific fixes in mind ?

@awilfox

@xhebox – Yes, I think AppStream should be an option as well. I have filed a bug with HexChat that mentions it.

great!

but I am flattered at the offer

<3

Python 2, which we do not support in Adélie

wait, you don't even have python2 in the package catalogue ? there's a huge trove of (especially info sec related) code that's python2 only, like sqlmap, angr, and many more (not that i think it would take a lot of work to make the plugin python3 compatible)

[TingPing]

there's a huge trove of (especially info sec related) code that's python2 only,

Python 2 is only supported for another year and a half. As a distro it might be smart to not invest effort into that.

(not that i think it would take a lot of work to make the plugin python3 compatible)

You would keep backporting hexchat work, at which point maybe just use the actual upstream hexchat.

heh, do you have any specific fixes in mind ?

We are talking about maybe 6 years worth of bug fixes with at least dozens of them being remotely exploitable, sorry I can't directly point to many of them.

[awilfox]

That's correct, we do not ship Python 2 at all, and we will never do so. It is a dead-end technology that has a suitable replacement in Python 3.

I know that sounds strange from someone who just spent a significant effort to port software to the PowerPC G4, but I feel Python 3 really does cleanly replace Python 2. It runs better than 2.x, even on older hardware, and is a much cleaner language overall. And since we plan on supporting every release for 3 years, we do not want to be stuck with Python 2 for years after it is dead upstream.

[awilfox]

While trying to build TigerVNC, I found another usage of this:

  find_program(INTLTOOL_MERGE_EXECUTABLE intltool-merge)
  if("${GETTEXT_VERSION_STRING}" VERSION_GREATER 0.18.99)
    add_custom_command(OUTPUT vncviewer.desktop
      COMMAND ${GETTEXT_MSGFMT_EXECUTABLE}
                --desktop --template vncviewer.desktop.in
                -d ${CMAKE_SOURCE_DIR}/po -o vncviewer.desktop
      DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/vncviewer.desktop.in
    )
  elseif(INTLTOOL_MERGE_EXECUTABLE)
    add_custom_command(OUTPUT vncviewer.desktop
      COMMAND sed -e 's/^Name/_Name/'
                -e 's/^GenericName/_GenericName/'
                -e 's/^Comment/_Comment/'
                vncviewer.desktop.in > vncviewer.desktop.intl
      COMMAND ${INTLTOOL_MERGE_EXECUTABLE}
                -d ${CMAKE_SOURCE_DIR}/po
                vncviewer.desktop.intl vncviewer.desktop
      DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/vncviewer.desktop.in
    )
  else()
    add_custom_command(OUTPUT vncviewer.desktop
      COMMAND cp vncviewer.desktop.in vncviewer.desktop
      DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/vncviewer.desktop.in
    )
  endif()

However, note that it looks like intltool can do it too. Maybe we should investigate that.

[xhebox]

That looks like a perl script, not a surprise. After all, --template is just adding some translations in the format of a template. Of course, it's very convenient to use some scripts to finish this task.

So, what do you mean, @awilfox ? Adding some real support than stub by external scripts/commands or some other methods? I mean, i dont see the point of investigation, if we only need to make a stub. But i am not against with adding the full support by invoking intltool(if exist). Do not know the thought of @rofl0r .

[awilfox]

I am thinking invoking intltool-merge if it exists, and otherwise a stub. But that may be too much work.

[rofl0r]

@TingPing

You would keep backporting hexchat work, at which point maybe just use the actual upstream hexchat.

thing is i prefer the look and feel of xchat2 over the emojis and smilies of hexchat. and the choice of meson as a build system really makes me scratch my head, forcing yet another build system down people's throats, as if cmake wasn't bad enough. soon the installations of build system will take up 50% of my harddrive space.

[TingPing]

thing is i prefer the look and feel of xchat2 over the emojis and smilies of hexchat.

That is a complete misunderstanding. The text box behaves the same.

and the choice of meson as a build system really makes me scratch my head

Well you can dislike that but it has no effect on the end user.

[rofl0r]

Well you can dislike that but it has no effect on the end user.

it has no effect on the end user who does not compile from source, correct. but it does have effect on those who do.

[awilfox]

We've just hit this building a GNOME 3 dependency, so I've fixed the title; it isn't specific to Meson.

make[3]: Entering directory '/home/aphrael/adelie.packages/user/zeitgeist/src/zeitgeist-1.0.1/data'
  GEN      org.gnome.zeitgeist.Engine.service
  GEN      zeitgeist.service
  GEN      zeitgeist-datahub.desktop
fopen: No such file or directory
make[3]: *** [Makefile:836: zeitgeist-datahub.desktop] Error 1

[Edit: After finding out what this service does, we're probably not going to ship it at all. I'm still leaving this comment here to note that there are packages out there which do need this functionality.]

[awilfox]

Let it be known for further reference that Evince, the GNOME PDF viewer software, is made entirely of .service and .desktop files that it wants to run through msgfmt. Temporarily I am doing the cp hack so that it builds at all, but there are 15 files, so it's extremely annoying. It would be very nice if there could be a way to hook to intltool-merge where available.

[TingPing]

Well yea, all software will eventually drop intltool, its a dead project that upstream gettext can replace now.

[awilfox]

I was under the impression there were no new releases because it was "done".

If it is true that intltool is actually dead, then gettext-tiny msgfmt needs to actually implement the handling of .desktop files.

[TingPing]

It has not had a release in 4 years and there are plenty of bugs, 61 reported against the Launchpad repo + 2 years worth of unreleased commits. It is just abandoned at this point.

Yes of course gettext-tiny should match the features of gettext is thats its goal.

[xhebox]

newpoparser has been merged, so it's turn to solve this problem.

@awilfox dont except too much on a full implementation. i'll try, but it's likely to be a stub. since linking to an external xml library is not what we want. if you have any idea on if string substitution could implement the task, i'd like to hear.

[TingPing]

I think you want at least partial real xml parsing. You need to be able to take arbitrary xml elements (as defined by itstool) extract info from those and expand them into new elements.

[awilfox]

Perhaps it could optionally link to expat, since that is needed by many other core tools (gdb, git, libarchive), and if not linked to expat then it uses a stub?

[xhebox]

thx for reply, i'll consider :)

[xhebox]

@awilfox tested stub implementation has been pushed to master. I'll consider a full implementation.