Inserted images lose transparency (if they had any)

[ocumo]

Version, Operative system Cherrytree 0.99.24 and 0.99.25 - Xubuntu 20.04.1 LTS -

Describe the bug Inserted images, which has transparency, e.g. PNG, (either pasting with crtl-V or inserting via menu) lose transparency. This problem only happens with the 0.99.xx series; in the older 0.3x series all the PNG with transparent pixels rendered correctly.

[Edit: Improved this paragraph, for clarity; thanks for the comments:] Initially, influenced by the #1303 feature request, I supposed the automatic conversion would be messing with the alpha channel. However, I was able to confirm that the problem seems to be with the rendering of the image in the rich text node, as discussed in detail in the posts below. In any case, the resulting image is absolutely terrible.

To Reproduce Insert or paste any PNG image which has transparent pixels, such as any standard png icon like for example /usr/share/icons/hicolor/64x64/apps/firefox.png (from an Ubuntu installation). The result for that file SHOULD look like this:

... but instead, it actually looks like this (screenshot from a Cherrytree node):

[P.S. Thank you so much for this magnific program!!]

[userfav]

(another user here) Maybe a Linux thing? Can't replicate on Windows 10 (v. 0.99.24)

This image was inserted to cherrytree, saved the changes, and the copied from the editor to another software and the transparency was mantained:

Had you tried to copy the image to a external editor (Gimp, for example) to see if the image keep the transparency or it is a cherrytree preview problem?

[dardoor]

I can reproduce this on Windows. Transparency is not maintained inside the CT rich text node window. This is more obvious with a dark background, at least in my case, because CT replaces transparency with a white-ish color.

I believe this is what the OP meant, not exporting the PNG from CT. Exporting does preserve transparency for me too.

[userfav]

@dardoor Nice, so the alpha-channel was not lost (for a conversion problem), is a problem with the preview on CT.

[ocumo]

Thank you for your comments.

The problem is that the Cherrytree rich text pane does not render the image correctly and it looks terrible. It's not a preview, it's the actual, rendered page.

Not to distract on semantics, but when I said "lose transparency" I meant "areas that should look transparent do not". I only imagined (a bit hasty, sorry for the inaccuracy of language) "alpha channel is lost" in conversion influenced by #1303 (and also #226). The bottom line is: while I cannot say exactly what happens to the "alpha channel", it -at least- is misused/ignored and looks very bad. That's the bug.

By now it should be "crystal clear" ( :-) ) that there is a problem in the rendering of the images.

Note that I am not proposing what is wished in #226, (i.e. that the original image should be preserved exactly in size/quality). That would mean that Cherrytree would become some kind of storage/archive of images, which I think would be wrong for many reasons, including missing the whole point of the program and why it's so useful as is. However, I am saying that the images should not be destroyed either, when rendered.

By the way, yes, I can confirm that copying the firefox logo image that looks terrible in Cherrytree, with no transparency, (right click on it, select "Copy Image") and pasting it in Gimp, then it does have transparency in Gimp. That's interesting, but useless for my use case: I wouldn't use CT to transfer or export images.

Edit: I have modified one paragraph in the original post, to improve on clarity. Thanks!

[ghost]

@ocumo You can read this one for background: Images management / investigations #1174

[ocumo]

@ciranus Thanks. That's interesting. It shows that there are so many ways and use cases for a program like CT.

I have always known that dealing with images other than just rendering them correctly, opens a whole new world of complexities that I would not expect a program like CT should embrace at this early stage. #1174 shows a few of those.

It's not nice of me to post here about that one, but that's closed. To be honest, the geek in me would love things such as those suggested. I however must say that for many people I have influenced to use CT, non-technical users, it would be bad to make CT more complicated in any way. That is an extremely powerful feature of CT: its simplicity. A lot of people are not looking for a sophisticated database-like program. They just want to document their notes or ideas without having to learn many things. As for us, geeks, yeah. Perhaps we could propose plug-ins or so? That could satisfy both worlds...

My typical use case since years involves taking notes and creating documentation for myself, using the formatting possibilities very sparingly, and inserting images also when they document something. For example, in a tutorial, I may insert some snapshot of some graph or dialog, but I try it to be small because of the size of the database. So far, I have not needed to search images, but that is only in this use case.

That's probably why this regression is so bad for me. I do frequently use png icons to e.g. highlight a paragraph as "Warning!" or "Reminder", and so on, and all of those icons are not square, they have irregular shape and thus have transparency. Because this bug, almost all of my nodes look absolutely horrible.

I see a huge amount of requests and whishes, but I am more concern of serious regressions like this one, which is probably not popular because I suppose not many people would be inserting png images with irregular shapes, such as icons in their nodes. But: it is a bug, a regression, it used to work and I think this is a really bad, show stopper regression for users that won't understand why some images look OK and other look terrible.

[ghost]

To come back more precisely to your firefox icon, this is what I see in CT:

Transparency is preserved (white background).

To convert a png to data/image, you can use this command to investigate

echo "data:image/png;base64,$(base64 -w 0 /usr/share/icons/hicolor/64x64/apps/firefox.png)"

and verify the transparency is still there in Firefox. Just copy 'data:image/png;base64........' as an url.

Now reverse operation, to see how the image is stored in CT, create a test node in CT, copy your icon in it, and then export the test node as CT document, then xml format (=> test_node.ctd file). Open this exported file in a text editor, and you will see something like:

<encoded_png char_offset="2" justification="left" link="webs data:image/png;base64, ...> 
</encoded_png>

Some secrets might be hidden there to understand.

[ocumo]

Ok, useful information, thanks! However, I am not sure I follow you 100% -perhaps the late hour...(ugh.. >> 3 AM !) So I did the experiment you propose - by the way, very cool the way to create the data URI for an image in the console; I didn't know the base64 command! . Here are my observations:

1. My exported node's <encoded_png> tag is not like yours. In your post, you show the data URI as the value of the link attribute, and includes the full data:[<media type>][;base64],<data> scheme, preceeded by webs. That puzzles me, because in my case, the data is not an attribute, it is the actual content of the <encoded_png> element. More explicitly:

<encoded_png char_offset="59" justification="left" link="">iVBORw0KGg...</encoded_png>

Interestingly, neither data:image/png;base64, preceeding the actual data, nor webs. In fact, I tried prepending data:image/png;base64, to the actual data, saved the file and tried importing it in CT, and... it just crashed immediately. Also, out of pure curiosity, I tried moving the data to the link attribute and importing it, and CT dies instantly. So I wonder: is the code in your post correct or am I missing any thing?

2. I don't have experience using XML, (thus my confusion here) but I have seen things like:

`<?xml version="1.0" encoding="UTF-8"?>

iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR42mP4z8AAAAMBAQD3A0FDAAAAAElFTkSuQmCC

where the mime type is an attribute, and the data is the element content. It makes sense to me. And of course, in HTML, CSS and SVG, the data URI always starts with:data:[][;base64],, as in e.g.data:image/png;base64,iVBORw0...`. If we omit that, then it doesn't work, in those protocols.

3. I would imagine, that the encoded_png element would not require a mime attribute if the element internally takes care of that, thus its name. I guess that is parsed by GTK, but I don't know GTK, sorry. But none of this, so far, rings an obvious bell that could explain why the transparency of this png is not rendered on the node pane. Well, unless... this would be so silly as a missing attribute with the mime type causing encoded_png to default as e.g flat jpg? It wouldn't be intentional, would it?

4. Finally, I don't understand why you would have transparency rendered correctly: different versions? In a previous post, @dardoor wrote:

I can reproduce this on Windows. Transparency is not maintained inside the CT rich text node window. This is more obvious with a dark background, at least in my case, because CT replaces transparency with a white-ish color.

Could it be that, in your case, transparency is replaced by some light color that appears transparent against a ligth background? Can you type several blank spaces before and after the icon, then select all and change the background color and see if the icon does have transparency?

[ghost]

1. My exported node's `<encoded_png>` tag is not like yours.

Sorry, we have the same. I just shortened a bit too much the content between <encoded_png ...>... tags. This content is not clear to me, and I don't understand the redundancy of image data code, and why it is necessary (it doubles the size).

I also had a look in these files src/ct/ct_image.cc & src/ct/ct_image.h , but did not help me. I don't understand the gtk png->data conversion mechanism.

Well, I cannot tell more. Needs more expertise.

[ocumo]

Oh, that's fine, we are just trying to understand and in the process we learn a few bits, so that's cool. We don't have to fix the bug ourselves :-)

However: I just don't understand whether you can reproduce the bug in your system. This comment seems to imply that your system doesn't have the bug:

To come back more precisely to your firefox icon, this is what I see in CT:

Transparency is preserved (white background).

Could you please confirm whether inserted e.g. png icons such as the one above, do render correctly the transparency, no matter what background color (i.e. transparent pixels have not been converted to some color similar to your background as @dardoor explained)?

[ghost]

This comment seems to imply that your system doesn't have the bug

That's correct. I don't see your 'bug', or let's say issue, in a linux environment (I don't have windows to compare). But I don't precisely understand how images are converted and stored in CT. You understood by reading #1174 that I already tried to better understand this CT image processing some time ago.

With CT, I don't store any images (huge impact on database growing too fast), but just some very small snapshots. When I save an 'image', I want to be sure I will easily find it weeks after, and then prefer to store them in a separated images folder. Then very small snaphots used for illustrations only, or saving images is a different requirement.

[ocumo]

Then if there is a linux environment and a Windows environment in which the regression is happening, and there is a linux environment in which the regression doesn't happen, it would be interesting to know what's different.

Could you say what linux distribution are you using? and just to be sure, your Cherrytree version too? Any special configurations? Mine is ubuntu 20.04 upgraded from 18.04. But I have the exact same problem in another machine, a laptop with much better hardware, also on ubuntu 20.04.

I wouldn't mind creating a Virtual Machine with another distro just to test it and try to see what I can find.

[ghost]

I currently use Debian testing, KDE-Plasma 5.19, libgtk-3-0 3.24 . No specific gtk3 theme added.

~/.config/gtk-3.0/settings.ini

[Settings]
gtk-application-prefer-dark-theme=false
gtk-theme-name=Default
gtk-toolbar-style=0

These are the relevant info I think. Not sure it helps to clarify.

[userfav]

I was reading the comments and it happens that recently I installed Manjaro XFCE on a virtual machine, so I tested what happens when I change xfce themes. I don't know if this helps but I noticed that when a "updated" theme is used, the colored background appears, and when an old theme is used (installed randomly) there is no background (that is, looks transparent) Examples:

Updated themes (and opaque background):

Old themes (and appears to have a transparent background, the light blue is the line selection color):

Note: also tested with darker rich text themes and also appears to look transparent. As a side effect of the test, old themes generate the problem with the toolbar mentioned on the issue #1419

[ghost]

More precision of previous analysis in my environment. It appears that the image is in fact printed in a square which background color is #fbfafa . Understanding how this extra layer is created should explain.

[userfav]

Following that in my tests on Manjaro the background color of the image changed according to what gtk theme was selected, I tested on Windows using a defined gtk theme (here I comment how). I set for this example the gtk theme "deepin-dark" for cherrytree on Windows.

By default this is how it looked:

So I searched for #fafbfb (thats the color of the background of the image in this case) in the "gtk.css" file of the theme that I was using (deepin-dark) and I found:

}
textview border {
    background-color: #fafbfb;
}

So I replaced that value with the word "transparent" in this way:

}
textview border {
    background-color: transparent;
}

Re-open Cherrytree and this is how it looks now, without the background color:

Maybe can be possible to override on Cherrytree this specific value for any theme to get a transparent background?

[ghost]

What is strange in my case is that I did not find this "#fbfafa" color or "textview border" string anywhere in my system or any gtk.css or color.css. As I don't have any specific gtk theme installed, I wonder if it does not come from a gtk module or lib used by default.

[userfav]

@ciranus Just to be sure, I did the same on a Linux Mint installation (that is: going to the gtk.css file and set "transparent" on the "textview border" setting), with their default gtk theme (Mint-Y-Dark) and could be replicated:

Before:

After:

[ghost]

Yes yes. It did not work because of your first extra "}" character I just copy/paste without enough care.

Specific cherrytree gtk css tunings can be added in a separate ~/.config/gtk-3.0/cherrytree.css file containing:

textview border {
    background-color: transparent;
}

which can be called by ~/.config/gtk-3.0/gtk.css with this first inserted line

@import 'cherrytree.css';

[ocumo]

Ok, very interesting comments and suggestions!

Update: These first block of lines is no longer valid, my mistake. Please skip this redacted block: From the previous two posts (about the background-color: transparent change), I tried @ciranus' procedure -- a cherrytree.css file @imported it in gtk.css. I supposed I needed to restart X to see changes, so to simplify, I rebooted. No visible changes in Cherrytree, though. So I restored the configurations. Then, I edited the gtk.css file per @userfav's last comment, out of curiosity, by appending the css rule textview border { background-color: transparent; } (there was no textview border selector rules before). Again, I rebooted. No changes.

Please continue here:

I tried changing xfce themes, like @userfav's experiments. Now finally I got something of interest:

Turns out, that from the themes I have available, the only one that appears to support the transparency on the rich text pane, is the Xfce-Simple-Dark theme.

Note that the yellow background (few blank spaces before and after, selected and changed background) was just to prove that there is transparency around the logo, not opaque pixels.

With all the other themes, (being Breeze-Dark the one I have been using (*see footnote)), the result is always bad:

Note that with the High Contrast theme the upper icon gives an "illusion": but the bottom one proves it's bad too (as I suspected it was @ciranus 's case here).

This issue seems to have some relation to the GTK themes in use, and potentially with conflicting Cherrytree CSS configurations.

Like nearly every other Linux user, since ever, I have battled :angry: before with getting Qt applications looking OK in GTK ambient and viceversa. To the day, one needs a lot of black magic and voodoo to get it OK in most cases, not all .

However, this is not the same issue: This is GTK ambient and Cherrytree is GTK. So: OMG, what's happened to our esteemed program? :confused:

Well, next, I will try the Xfce-Simple-Dark theme. I must test it because I have had a lot of work to find a theme that works for all GTK and Qt applications and still looks good to me (I need a dark theme); so far, the only one that works is Breeze-Dark (until now!).

Thank you all for your interest and investigations in this issue so far. I do use this program a lot, really. I need to see my rich text looking nice with all its icons correctly rendered!

---

NOTE: I have both KDE Plasma and Xfce Desktop in the same box. That's why I use Breeze-dark, which is the only one in Plasma that works fine in Xfce. Every other thing I have tried in Plasma, sucks in Xfce. My most used ambient is Xfce, though.

[ocumo]

When I have some time, I still have to do some testing to the previously discussed textview border { background-color: transparent; } edit in the gtk.css file of the theme.

[ghost]

My experience with gtk used in KDE is that gtk is totally over-complicated. Too much time wasted with gtk themes and spending hours to find the right parameters. Example: Breeze/gtk-3.0/gtk.css is a 3.5MB file !!!!!!

This is why I don't use any specific gtk3 theme. Not a single one, but the 'default one' (even not clear which one it is).

$ dpkg -l '*gtk*' |awk '/^i/ {print $2}'

gir1.2-gtk-3.0         libgtk-3-0       libgtksourceview-3.0-1       libgtkmm-3.0-1v5
gir1.2-gtksource-3.0   libgtk-3-bin     libgtksourceview-3.0-common
gtk-update-icon-cache  libgtk-3-common  libgtksourceviewmm-3.0-0v5

The '#fbfafa' background-color I had identified as "the" problem was found with a color-picker. Impossible to find where this '#fbfafa' color is hidden in my system.

Result with default theme. No trouble with this transparency "issue", which is a gtk theme issue, and not an image loosing its transparency.

[ocumo]

@ciranus, I think you are absolutely right.

It is a rather nasty issue. I just want to make it clear that this is not about terminology/semantics. And it will be wrong dismissing the real issue. What's that?

The issue persists and I doubt that there would be a positive outcome for normal users, unless those users would clearly identify what's the real issue and address that instead.

Here is what and why, IMO:

We are in a three party conflict: the program developers, the (GTK|Qt|Whatnot)-(framework|library|theme|whatever) developers, and ..."We", the Users.

So, Cherrytree devs might come and say: "Hey, that's not our problem: that's the (GTK|Qt|Whatnot|...) folks who make everyone else's life difficult with their aversion (repugnance?) to compatibility and sound standards". It would be difficult to argue against that.

But then, "the other" folks might respond: "Hey, that's not our problem either. Apps devs should make their programs work correctly on certain framework and tell their users to stick to that. We don't (support|tolerate|accept|like|understand|want|care for|...) compatibility with competitors". It would be difficult to argue against that.

And then, there is the "last" party: "We", the USERS. And very specifically, and more importantly, the non-technical users. It is, always, our problem.

We could come and say:

• "_Hey, the program was working before the upgrade to version XYZ. The 'upgrade' broke it; it's a regression_". Or:

• "Hey, the program is written in (GTK|Qt|Whatever platform|theme|framework), so ...what the hell? It should work at least in that very same environment!". Or:

• "Hey, I use and love the (GTK|Qt|Whatever platform|theme|framework), but that thing does NOT support any longer my favorite program: it looks terrible in it!"

It would be extremely difficult to argue against any of that.

So: What to do?

Here are our options:

a) Each of the three parties could indefinitely keep shouting their conflicting reasons, confronting their views with logical but opposing arguments, never any of them "surrendering" to the other parties. Kind of, more or less, what has been the case since ever.

b) One of the parties could "surrender" or, better said, admit that at least one of the other parties, or both, may have a reasonable point. Therefore, act upon to resolve/mitigate the triangular conflict. That means either:

• Devs fixing their (program|framework|whatever) so that it works with the other (program|framework|whatever). That is, work on compatibility and usability. Or:
• Users, quit complaining and drop one of the conflicting (program|framework|whatever) from their lives. Accept and embrace that compatibility and usability are utopic in this industry. Learn to live with discovering incompatibilities on their own, the hard way, and move on to something else when you do. (Sorry, aunt Gertrude: computers are not for you).

c) Devs for either (both?) "sides" explicitly document what is it that their (program|platform|framework|library|theme|whatever) do support and do NOT support and what are the requirements to use it; then, take all steps to honor that. In other words: set and document realistic and measurable expectations. Users could then check easy-to-find documentation with "Requirements", "Not supported", and "Known Issues", information and avoid complaining about out-of well defined boundaries and scopes. A simple: "Warning: this (program|platform|library|whatever) does NOT support (foo|bar|baz). You must (not)? (do|use) (blah|bleh|blih)." ...would be extremely useful and frank.

That's why Microsoft don't get complaints about Windows not assigning letters to the Linux partitions.

d) All the parties let go, forget about it, and each one live with their own issues and frustrations: ranting about if things don't work for them, or happily ignoring others' troubles when it "works" for themselves, each one in their own little world. In other words, the "I don't care for the others" way of living, so popular nowadays.

e) ... (Anyone has an idea for another option?)

[giuspen]

will be fixed in 0.99.51