EMF - problems with colorscale

[antoinerg]

A user reported that the color scale doesn't render properly in an EMF file produced by Orca:

More details about how this result was obtained will follow.

[antoinerg]

The issue lies with the specific build of Inkscape that is packaged in our Docker container.

Back when we merged https://github.com/plotly/orca/pull/152, building our Dockerfile would install Inkscape 0.92.3 which was tested to work properly. https://github.com/plotly/orca/blob/a2ce478735857ca6bcc03ebc798de017e87628bb/deployment/Dockerfile#L112-L118 Today, building the same Dockerfile installs Inkscape 0.92.4. Although the newer 0.92.4 works fine on my local machine, the particular build retrieved on the PPA ppa:inkscape.dev/stable does not work properly and screw up the color scale :open_mouth:. This made the problem very hard to pinpoint.

Here are a few thoughts that came to my mind:

• We should consider compiling Inkscape ourselves and pin its version to ensure our docker build are as reproducible as possible in time.
• We should run our tests on the final Docker image we produce and not some throwaway environment like we're currenty doing
• We should run image tests on Orca and potentially use that in plotly.js as well
• Because we need to clean the SVG before converting it to EMF, a plotly.js update that changes the structure of the SVG might break things. Again, we need to test that.

[etpinard]

• We should consider compiling Inkscape ourselves and pin its version to ensure our docker build are as reproducible as possible in time.

That sounds like the simplest and most effective thing to do to start :ok_hand:

[etpinard]

• Because we need to clean the SVG before converting it to EMF, a plotly.js update that changes the structure of the SVG might break things. Again, we need to test that.

I think we talked about this before, but another way to do this would be to move the SVG -> EMF conversion code in plotly.js so that Plotly.toImage(gd, {format: 'emf'}) would do the right thing almost. That should make things much easier to tests.

[antoinerg]

It turns out I misidentified the issue at hand. Running the exact same Inkscape build in a Docker container (or a virtual machine for that matter) yields a different result than on my machine. I suspect Inkscape, or one of its dependencies, is making a system call that gives a different result depending on the hardware (probably the graphics cards?) it is running on.

I guess we shouldn't be surprised that producing figures in a Docker container gives different results considering OpenGL (and possibly other API) is not a pixel-exact specification. We should probably use a recent OpenGL software renderer, test it and pin it once and for all.

To be followed...

[antoinerg]

I made a gist containing a cleaned up SVG ready for EMF conversion to showcase the problem.

To test the conversion routine, install Inkscape 0.92.3 or newer then:

$ git clone https://gist.github.com/antoinerg/1ab82ce661c47acadfb7aa84f1a0e76b
$ cd 1ab82ce661c47acadfb7aa84f1a0e76b
$ ./convert.sh

Then look at the color bar in the resulting PNG file.

On my host machine I get:

On a guest machine (be it a Docker container or a VM), running the exact same Inkscape build I get:

[archmoj]

@antoinerg On Ubuntu 18.04.2 LTS with Inkscape 0.92.3 (2405546, 2018-03-11) I've got this:

[archmoj]

@antoinerg On Windows 7 with Inkscape 0.92.3, this is produced.

[antoinerg]

@archmoj Thank you for reporting this. I think I found the issue!

[antoinerg]

The reason it was working on my machine is that I manually set sane preferences via the GUI a while ago. There a bunch of flags to fix rendering bugs of Inkscape's EMF output in Powerpoint.

See the FixPPT* options below:

  <group
     id="extensions"
     org.inkscape.output.emf.FixPPTGrad2Polys="1"
     org.inkscape.print.emf.FixPPTCharPos="1"
     org.inkscape.print.emf.FixPPTDashLine="1"
     org.inkscape.print.emf.FixPPTGrad2Polys="1"
     org.inkscape.print.emf.FixPPTLinGrad="1"
     org.inkscape.print.emf.FixPPTPatternAsHatch="0"
     org.inkscape.print.emf.FixImageRot="0"
     org.inkscape.print.emf.textToPath="1"
     org.inkscape.output.emf.FixPPTPatternAsHatch="1"
     org.inkscape.output.emf.FixPPTLinGrad="1" />

I will submit a PR in which we copy those preferences over to the Docker image we build.

[scjody]

@antoinerg The example "Choropleth" trace in Chart Studio uses a colourscale. Is verifying that this works in EMF sufficient to test this issue?

[antoinerg]

@scjody yes it should does!