Image clipping and combining support

[johnspackman]

We need support for image clipping and combining in the ResourceManager, to support the functionality here: http://www.qooxdoo.org/5.0.1/pages/development/image_clipping_and_combining.html

[johnspackman]

Given that the generator does image combining and clipping into the resource directory (and therefore the results would tend to be committed as part of the source), if the compiler took a different approach (ie fully automatic) can we avoid reimplementing the generator method? EG the generator method is deprecated but still available for existing projects because the generator is available?

For image combining, it should be entirely automatic for the compiler to be able to add images into one big image and keep track of offsets; when the code refers to an image which has been copied into a combined image, the framework would be able to select the correct part with nothing more that the original image resource path.

Combining would be done based on rules which are provided in config.json and done on a path basis, for example:

    resources: [
      "$defaultIcons": {
        "filesize < 100": "inline", 
        "imageWidth <= 50 && imageHeight <= 50": "combine"
      },
      "qx/decoration/Classic/**": {
        "filesize < 100": "inline", 
        "filename.startsWith('qx/decoration/Classic/colorselector/')": "none", 
        "imageWidth <= 50 && imageHeight <= 50": "combine"
      },
      "qx/decoration/Indigo/**": {
        "filesize < 100": "inline", 
        "filename.startsWith('qx/decoration/Indigo/colorselector/')": "none", 
        "imageWidth <= 50 && imageHeight <= 50": "combine"
      },
      "qx/decoration/icon/Oxygen/16/**": "$defaultIcons",
      "qx/decoration/icon/Oxygen/22/**": "$defaultIcons",
      "qx/decoration/icon/Oxygen/32/**": "$defaultIcons",
      "qx/decoration/icon/Oxygen/48/**": "$defaultIcons",
      "qx/decoration/icon/Oxygen/64/**": "$defaultIcons"
    ]

In this hypothetical example, the rules are mini expressions which are just javascript expressions; well defined variables exist such as filesize, imageWidth, imageHeight etc. When a match is found for the path and for a rule, the mapped value can be either inline (to convert to base64 inline encoding), combine (to combine into a single file for the path), or none (to allow for exceptions to the rule.

The last few paths (beginning "qx/decoration/icon/Oxygen/...) don't have an object of rules, instead it has a string which is used to match against the list of paths - by specifying an impossible path like $defaultIcons this allows for simple reuse of rule sets between paths.

For image splitting, the same process should work but would use the original image and meta files to create virtual paths that use the original, pre-combined image.

[cboulanger]

Sounds great! I hope that most of what it takes to implement the image processing / CSS stuff is already available on NPM and you would only have to add glue code that uses the rule syntax above to the exeute the neccessary library API methods. one thing: maybe you might want to use "file.name" and "image.with" in the javascript expressions? I find that easier to remember than to have to think about "Was it fileName or filename vs. imagewidth or imageWidth??"

[gnikolaidis]

We are migrating our tool-stack from the old generator to the node-based compiler and everything works out quite well so far. The lack of the image combining feature though is posing quite a problem. Do you think it is going to be ported soon? If not, are there any alternatives that you can point us to for us to investigate?

[johnspackman]

Unfortunately this is something that keeps slipping due to other pressures and I don't have any time to look at implementing this soon enough for your needs.

However, if you have a lot of investment in split/combined images, and as the original generator based split/combine is based on imagemagick, it should be fairly straight forward to implement something yourself which batch splits/combines.

For splitting, you just have to output the portions of the image, for combining there needs to be a JSON structure output that can be incorporated into the resources.

If you can write something in node.js to do this, I can help you get it integrated into the compiler code?

[gnikolaidis]

I am unfortunately not familiar enough with node.js, so I toyed a bit with the "montage" command of ImageMagick and performed some XSLT wizardry on its HTML output. Everything is wrapped in an ANT build script.

My experimental setup results in the exact same content as what the Python tool-stack was generating, i.e. one big PNG in the resources folder, and an associated .meta file with JSON information for each named resource - see below example

{
  "waffle/icons/fatcow-16/accept.png": [ 16, 16, "png", "source/resource/quadro/combined/180/fatcow-16.png", 0, 0 ],
  "waffle/icons/fatcow-16/add.png": [ 16, 16, "png", "source/resource/quadro/combined/180/fatcow-16.png", -16, 0 ],
  ...
}

Do you think it is possible to integrate this information into the new ResourceManager without too much hassle from your side?

[cboulanger]

Could this be plugged into the compiler using a package that contains a compile.js that hooks into the compiler workflow and calls the ANT script?

https://qooxdoo.org/documentation/6.0/#/development/compiler/configuration/api?id=events-and-hook-methods

Then you simply need to make this package a dependency of your app to get image combining. This would be a quick-and-dirty way that doesn't require us to define any format or protocol at this point, but would make it possible for others to use it as well. This way we don't have to commit to any particular schema or implementation.

[cboulanger]

Also, @gnikolaidis can use what he has already written and the ANT dependency isn't a problem because it is a package and not a core component.

[gnikolaidis]

As long as the .meta file is properly read and used at runtime (I didn't have the time to test that), the current ANT-based setup is doing the job really well for us for now.

I have the feeling that it is not the compiler's job to perform the actual combining, as there are too many use cases and accommodating for everyone's needs will be a daunting task. It would be sufficient to write some documentation about the meta file, i.e. its format, its place in the directory structure, its use at runtime.

In the future, a solution like @cboulanger suggests is probably the way to go, as it separates concerns, doesn't add unnecessary dependencies, and fits well within the compiler's pluggable architecture.

[gnikolaidis]

Thanks to a dive into the compiler code, namely qx.tool.compiler.resources.Manager, and some luck, I managed to get image combining to work! In my .meta sample file above, I had to remove the "source/resource" prefixes from the composite image paths (position 3 of each array). Thinking again about it, it was an obvious mistake from my part- the platform runtime works with resource names instead of paths.

BUT I stumbled upon what I believe is a bug of the compiler: the first time I build the project with --clean, composite information ( the "composite", "x" and "y" keys of combined resources) is missing from resource-db.json. Invoking the compiler a second time, without --clean, makes them reappear.

# start with a clean build
> qx compile --target build --clean
> grep composite compiled/build/resource-db.json
# image combining information missing from resource-db.json
# invoking the compiler a second time, without --clean this time
> qx compile --target build
> grep composite compiled/build/resource-db.json
# image combining information found in resource-db.json, and properly working at runtime

This is 100% reproducible; the qx/decoration/... split PNGs of the stock themes display the same problem.

[johnspackman]

@gnikolaidis glad you've got a solution!

Please file an issue for the --clean bug

[rad-pat]

A gist for some basic horizontal image combining using the new tooling, inspired by comments above. https://gist.github.com/rad-pat/c568fa4467f85f56beb0a53bf71d9ec1