Feature request: Sprite images

[am11]

Take a look at this article:

http://www.codeproject.com/Articles/210979/Fast-optimizing-rectangle-packing-algorithm-for-bu (now available at https://github.com/am11/RectanglePacker)

Courtesy of Matt Perdeck (@mperdeck). Thank you for your outstanding work! :) (please bring this project to a GitHub repo, so more people can get benefit from it)

I have been digging into the theory behind optimum sprite images. http://math.stackexchange.com/questions/629018/area-optimization-packing-rectangles-inside-rectangle and found @mperdeck's implementation extremely useful. The question on stackexchange was meant to gather knowledge about the tangible mathematical solution to the problem. Apparently, its a NP hard problem (going nowhere in real-time; coinciding travelling salesman problem).

Having said that, the idea really is to provide a convenient way for user to create and consume sprite images, when using Web Essentials.

On that note, I propose a JSON based map file with the following format for two-way Sprite manipulation:

{
    file-name: "path/to/sprite.png",
    contained-images: [
        image-info: {
            name: "image1.ext",
            width: (integer),
            height: (integer),
            startX: (integer),
            startY: (integer)
        },
        image-info: {
            name: "image2.ext",
            width: (integer),
            height: (integer),
            startX: (integer),
            startY: (integer)
        }
        /* and so on */
    ]
}

In Matt's solution there is a MappedImageInfo class implementing IMappedImageInfo interface, which reports the resultant coordinates of each image.

Now, continuing on #442, we can provide a SmartTag for HTML content type. So when the selection of images are dropped, user can select the option from smart tag to generate sprite and corresponding JSON based map with one click. This will shrink multiple <img .. /> tags into one.

The map file would preferably be nested under the image with same name and .json extension.

After that, we can provide another smart tag for CSS content type, so to provide a list of contained images for background-position coordinates (and the image preview of selection). This may trigger for any rule when background, background-image or background-position is used for the image with map file available.

Please share your opinions and thoughts, so we can provide the best UX for this feature.

Thanks.

[madskristensen]

This feature request is also captured here: http://webessentials.uservoice.com/forums/140520-general/suggestions/2525274-add-auto-sprite-capabilities.

I don't see sprites have anything to do with dropped images. Instead, you probably want to select n number of files in Solution Explorer and right-click them to invoke the sprite generation. Just like for .bundle files.

We could call the sprite file for foo.sprite and then nest the output image file under it. It should also output a .css file with the right background-position for each of the files in the image sprite.

Also, spriting is useless without the output image being super optimized. Right now we only have the optimization logic found in the Image Optimizer extension and it isn't good enough. Google Page Speed is still going to complain about unoptimized images when using it.

Without output image optimization, we shouldn't build this feature. This is why I haven't built this yet, since it's been on the backlog for a looong time.

[am11]

@madskristensen, thanks for chipping in.

If you are referring to image optimization as in; with-no empty spaces, then the Matt's C# code on codeproject I referred earlier, has several approaches involving heuristics to solve this problem (time vs space optimization, all options are provided). It will draw the set images on empty canvas for us. Just needs configuration and call it with list of images.

If its about the resultant file size, as in image resolution, we can always do post-processing for reduce size and de-pixelating.

[madskristensen]

I'm talking about file size and we would need post-processing for that. What I'm saying is that without the post-processing step, the sprite feature makes no sense. The whole idea of image spriting is to minimize HTTP requests to speed up your website. So if the image isn't optimized, then it kinda defeats the entire purpose.

Is there a nodejs based image optimizer out there that is as good (or almost as good) as http://kraken.io?

[am11]

Also, for the "generating CSS with all background-positions", how about we provide a CSS smart tag so user can select from list of embedded images and see preview of selection for set of background properties? This way they can freely use it pretty quickly (as opposed to copy pasting from auto-generated CSS file).

[madskristensen]

Yes, either a smart tag or as part of Intellisense with a custom XAML control in the Presenter like the inline color picker. We could then use CSS code comments to point to the right sprite information so that the CSS file is kept up to date with changes to the .sprite file.

[am11]

Why not simply using RectanglePacker project from Code Project; to generate sprite? Its well written in C# and the license says we are free to use commercially.

If you look at the sample implementation of the project (MapperTestSite), Its using System.Drawing.Imaging to save images as PNG. We can always use Encoder.Quality prop to adjust quality.

[madskristensen]

We can use the RectanglePacker project, but we can't use System.Drawing.Imaging to minify the file size of the images. .NET has nothing built in for that.

[madskristensen]

For image optimization, people use command line tools such as PNGauntlet and Image Magic. The best in class is the algorithm developed by kraken.io

[am11]

@madskristensen, I created a repository RectanglePacker https://github.com/am11/RectanglePacker importing the code. The folder Mapper contains library and MapperTestSite is an ASP.NET test site (sample implementing the lib).

[am11]

I found another Sprite framework with image optimization: http://weblogs.asp.net/rchartier/archive/2010/08/09/sprite-and-image-optimization-framework-amp-dotnetnuke.aspx (referring to https://aspnet.codeplex.com/releases/view/65787).

[madskristensen]

It looks like the project is outdated and dead. There must be a node module for this somewhere

[am11]

ImageMagick has binaries for .NET http://sourceforge.net/projects/imagemagickapp/. We can compile it with .NET 4 5 1 and add its reference.

Also, we can down-sample raster image by adjusting Encoder Quality to reduce the file size, while retaining the image dimensions. The value of quality parameter is 1-100 (level).

[madskristensen]

I'm investigating possibilities with PngOut, pngslim and others now. ImageMagic alone doesn't help and we can't use Encoder Quality in System.Drawing for anything, since we can't modify the quality of the image - only optimize the palettes etc.

[am11]

@madskristensen, check out pngcs. Its open source Java's PngJ port for .NET (unlike PngOut, which is closed-source). Here is the wiki link http://code.google.com/p/pngcs/wiki/Overview.

[madskristensen]

Ok, for PNGs I think I found the optimal compression in terms of compression level and speed. It's to combine PngOut and OptiPng like this: pngout original.png optimized.png /s0 & optipng optimized.png -o7. It's really fast and compresses better than SmushIt.com and PunyPng.com.

For some images, PngOut is best and for some OptiPng wins. So combining them is needed. Next: Jpeg compression...

[am11]

@madskristensen, that's great!

Would we pack those exes in VISX?

[madskristensen]

Yes

[am11]

PngOut's license says:

• Redistributing, repackaging, or reusing the PNGOUT or KZIP executable is prohibited without the express consent of Ardfry Imaging, LLC, and a formal business agreement.

which, I guess, can be sorted out. :)

[madskristensen]

Oh damn.

It looks like JpegTran is best for JPEGs where images over 10K benefit from using Progressive optimization and images below 10K should use Baseline optimization.

[am11]

Great! JPGTarn is open source with a separate Windows source and binary packs http://jpegclub.org/jpegtran/. :+1:

[am11]

@madskristensen, do we need JPG at all? I mean; we will be drawing all the files on a transparent canvas, at the respective coordinates returned by RectanglePacker, which essentially would be a PNG image. CMIIW please.

[madskristensen]

No, we don't need JPG support for image sprites, but we need it for general image optimization.

[madskristensen]

Hmmm, JpegTran is not better than SmushIt.com. I'll keep looking

[madskristensen]

I'll implement the image optimization feature that the spriting can then hook into

[madskristensen]

So, are you calling the JSON file for foo.sprite?

Also, I'm wondering why you keep all the meta data such as height, width, and coordinates in there. This must be an generated output file, right? What is it for?

Shouldn't the generated output be 1) A .png file and 2) a .css file?

[am11]

@madskristensen, the proposed json format has two purposes.

1) It acts like a source map file; so humans and software (may be browsers use it in developer tools) can reproduce / infer the source images. 2) The intellisense can find this file and consume it, for the project-wide CSS. For non-project, or cross-project scenarios, any image file referred in CSS (background-image) with JSON (<same-name>.sprite), the info will be consumed by intellisense.

So having auto generated CSS may be pointless, if we have verbose json (.sprite file). Users can always get details on the required info of contained images: OffsetX and OffsetY for background-position and height / width for the container dimensions. Normally users require all four properties when dealing with sprites in CSS.

[am11]

The project is converted to .NET 4.5.1 https://github.com/am11/RectanglePacker

[SLaks]

BTW, you should put that on NuGet instead of referencing a local copy.

[madskristensen]

+1

[madskristensen]

I was thinking this structure:

/foo.sprite // This is the user-modifiable file ala. .bundle files
    foo.png
        foo.png.map // This is the map file with the coordinates

This would make it consistent with how LESS- and bundles files are structured.

[madskristensen]

Instead of having a Sprite folder in the root of the extension, why not move it under the Optimization folder. That's why I created it. I plan on moving minification and bundling under that folder as well

[am11]

@madskristensen, I moved sprite folder under optimization with https://github.com/madskristensen/WebEssentials2013/pull/474.

[am11]

@SLaks, thanks for the weighing in. Once its in working condition, we will build a Nuget package.

I never built a NuGet package, I would need your help. 8-)

Also, should we use NuGet for Zencoding, Markdown and CssSorter?

[SLaks]

Yes.

Publishing a NuGet package is not hard; see http://docs.nuget.org/docs/creating-packages/creating-and-publishing-a-package

Basically, run nuget spec in the csproj directory, edit the generated file, build your project in Release mode, and run nuget pack -Prop Configuration=Release.

See also http://www.hanselman.com/blog/UpdatingAndPublishingANuGetPackagePlusMakingNuGetPackagesSmarterAndAvoidingSourceEditsWithWebActivator.aspx

[am11]

@madskristensen, should we create .bundle file for sprite with same format?

For instance:

<?xml version="1.0" encoding="utf-8"?>
<bundle runOnBuild="true" output="sprite1.png">
  <!--The order of the <file> elements determines the order of the file contents when bundled.-->
  <file>/img1.jpg</file>
  <file>/img2.png</file>
  <file>/Common/EN/Images/img3.jpeg</file>
</bundle>

[am11]

@SLaks, thanks! I will look into it and try to pack them one at a time. :)

(Just for my understanding) In this workflow, why would NuGet be a good idea; given we are maintaining all those repos by ourselves?

[SLaks]

And, yes. https://github.com/SLaks/WebEssentials2013/commit/a6d3cc4f5aa1bb6b0af0bde1721e8a848022501c

[madskristensen]

I think we should call it .sprite instead and we don't need the runOnBuild attributes.

[SLaks]

So that we don't need to put DLLs in source control. It makes updates simpler. (nuget push, nuget update)

For more-closely-coupled projects like CssSorter, it may make more sense to include the source using git modules.

[am11]

@madskristensen, the bundle file is for generating sprite.

<?xml version="1.0" encoding="utf-8"?>
<bundle output="sprite1.png">
  <!--The order of the <file> elements determines the order of the file contents when bundled.-->
  <file>/img1.jpg</file>
  <file>/img2.png</file>
  <file>/Common/EN/Images/img3.jpeg</file>
</bundle>

and the corresponding .sprite file is the map to generated sprite, so software and users can identify the location of each embedded image (for one, we can extract /construct the source images with this information -- or even display image preview):

{
    File: "sprite1.png",
    Constituents:
    [
        SpriteMapConstituent:
        {
            Name: "img1.jpg",
            Width: (integer),
            Height: (integer),
            OffsetX: (integer),
            OffsetY: (integer)
        },
        SpriteMapConstituent:
        {
            Name: "img2.png",
            Width: (integer),
            Height: (integer),
            OffsetX: (integer),
            OffsetY: (integer)
        },
        SpriteMapConstituent:
        {
            Name: "img3.jpeg",
            Width: (integer),
            Height: (integer),
            OffsetX: (integer),
            OffsetY: (integer)
        }
    ]
}

Does it make sense?

[SLaks]

BTW, please do not add any settings until I finish porting them to ConfOxide. (which should be today)

[am11]

@SLaks, Ok. I guess we won't have settings for both the features (optimization and sprite).

[madskristensen]

@am11 I think we should call the XML file for .sprite and the JSON file for *.png.map. The reason for calling the parent XML file for .sprite is so we can give it a separate icon from .bundle files. Also, by calling the JSON file for *.png.map we follow the convention of the Source Maps currently in place for LESS/SASS/JS/Bundles

[madskristensen]

@am11 Another reason is that we can then more easily separate the bundle code from the sprite code

[am11]

@madskristensen, yes, it makes sense. Question: should we reuse the existing bundling code (refactor it, if necessary) or create independent methods -- for creating .png.sprite and .png.map in SpriteRunner.cs like this one?

[madskristensen]

The bundle code is a total mess right now and should be refactored into the Optimization folder. Then we can probably reuse some of the logic.

[am11]

The order of images may not be respected by packing algorithm. Which means, we can't claim:

<!--The order of the <file> elements determines the order of the file contents when bundled.-->

in *.png.sprite file.

[madskristensen]

Yep, so we shouldn't add that comment :)

[am11]

The static methods in Bundle files (for writing and updating bundles) look more like candidate of Helpers than Optimization. :confused:

Thoughts?

[madskristensen]

I think that the entire bundle logic need to be rewritten. After that, it might not be static or look like a helper