make PixelWindow able to display bitmap images, add jpg read/write to IO

[bjornregnell]

If we want the ImageProcessing project to use introprog.PixewlWindow this is a must-have.

[bjornregnell]

@theolundqvist Could you investigate what it would take to do this and report here? Don't knwo yet if it would be worth it for the img_project

[theolundqvist]

jpg read/write should not be too much trouble.

PixelWindow can be modified to draw bufferedImage directly with Graphics2D.drawImage. Drawing arrays of colors could be done easily if first converted to bufferedImage.

Should be careful when drawing sprites every frame so that conversion colorArray->bufferedImage isn't needed every frame.

edit: java.awt.geom.AffineTransform can be used with drawImage for applying transformations to images. scale, stretch, rotation and flipping.

[bjornregnell]

Just merged! Thanks! Do you need an official release to continue or can you test this library with the new photo lab (was imageprocessing) before we make a new release om maven central to be sure everything works as expected? You can always package introprog-scalalib in sbt and put the jar in lib in the build of the lab while testing... Or even simpler, use publishLocal. Let me know here if everything works as expected and I'll close this issue after publishing the new release then.

[theolundqvist]

Just merged! Thanks! Do you need an official release to continue or can you test this library with the new photo lab (was imageprocessing) before we make a new release om maven central to be sure everything works as expected? You can always package introprog-scalalib in sbt and put the jar in lib in the build of the lab while testing... Or even simpler, use publishLocal. Let me know here if everything works as expected and I'll close this issue after publishing the new release then.

I'm still doing some work on translating ImageFilter.java to Scala and updating the textbook. Will update this issue when I get everything to work together.

1. Should ImageFilter.java be deleted from cslib when ImageFilter.scala is added to introprog-scalalib?
2. Should updates to introprog-scalalib be pushed to both git/introprog-scalalib and git/introprog?

[bjornregnell]

1. We leave cslib as is. It is legacy never updated.
2. When introprog-scalalib is released I make a copy of it into introprog using a script, so whatever is in introprog's introprog-scalaliv gets overwritten during release. But if you want to update introprog to make lab testing easier you can do that?

[bjornregnell]

I just got an idea: Instead of exposing the java image api, perhaps introprog IO should give a simple Array[Array[java.awt.Color]]? And then PixelWindow could take a simple Array matrix of Color as input. Would be less magical... What do you think?

[bjornregnell]

Or we could do our own simple ColorMatrix class backed by an underlying Array matrix. Or even simpler type ColorMatrix = Array[Array[java.awt.Color]], but then we don't get protection from not creating a true matrix. Think about the pros and cons of each option from a pedagogical point of view and we can discuss them:

1. using java.awt.image.BufferedImage
2. using Array[Array[java.awt.Color]]
3. using type ColorMatrix = Array[Array[java.awt.Color]]
4. using

   class ColorMatrix private (underlying: Array[Array[java.awt.Color]]):
   def apply(x: Int, y: Int): java.awt.Color = ???
   def update(x: Int, y: Int, c: java.awt.Color): Unit = ???

   Also with suitable factories in the companion creating a color matrix from a Seq[Seq[Color]] and also perhaps from parts of a PixelWindow. And the update method above wolud make a nice matrix usage like m(x,y) = c

[bjornregnell]

Some performance measurements for reference: https://stackoverflow.com/questions/6524196/java-get-pixel-array-from-image I'm inclined not to bother about performance, but if we do want it to be as fast as possible we perhaps should have our own ColorMatrix being backed by a buffered image and do all the bit shift calculations hidden inside... The performance gain seems to be in the order of 10x, see the convertTo2DWithoutUsingGetRGB method in the above SO accepted answer.

[bjornregnell]

So we have a 5th option: in file "ColorMatrix.scala"

package introprog

class ColorMatrix private (underlying: java.awt.image.BufferedImage):
  import java.awt.Color
  def apply(x: Int, y: Int): Color = ???  
  def update(x: Int, y: Int, c: Color): Unit = ???

  def update(f: (Int, Int) => Color): Unit = ??? // for x <- ... do for y <- ... do update(x,y, f(x,y))
  ...
  def toImage: java.awt.image.BufferedImage = underlying
object ColorMatrix:
  def fromImage(img: java.awt.image.BufferedImage): ColorMatrix = new ColorMatrix(img)

extension (img: java.awt.image.BufferedImage) def toColorMatrix = ColorMatrix.fromImage(img)

img.toColorMatrix

[bjornregnell]

object pixel:                                                            
  import java.awt.Color
  opaque type Pixel = Int
  inline def Pixel(i: Int): Pixel = i
  inline def Pixel(red: Int, green: Int, blue: Int, alpha: Int = 255): Pixel = 
    alpha << 24 + (red & 0xff) << 16 + (green & 0xff) << 8 + (blue & 0xff) 
  extension (p: Pixel) inline def toInt: Int = p
  inline def Pixel(c: Color): Pixel = c.getRGB
  extension (p: Pixel) inline def toColor: Color = Color(p)
  extension (p: Pixel) inline def blue: Int = p & 0xff   
  extension (p: Pixel) inline def green: Int = (p >> 8) & 0xff
  extension (p: Pixel) inline def red: Int = (p >> 16) & 0xff 
  extension (p: Pixel) inline def alpha: Int = (p >> 24) & 0xff           

[bjornregnell]

Let's use option 5 with underlying java image, e.g.

val cm = ColorMatrix.empty(100,100)
for i <- 0 until cm.width do
  for j <- 0 until cm.height do
   cm(i,j) = Color.green

[bjornregnell]

After some though I'm inclined to go for simplicity and renaming ColorMatrix to simply Image and simplify it as:

package introprog

class Image(val underlying: java.awt.image.BufferedImage):
  import java.awt.Color
  def apply(x: Int, y: Int): Color = ???  
  def update(x: Int, y: Int, c: Color): Unit = ???

And in introprog.IO give/take our own introprog.Image on load and save.

I also think we need a scheme for giving the image type PNG /JPG on load/save. Either the image type is baked into the method names as in IO.loadJPG and IO.loadPNG or we have a method loadImage that takes a parameter ImageType which is an enum ImageType { case JPG, PNG }

What do you think? @theolundqvist @daghemberg

[bjornregnell]

Also, when saving JPG there is the issue of compression, so perhaps save only as png is reasonable option for now; but this requires some digging into the java...ImageIO api to be sure what is best/simplest for the user. The risk of saving a png is that it will be inflated to take ALOT of disk if it was a huge but highly compressed jpg (which originally did not take much space).

[bjornregnell]

And saving to jpg should give to user possibility to choose compression level...

[bjornregnell]

Closed by #24 :tada: