PHP too slow

[davekimble2]

As you may have guessed, I am trying to write an image editor. It attempts to mirror the functions of Paint Shop Pro that I use the most in my photographic work.

Now that GetData() and SetData() are working, it becomes clear that the whole concept of adjusting every pixel in an image is far too slow in standard PHP. Merely picking the pixels up and putting them back again takes at least 1.3 seconds for 1280x1024, and 2.1 for 1920x1080.

for($i=0; $i<$n; $i += 3) 
{   $r = ord($data[$i]); 
    $data[$i] = chr($r); 
    $g = ord($data[$i+1]); 
    $data[$i+1] = chr($g); 
    $b = ord($data[$i+2]); 
    $data[$i+2] = chr($b); 
}

Which makes me wonder whether the standard way of running PHP, with Zend and ?caching? is best suited for this task. Alternative engines? Or writing it in C and compiling it?

Any thoughts before I waste a lot of time on research would be helpful.

[cztomczak]

You might wanna have a look at http://www.php-cpp.com/

[davekimble2]

Thanks, that looks promising. Did I mention that I haven't used C++ before?

I've download and built php-cpp, and I've written my first .cpp, but g++ is fretting over parameter checking (I think): "error: call of overloaded ‘add(const char [18], char* (&)(char, int))’ is ambiguous" I don't know where "const char [18]" came from - (char, int) is what I wrote.

[jgmdev]

Since you already became familiar with wxWidgets, using it with c++ instead of php would feel almost the same.

[NetWielder]

You might consider approaching the task using pthreads for PHP. For example, you might divide the task into threads. Perhaps divide the image into rectangular regions and assign a thread to adjust pixels in each region. It's not the best option but you wouldn't need ramp up on C++.

http://php.net/manual/en/book.pthreads.php

I'm not familiar with Paint Shop Pro however you might want to also review the graphic manipulation functions offered by Imagick and GD. You may be able to leverage those functions instead of manipulation of pixels in PHP.

http://php.net/manual/en/book.image.php

[davekimble2]

Yes, it is a bit galling to see only 1 CPU core working on the problem, and 3 sitting doing nothing, but since this doesn't do any i/o, there's no pauses in cpu activity to take advantage of by multi-threading. Isn't that right?

A cursory look through available functions in GD suggests that they don't do manipulation of all pixels. I'm aiming to do Brightness, Contrast, Gamma Correction (not sure if PSP's idea of Gamma is the same as others') and Sharpen/Unsharp Mask (I've no idea how to go about that).

Imagemagick, on the other hand does seem to have exactly those functions (and a million others) and has PHP wrappers in PECL. Of course it might be dead slow too, but I'll give that a try. Thanks.

[NetWielder]

In your reference to 1 CPU working on the problem, I believe you are referring to CPU affinity-- which is a pthread setting that assigns threads to a specific CPU resource. Unfortunately, This setting may not be exposed in the pthread extension. However, if you don't mind consuming RAM you could always get a bit creative with multi-process execution where you could set the affinity you seek.

http://php.net/manual/en/ref.exec.php

ZeroMQ might be an option as well. You can still write the important part in C and execute asynchronously via a socket. Really depends on how badly you want to continue writing in PHP.

http://zguide.zeromq.org/page:all

Good luck on your endeavors!

[davekimble2]

I probably didn't explain myself properly. I have a quad-core CPU, but when one core is using ALL the available CPU cycles (on going round a loop 4 million times), and wasting no cycles on waiting for i/o to complete, I don't think it would improve overall speed to farm out a quarter of the task to each of the 4 cores. Although (average) CPU Usage looks like it is only 25%, it is in fact 100% of 1 core.

ImageMagick looks promising. I have installed it (and MagickWand) from Ubuntu repository, and downloaded the source for the stable PHP wrappers. Trying to compile it at the moment, but getting flooded with "warning: deprecated" messages before it stops. Warnings shouldn't stop it compiling, but maybe too many? Probably another version compatibility issue. Need to sleep.

[NetWielder]

On Ubuntu 14.04+ you can easily install all the binaries from their repository using apt-get. There isn't a real need for you to build from source for most modern Linux operating systems unless you have a custom built PHP extension(s), LAMP stack, etc...

Glad to hear ImageMagick appeals to you. Alternately, Perl + wxPerl + Gimp-Perl may offer the software components you need to accomplish what you are seeking much better than PHP. Perl has had bindings for Wx for about 2 decades and Gimp would provide everything you need in terms of image manipulations.

http://www.perl.org/ http://www.wxperl.it/ http://www.gimp.org/tutorials/Basic_Perl/

You might find this article informative too: https://software.intel.com/en-us/forums/topic/289210

[davekimble2]

OK, ImageMagick's MagickWandForPHP does the trick, and is nearly six times faster.

The only problem is $image->GetData() is returning a string not an array. Now the slow PHP loop can handle that, but MagickSetImagePixels() can't. If I convert the string to an array, it works, but that loop takes time. Likewise, $image->SetData() expects a string and won't accept an array, and another conversion loop takes time.

This could be fixed by getting GetData() and SetData() right (issue #75), but alternatively, if $pixel_values could be rendered directly to the screen then I could stop this messy process:

    $bitmap = new wxBitmap($filename, wxBITMAP_TYPE_JPEG);
    $image = $bitmap->ConvertToImage();
    $pixel_values = $image->GetData();
    $brightness = +10.0 ; // -100 < %age < +100
    adjust_brightness();
    $image->SetData($pixel_values);
    $bitmap = new wxBitmap($image, 24)
    render_bitmap($bitmap);

function adjust_brightness()
{   global $width, $height, $pixel_values, $brightness;
    $mw = NewMagickWand();
    MagickSetImagePixels($mw, 0, 0, $width, $height, "RGB", MW_CharPixel, $pixel_values);
    MagickModulateImage($mw, $brightness, 0.0, 0.0);
    $pixel_values = MagickGetImagePixels($mw, 0, 0, $width, $height, "RGB", MW_CharPixel);
}

There can't be much difference between ($width, $height, $pixel_values) and ($bitmap) can there?

[NetWielder]

Exploring your alternatives from a Wx perspective I believe is valuable and necessary however from a PHP perspective, you might consider using "generators" which are available in PHP 5.5. Generators provide a way to address resource consumption as well.

http://php.net/manual/en/language.generators.overview.php

For example, you might create a generator instead of performing a string-array conversion. Essentially, you are using generators to create an iterator for your string so that you may avoid the string-array conversion. Below is a snippet demonstrating how you might implement "yield" in a generator function:

function str_generator(&$string) {
        for ($position = 0; $position <= strlen($string); $position++) {
                yield substr($string, $position, 1);
        }
}

$data = str_repeat('ABCDEFGHIJKLMNOPQRSTUVWXYZ', 201649); # apprx 5 MB

foreach (str_generator($data) as $character) {
        echo $character;
}

You may find the ArrayAccess interface useful too. Also, PHP strings allow array access as well. For example here is how you remove a character in a string using array access:

$string = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ';
$string[3] = null; # removes the letter "D" from the string

http://php.net/manual/en/class.arrayaccess.php

[chrisnharvey]

On a side note, I'm really interested to see your image editor. Sounds like a pretty cool project.

[jgmdev]

There is str_split, but creating an array of a big magnitude will still be pretty slow and consume lots of memory, lets take for example a 1920x1080 image == 2073600 pixels, multiplied by 3 (RGB) and that would be 6220800 elements for an array. Due to dynamic nature of PHP variables arrays are linked lists of the zval struct which can represent any type of data. I'm not sure of the exact size of the zval struct (need to sizeof it) but lets say it is 256 bytes (should be near to real size) now multiply that by 6220800 and you would get 1518Megs of ram just to hold the array.

So even using str_split which is a native function to convert a string to an array the process would be slow and consume lots of memory.

[NetWielder]

I certainly empathize. It appears that moving string and array data to and from ImageMagick and Wx is insufficient. And it appears that it may be necessary because ImageMagick and Wx methods are in-congruent. You might consider how you will buffer both image manipulation and when using the device context in order to handle arbitrarily large files. However, using an approach that doesn't use str_split() or preg_split() to convert to an array is probably a good direction to explore.

[davekimble2]

There is wxNativePixelData - "A class template with ready to use implementations for getting direct and efficient access to wxBitmap's internal data and wxImage's internal data through a standard interface."

It doesn't say anything about setting, but if it allows that, then you could create a wxBitmap from $pixel_values, $width, $height just for the wxPaintDC process, and do all the other work with Magick functions.

Hmmm - no Set() methods, that's annoying.

[NetWielder]

I do not know all of the types of image manipulation operations you intend to implement. Have you considered "wrapping" the functionality offered by ImageMagick instead of manipulating pixels independently in PHP? Considering you can quickly modify an image on the command line, you could use exec() . The following command brightens the image dark-image.jpg by 200%.

mogrify -modulate 200,50 dark-image.jpg

[davekimble2]

Yes, first off I just wanted to convert the smallest part of the work over to Magick, to see how much faster it was. Since that is good, it would make sense to turn over as much as possible. The only thing it can't handle seems to be drawing on the screen, which has to be done with wxPaintDC::DrawBitmap($bitmap), so I need to be able to construct a wxBitmap from $pixel_values (array). Or find another solution.

[davekimble2]

$temp = "/home/dk/temp/temp.jpg";
MagickWriteImage($mw, $temp);
$bitmap = new wxBitmap($temp, wxBITMAP_TYPE_JPEG);`

only takes 0.057 seconds on SSD, so that's the way to do it. The file size of temp.jpg is the same as the original file, so I assume there is no JPEG compression loss.

[NetWielder]

Fantastic! I'm very glad you discovered the most efficient way to interact with ImageMagick. :-)

[NetWielder]

I ran across this YouTube video about Converseen, a graphical user interface for ImageMagick. You might find it interesting.

http://converseen.fasterland.net/ https://www.youtube.com/watch?v=PNQycX8o1eA

[davekimble2]

The Ubuntu repository has a GUI interface to ImageMagick API which is a photo editor called display.im6 when installed. The menu bar and file dialogs are very amateurish compared to wxWidgets - quite a poor demonstrator for the underlying power of the functions.
We will do better ;-p

One thing it has is the ability to edit an image that fills the screen. Why waste screen space on stuff you are not concentrating on?

A quick test, making one brightness change to 1920x1080 image and getting back to wxBitmap format, timed at 0.22 seconds compared to the PHP loop at "timed out" (> 60 seconds).

[davekimble2]

A beta version of PhotoEditor is available at http://www.davekimble.org.au/PhotoEditor

ImageMagick is a real memory hog - if you try out all the options on a 4.7 MB .jpg file, the resident memory goes as high as 1.4 GB, which is not released until the next image is opened.

It is also fairly slow on 20 megapixel images - the first (and slowest) thing you would want to do to this image is Crop by dragging out a rectangle with the mouse, after which it runs much better.

[jgmdev]

Cool! would be nice if you added it here http://wxphp.org/applications to showcase it as a wxphp application and maybe create a github repo for collaboration :)

[davekimble2]

Will do.

Don't know if it is related, but my web server just got hit with a LOT of nasty probes targetting /fckeditor, /wp-admin, /bbs, /forum, /includes/general.js, /api/ap_user.xml - holding up

[jgmdev]

I was just fixing ddos deflate to properly work under debian :D https://github.com/jgmdev/ddos-deflate it may serve you my modifications if you are running a debian server

[davekimble2]

https://github.com/davekimble2/PhotoEditor

Geez, that is one seriously horrible interface for uploading/syncing files ! Haven't they heard of http uploads, or a nice dual pane FTP like FireFTP ?

The local git seems to have put all of my home directory in its staging area and sorted them into some incomprehensible order with folders mixed up with files. Now I've got to find the horrible syntax to clean it all up. Please, don't anyone change anything just yet.

[jgmdev]

Nice!!! now you should extract the tar file and only leave the source files on the repository tree.

[NetWielder]

Take a look at ImageMagick documentation on how to address memory allocation for your image processing. You may need to re-compile with specific quantum settings.

http://imagemagick.sourceforge.net/http/www/FAQ.html#C9

Congratulations on PhotoEditor :-)

Not sure why Gimp isn't in this comparison. Hmm. But you might find this article interesting:

https://foliovision.com/2010/03/imagemagick-vs-gd

[davekimble2]

Done that, but I can't work out how to remove the .tar.gz , and some temp files in /temp (or how they got there in the first place).

[davekimble2]

ImageMagick has a "--contrast-stretch n%" option, but MagickWand doesn't give access to it. Consequently the only values for "Adjust > Contrast =" which make a difference are +1 and -1 (for more or less sharpening). MagickWand seems to hit on a good value all by itself, but it would be nice to have more control.

Given that the only features MagicWand has over wxWidgets are these adjustments, it might be better to write extensions for these functions after all. I did manage to get the brightness extension to compile and install, but it didn't work due to me not knowing enough about C++.

I don't know how people can say PHP and C++ are very similar - tight variable typing is an impediment to programming if loose typing is possible, which it is. Why would anyone bother? I think C/C++ is a step backwards as far as programming is concerned. PHP could be improved by tightening the rules on line indents, and dropping the insistence on those ghastly semi-colons.

[NetWielder]

Often it is difficult to understand why we have different programming languages. There are many reasons. You'll discover these reasons if you are passionate about software development. In short, C++ is a "compiled" language. PHP is an "interpretive" language that was written in C++ to provide the flexibility we all enjoy. PHP is also focused on web development not desktop development. There are many forums for such discussions. You might find the following article useful too:

http://www.cplusplus.com/info/description/

Consider learning how to write a PHP extension in C++, Python, Wx and C integrate well and so does Perl, Wx and C or why not explore GD or Gimp.

If that doesn't appeal to you here is a list of multitude of ImageMagic scripts that you might find useful or a point of reference. Certainly, these developers have invested much in the way of manipulating pixels. You might benefit from their considerable efforts.

http://www.fmwconcepts.com/imagemagick/histmatch/index.php