Add the possibility to disable Image Sizes Remove/Restore

[ghost]

cc @danyj https://github.com/ThemeFuse/Unyson/issues/1818#issuecomment-251994766

[danyj]

yes , this would be useful to eliminate the long resizing images process

[ghost]

What about this?

On sizes restore start do:

1. Get first attachment file name hello.jpg, find its sizes by file name hello-...x....jpg and remember the list
2. Regenerate sizes for that first image and load again the files list
3. If the initial file list is identical to the one after regenerate, this mean that the zip contains all image sizes and there is no point to regenerate all images

[danyj]

sounds doable, but what if user has switched themes in lets say last 3 months , and theme he uses now has different image sizes , the check would not be reliable.

[ghost]

Do you mean a demo install designed for one theme will be used for completely different theme?

[danyj]

no I dont mean that ,

you make theme , I make theme

both based on Unyson

user uses yours , than uses mine ,

than decides to make demo

[danyj]

here we have to think that demo functionality is available to all of us, users and developers

[danyj]

in ref to that unzip function , I just checked our unzip code and we ended up uzing this lib http://www.phpclasses.org/browse/file/10410.html in combo with dZip

instead of that function. It was more reliable and gave us option to restrict up/down per user. but this one is also widely used http://www.phpconcept.net/pclzip/ and does not require any additional php.ini settings

[ghost]

http://www.phpconcept.net/pclzip/ Contains too much "noise", defined global constants. http://www.phpclasses.org/browse/file/10410.html is just a class, I will inspect it and figure out if it can be used to unzip in steps.

[ghost]

I am concerned about compatibility with different zip archives created with different zip clients. For example here is a comment from that file:

    Function _loadFileListByEOF(&$fh, $stopOnFile=false){
        // Check if there's a valid Central Dir signature.
        // Let's consider a file comment smaller than 1024 characters...
        // Actually, it length can be 65536.. But we're not going to support it.

        for($x = 0; $x < 1024; $x++){

I hope everything will be fine.

[danyj]

this guy did it with zipacrchive , http://stackoverflow.com/questions/3263129/unzipping-larger-files-with-php ,

but see this comment http://stackoverflow.com/a/3263293/594423

looks like this does it in chunks already

$buf = zip_entry_read($zip_entry, zip_entry_filesize($zip_entry));
 fwrite($fp,"$buf");

[ghost]

I didn't know about zip_*() functions. I just did a test and it woks and the code is simple. Thank you very much! I will try to integrate them in unzip process.

[danyj]

before you dig in check this please , http://prntscr.com/cqzq6z

I am not sure if it requires PECL zip , or is also available for it . If it does requires it note that many servers dont have it by default

[ghost]

ZipArchive also has this, of course we will test it on some hosting servers, and you will test it when we will create the approval.

[ghost]

I remember that previously I made zip and unzip in steps, but when I added/extracted one file from zip, the entire zip was loaded and processed in background (in FTP I saw a big temp file) and the process was way slower that doing everything at once. I hope with zip_*() functions that will not happen.

[ghost]

I edited a Content Backup zip and added 45k files in it (Unyson svn respository). The archive is ~215Mb, my php memory limit is 40Mb. The backup restore worked fine, the unzip process was made in 2 requests/steps (also I decreased manually unzip timeout and it was made in multiple steps).

error_log(memory_get_peak_usage().':'.memory_get_peak_usage(true));

showed 18791320:2097152 which is 17.9MB:2MB. If it will work on slow servers it will be awesome.

[ghost]

https://github.com/ThemeFuse/Unyson-Backups-Extension/commit/59e33f4edb591a53f6244cb0479903f2bd8534c2

Please test

[danyj]

on it

[danyj]

Im getting this

[07-Oct-2016 17:42:55 UTC] PHP Warning:  copy(T:/wamp64/www/Development/DemosTest/wordpress/wp-content/themes/theme_name/demo-content/clean\f\plugins\unyson\framework\extensions\learning\extensions\learning-quiz\includes\option-types\quiz-builder\items\gap-fill\class-fw-option-type-quiz-builder-item-gap-fill.php): failed to open stream: Invalid argument in T:\wamp64\www\Development\DemosTest\wordpress\wp-content\plugins\unyson\framework\extensions\backups\helpers.php on line 147

using local theme install , demo is located in

theme/demo-content

[ghost]

I think it's Windows path length limit

> 'T:/wamp64/www/Development/DemosTest/wordpress/wp-content/themes/theme_name/demo-content/clean/f/plugins/unyson/framework/extensions/learning/extensions/learning-quiz/includes/option-types/quiz-builder/items/gap-fill/class-fw-option-type-quiz-builder-item-gap-fill.php'.length
267

[danyj]

me to , doing new backup ,

[danyj]

well seems like same process if done from demo-content folder , will do next test on demo download

[danyj]

this will take looong http://prntscr.com/cr1y7h

[danyj]

tell me , is the process interrupted if I get out of demo install page ?

[ghost]

1. This change https://github.com/ThemeFuse/Unyson-Backups-Extension/commit/59e33f4edb591a53f6244cb0479903f2bd8534c2 contains only unzip improvements
2. Image Sizes process wasn't touched
3. Demo Install has nothing to do with plugins and themes

[ghost]

Yes

[ghost]

Your theme should require the plugins with TGM

[ghost]

That is a another issue.

[danyj]

k deleted, will test this more

[ghost]

I haven't noticed that the download task had it's own unzip script, now I changed it to use the partial unzip https://github.com/ThemeFuse/Unyson-Backups-Extension/commit/19e53488c6f9c6ccea3b03033bfec25371495d62

[ghost]

Code snippets soon...

[ghost]

This possibility was already there https://github.com/ThemeFuse/Unyson-Backups-Extension/issues/15

How to disable image sizes remove on Content Backup

Important! Use this code only while developing, to export a demo install with image sizes included. Do not include this code in theme in production, because all Content Backups created by user will be with image sizes.

1. Add the filter in {theme}/functions.php

   add_filter(
      'fw:ext:backups:add-backup-task:image-sizes-remove',
      '_filter_theme_fw_ext_backups_disable_image_sizes_remove',
      10, 2
   );
   function _filter_theme_fw_ext_backups_disable_image_sizes_remove(
      $do, FW_Ext_Backups_Task_Collection $collection
   ) {
      $do = false;
      return $do;
   }

2. Do Content Backup
3. Remove the filter from 1.

How to disable image sizes restore for specific demo

add_filter(
    'fw:ext:backups:add-restore-task:image-sizes-restore', 
    '_filter_theme_fw_backups_disable_image_sizes_restore'.
    10, 2
);
function _filter_theme_fw_backups_disable_image_sizes_restore(
    $do, FW_Ext_Backups_Task_Collection $collection
) {
    if (
        $collection->get_id() === 'demo-content-install'
        &&
        ($task = $collection->get_task('demo:demo-download'))
        &&
        ($task_args = $task->get_args())
        &&
        isset($task_args['demo_id'])
        &&
        in_array($task_args['demo_id'], array('DEMO_ID_1', 'DEMO_ID_2'))
    ) {
        $do = false;
    }

    return $do;
}

[danyj]

Im lost.

Use this code only while developing, to export a demo install with image sizes included

I thought we will have off switch for image sizes restore. What is image sizes remove?

Sorry but I dont understand the usage at all.

[danyj]

I add filters Export demo Remove filters from theme

Demo installs without image sizes restore

Right?

[ghost]

I add filters Export demo Remove filters from theme

Yes, this is for first filter.

Demo installs without image sizes restore

This is for the second filter. (note the DEMO_ID_X in array)

[danyj]

ok but What is image sizes remove? http://prntscr.com/csnd9b

[ghost]

How content backup works:

1. db is exported to a txt file
2. uploads/ is copied
3. image sizes are deleted from the copied uploads/ (only original images are left)
4. everything is ziped

[danyj]

@moldcraft ,

does the second filter , fw:ext:backups:add-restore-task:image-sizes-restore

also need to be removed? Not right?