ansible / ansible-modules-core

Ansible modules - these modules ship with ansible
1.3k stars 1.95k forks source link

unarchive module: improve error display for failing extract #74

Closed relaxnow closed 8 years ago

relaxnow commented 10 years ago
Issue Type:

Feature Idea

Ansible Version:

ansible 1.7.2

Environment:

Centos 6.5

Summary:

When the 'unarchive module' unpacks to a disk that is almost full it will display the "check_results" first, instead of the actual results for the unpacking "extract_results". With lots of output this may mean missing the original error.

Steps To Reproduce:

A disk that is almost full and contains a zip file to be unpacked:

[root@t01 tmp]# df -h | grep /tmp
/dev/mapper/VG0-tmp  1008M  914M   43M  96% /tmp
[root@t01 tmp]$ ls -lah OpenConext-engineblock-*
-rw-r--r-- 1 boy users 34M Sep 29 11:42 OpenConext-engineblock-4.2.7.tar.gz

Unpacking with latest Ansible 1.7.2:

[root@t01 tmp]# ansible -i localhost, -c local -m unarchive -a "src=/tmp/OpenConext-engineblock-4.2.7.tar.gz dest=/tmp/ copy=no" -v localhost
localhost | FAILED >> {
    "check_results": {
        "cmd": "/bin/tar -v -C \"/tmp/\" --diff -zf \"/tmp/OpenConext-engineblock-4.2.7.tar.gz\"", 
        "err": "/bin/tar: OpenConext-engineblock-4.2.7: Warning: Cannot stat: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/database: Warning: Cannot stat: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/database/patch: Warning: Cannot stat: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/database/patch/patch-0011.sql: Warning: Cannot stat: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/database/patch/patch-0016.php: Warning: Cannot stat: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/database/patch/patch-0006.sql: Warning: Cannot stat: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/database/patch/patch-0007.sql: Warning: Cannot stat: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/database/patch/patch-0017.sql: Warning: Cannot stat: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/database/patch/patch-0001.php: Warning: Cannot stat: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/database/patch/patch-0008.php: Warning: Cannot stat: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/database/patch/patch-0003.sql: Warning: Cannot stat: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/database/patch/patch-0014.sql: Warning: Cannot stat: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/database/patch/patch-0009.sql: Warning: Cannot stat: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/database/patch/patch-0015.sql: Warning: Cannot stat: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/database/patch/patch-0002.sql: Warning: Cannot stat: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/database/patch/patch-0012.sql: Warning: Cannot stat: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/database/patch/patch-0010.php: Warning: Cannot stat: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/database/patch/patch-0004.sql: Warning: Cannot stat: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/database/patch/patch-0018.sql: Warning: Cannot stat: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/database/patch/patch-0005.sql: Warning: Cannot stat: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/database/patch/patch-0013.sql: Warning: Cannot stat: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/database/patch-template.php: Warning: Cannot stat: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/database/patch-template.sql: Warning: Cannot stat: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/NOTICE.txt: Warning: Cannot stat: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/bin: Warning: Cannot stat: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/bin/makeRelease.sh: Warning: Cannot stat: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/bin/ldaptest.php: Warning: Cannot stat: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/bin/formatLog.sh: Warning: Cannot stat: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/bin/composer: Warning: Cannot stat: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/bin/composer/simplesamlphp-post-install.sh: Warning: Cannot stat: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/bin/composer/zend-framework-post-install.sh: Warning: Cannot stat: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/bin/composer/dbpatch-post-install.sh: Warning: Cannot stat:
... SNIP PAGES AND PAGES OF OUTPUT ...
/www/authentication/css/ext/jqueryui/1.8.16/custom-theme/images/ui-icons_2e83ff_256x240.png\nOpenConext-engineblock-4.2.7/www/authentication/css/ext/jqueryui/1.8.16/custom-theme/images/ui-bg_glass_75_dadada_1x400.png\nOpenConext-engineblock-4.2.7/www/authentication/css/ext/jqueryui/1.8.16/custom-theme/images/ui-icons_222222_256x240.png\nOpenConext-engineblock-4.2.7/www/authentication/css/ext/jqueryui/1.8.16/custom-theme/images/ui-icons_454545_256x240.png\nOpenConext-engineblock-4.2.7/www/authentication/css/ext/jqueryui/1.8.16/custom-theme/images/ui-bg_glass_75_e6e6e6_1x400.png\nOpenConext-engineblock-4.2.7/www/authentication/css/ext/jqueryui/1.8.16/custom-theme/images/ui-bg_glass_55_fbf9ee_1x400.png\nOpenConext-engineblock-4.2.7/www/authentication/css/ext/jqueryui/1.8.16/custom-theme/images/ui-bg_flat_75_ffffff_40x100.png\nOpenConext-engineblock-4.2.7/www/authentication/css/ext/jqueryui/1.8.16/custom-theme/images/ui-bg_highlight-soft_75_cccccc_1x100.png\nOpenConext-engineblock-4.2.7/www/authentication/css/ext/jqueryui/1.8.16/custom-theme/images/ui-bg_glass_95_fef1ec_1x400.png\nOpenConext-engineblock-4.2.7/www/authentication/css/ext/jqueryui/1.8.16/custom-theme/images/ui-icons_cd0a0a_256x240.png\nOpenConext-engineblock-4.2.7/www/authentication/css/error.css\nOpenConext-engineblock-4.2.7/www/authentication/css/wide/\nOpenConext-engineblock-4.2.7/www/authentication/css/wide/screen_ie.css\nOpenConext-engineblock-4.2.7/www/authentication/css/wide/screen.css\nOpenConext-engineblock-4.2.7/www/authentication/index.php\nOpenConext-engineblock-4.2.7/www/authentication/generated/\nOpenConext-engineblock-4.2.7/www/authentication/generated/js/\nOpenConext-engineblock-4.2.7/www/authentication/generated/js/1411979426/\nOpenConext-engineblock-4.2.7/www/authentication/generated/js/1411979426/js.min.js\nOpenConext-engineblock-4.2.7/www/authentication/generated/css/\nOpenConext-engineblock-4.2.7/www/authentication/generated/css/1411979426/\nOpenConext-engineblock-4.2.7/www/authentication/generated/css/1411979426/css.min.css\nOpenConext-engineblock-4.2.7/www/authentication/robots.txt\nOpenConext-engineblock-4.2.7/www/profile/\nOpenConext-engineblock-4.2.7/www/profile/media\nOpenConext-engineblock-4.2.7/www/profile/favicon.ico\nOpenConext-engineblock-4.2.7/www/profile/javascript\nOpenConext-engineblock-4.2.7/www/profile/css\nOpenConext-engineblock-4.2.7/www/profile/index.php\nOpenConext-engineblock-4.2.7/www/profile/robots.txt\n", 
        "rc": 1, 
        "unarchived": false
    }, 
    "dest": "/tmp/", 
    "extract_results": {
        "cmd": "/bin/tar -xzf \"/tmp/OpenConext-engineblock-4.2.7.tar.gz\"", 
        "err": "/bin/tar: OpenConext-engineblock-4.2.7/vendor/zendframework/zendframework1/tests/Zend/OpenId/ProviderTest.php: Wrote only 6144 of 10240 bytes\n/bin/tar: OpenConext-engineblock-4.2.7/vendor/zendframework/zendframework1/tests/Zend/OpenId/Consumer: Cannot mkdir: No space left on device\n/bin/tar: OpenConext-engineblock-4.2.7/vendor/zendframework/zendframework1/tests/Zend/OpenId/Consumer/Storage: Cannot mkdir: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/vendor/zendframework/zendframework1/tests/Zend/OpenId/Consumer/Storage/FileTest.php: Cannot open: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/vendor/zendframework/zendframework1/tests/Zend/OpenId/Extension: Cannot mkdir: No space left on device\n/bin/tar: OpenConext-engineblock-4.2.7/vendor/zendframework/zendframework1/tests/Zend/OpenId/Extension/SregTest.php: Cannot open: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/vendor/zendframework/zendframework1/tests/Zend/OpenId/Provider: Cannot mkdir: No space left on device\n/bin/tar: OpenConext-engineblock-4.2.7/vendor/zendframework/zendframework1/tests/Zend/OpenId/Provider/Storage: Cannot mkdir: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/vendor/zendframework/zendframework1/tests/Zend/OpenId/Provider/Storage/FileTest.php: Cannot open: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/vendor/zendframework/zendframework1/tests/Zend/OpenId/Provider/User: Cannot mkdir: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/vendor/zendframework/zendframework1/tests/Zend/OpenId/Provider/User/SessionTest.php: Cannot open: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/vendor/zendframework/zendframework1/tests/Zend/OpenId/AllTests.php: Cannot write: No space left on device\n/bin/tar: OpenConext-engineblock-4.2.7/vendor/zendframework/zendframework1/tests/Zend/File: Cannot mkdir: No space left on device\n/bin/tar: OpenConext-engineblock-4.2.7/vendor/zendframework/zendframework1/tests/Zend/File/TestAsset: Cannot mkdir: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/vendor/zendframework/zendframework1/tests/Zend/File/TestAsset/MultipleClassesInMultipleNamespaces.php: Cannot open: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/vendor/zendframework/zendframework1/tests/Zend/File/TestAsset/LocatorShouldFindThis.php: Cannot open: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/vendor/zendframework/zendframework1/tests/Zend/File/_files: Cannot mkdir: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/vendor/zendframework/zendframework1/tests/Zend/File/_files/locator-should-skip-this.php: Cannot open: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/vendor/zendframework/zendframework1/tests/Zend/File/_files/locator-should-skip-this.xml: Cannot open: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/vendor/zendframework/zendframework1/tests/Zend/File/ClassFileLocatorTest.php: Cannot open: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/vendor/zendframework/zendframework1/tests/Zend/File/Transfer: Cannot mkdir: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/vendor/zendframework/zendframework1/tests/Zend/File/Transfer/Adapter: Cannot mkdir: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/vendor/zendframework/zendframework1/tests/Zend/File/Transfer/Adapter/_files: Cannot mkdir: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/vendor/zendframework/zendframework1/tests/Zend/File/Transfer/Adapter/_files/file2
.... SNIP PAGES AND PAGES OF OUTPUT ...
ss: Cannot open: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/www/authentication/css/wide: Cannot mkdir: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/www/authentication/css/wide/screen_ie.css: Cannot open: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/www/authentication/css/wide/screen.css: Cannot open: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/www/authentication/index.php: Cannot open: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/www/authentication/generated: Cannot mkdir: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/www/authentication/generated/js: Cannot mkdir: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/www/authentication/generated/js/1411979426: Cannot mkdir: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/www/authentication/generated/js/1411979426/js.min.js: Cannot open: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/www/authentication/generated/css: Cannot mkdir: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/www/authentication/generated/css/1411979426: Cannot mkdir: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/www/authentication/generated/css/1411979426/css.min.css: Cannot open: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/www/authentication/robots.txt: Cannot open: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/www/profile: Cannot mkdir: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/www/profile/media: Cannot open: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/www/profile/favicon.ico: Cannot open: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/www/profile/javascript: Cannot open: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/www/profile/css: Cannot open: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/www/profile/index.php: Cannot open: No such file or directory\n/bin/tar: OpenConext-engineblock-4.2.7/www/profile/robots.txt: Cannot open: No such file or directory\n/bin/tar: Exiting with failure status due to previous errors\n", 
        "out": "", 
        "rc": 2
    }, 
    "failed": true, 
    "gid": 0, 
    "group": "root", 
    "handler": "TgzFile", 
    "mode": "01777", 
    "msg": "failed to unpack /tmp/OpenConext-engineblock-4.2.7.tar.gz to /tmp/", 
    "owner": "root", 
    "size": 270336, 
    "src": "/tmp/OpenConext-engineblock-4.2.7.tar.gz", 
    "state": "directory", 
    "uid": 0
}
Expected Results:

I expect to see the failing 'extract' first, the diff second.

Actual Results:

Instead I saw the diff first which gave confusing output.

relaxnow commented 10 years ago

Also my Terminal is set to wrap lines by default, so it was really hard to see that there was a second part to all the output at all.

razvanphp commented 9 years ago

+1

The strange thing is that in my case, I have enough free space, so I don't understand why the error is there..

joaocc commented 9 years ago

I am also getting the same error with plenty of space left. In my case I am running a remote that is a hosted server, so it may be due to some lockdown.

However, all the files are extracted correctly. It's simply the annoyance and loosing the notion of "real" problems that make this a problem to me.

2015-07-27 02:40:07,068 p=26755 u=vagrant | changed: [hostedsite1] => {"changed": true, "check_results": {"cmd": "/bin/gtar -C \"/home/hostedsite1/__hsz/src.nodejs/nodejs-v0.12.7.tmp\" --diff -zf \"/home/hsz000manudemo1/__hsz/src.nodejs/node-v0.12.7.tar.gz\"", "err": "/bin/gtar: node-v0.12.7: Warning: Cannot stat: No such file or directory\n/bin/gtar: node-v0.12.7/benchmark: Warning: Cannot stat: No such file or directory\n/bin/gtar: node-v0.12.7/benchmark/idle_clients.js: Warning: Cannot stat: No such file or directory\n/bin/gtar: node-v0.12.7/benchmark/http_simple.rb: Warning: Cannot stat: No such file or directory\n/bin/gtar: node-v0.12.7/benchmark/fs-write-stream-throughput.js: Warning: Cannot stat: No such file or directory\n/bin/gtar: node-v0.12.7/benchmark/http_server_lag.js: Warning: Cannot stat: No such file or directory\n/bin/gtar: node-v0.12.7/benchmark/README.md: Warning: Cannot stat: No such file or directory\n/bin/gtar: node-v0.12.7/benchmark/misc: Warning: Cannot stat: No such file or directory\n/bin/gtar: node-v0.12.7/benchmark/misc/function_call: Warning: Cannot stat: No such file or directory\n/bin/gtar: node-v0.12.7/benchmark/misc/function_call/index.js: Warning: Cannot stat: No such file or directory\n/bin/gtar: node-v0.12.7/benchmark/misc/function_call/binding.gyp: Warning: Cannot stat: No such file or directory\n/bin/gtar: node-v0.12.7/benchmark/misc/function_call/binding.cc: Warning: Cannot stat: No such file or directory\n/bin/gtar: node-v0.12.7/benchmark/misc/function_call/.gitignore: Warning: Cannot stat: No such file or directory\n/bin/gtar: node-v0.12.7/benchmark/misc/function_call/Makefile: Warning: Cannot stat: No such file or directory\n/bin/gtar: node-v0.12.7/benchmark/misc/spawn-echo.js: Warning: Cannot stat: No such file or directory\n/bin/gtar: node-v0.12.7/benchmark/misc/startup.js: Warning: Cannot stat: No such file or directory\n/bin/gtar: node-v0.12.7/benchmark/misc/string-creation.js: Warning: Cannot stat: No such file or directory\n/bin/gtar: node-v0.12.7/benchmark/misc/next-tick-breadth.js: Warning: Cannot stat: No such file or directory\n/bin/gtar: node-v0.12.7/benchmark/misc/next-tick-depth.js: Warning: Cannot stat: No such file or directory\n/bin/gtar: node-v0.12.7/benchmark/misc/url.js: Warning: Cannot stat: No such file or directory\n/bin/gtar: node-v0.12.7/benchmark/misc/v8-bench.js: Warning: Cannot stat: No such file or directory\n/bin/gtar: node-v0.12.7/benchmark/misc/timers.js: Warning: Cannot stat: No such file or directory\n/bin/gtar: node-v0.12.7/benchmark/misc/child-process-read.js: Warning: Cannot stat: No such file or directory\n/bin/gtar: node-v0.12.7/benchmark/misc/module-loader.js: Warning: Cannot stat: No such file or directory\n/bin/gtar: node-v0.12.7/benchmark/common.js: Warning: Cannot stat: No such file or directory\n/bin/gtar: node-v0.12.7/benchmark/fs: Warning: Cannot stat: No such file or directory\n/bin/gtar: node-v0.12.7/benchmark/fs/readfile.js: Warning: Cannot stat: No such file or directory\n/bin/gtar: node-v0.12.7/benchmar

DenisGorbachev commented 9 years ago

+1 plenty of free space on a clean DigitalOcean droplet, Ubuntu 14.04

aussielunix commented 9 years ago

Seeing this issue too with ansible 1.9.3 on Ubuntu Trusty amd64

It is possible that this is not directly an Ansible issue but more an issue with /bin/tar

We are using the following task:

- name: Unpack Mule download
  unarchive:
    src="{{ mule_download_dir }}/{{ mule_package }}"
    dest="{{ mule_installation_dir }}"
    force=no
    copy=false
    owner={{ mule_os_user }}
    group={{ mule_os_group }}
    mode=0755
  tags: unpack

This task results in the following /bin/tar command but fails.

/bin/tar -C "share/" --diff -zf mule-standalone-3.6.1.tar.gz
foo@bar:/tmp/mp$ /bin/tar -C "share/" --diff -zf mule-standalone-3.6.1.tar.gz
/bin/tar: mule-standalone-3.6.1/lib/boot/commons-cli-1.2.jar: Warning: Cannot stat: No such file or directory
/bin/tar: mule-standalone-3.6.1/lib/boot/disruptor-3.3.0.jar: Warning: Cannot stat: No such file or directory
/bin/tar: mule-standalone-3.6.1/lib/boot/slf4j-api-1.7.7.jar: Warning: Cannot stat: No such file or directory
/bin/tar: mule-standalone-3.6.1/lib/boot/log4j-api-2.1.jar: Warning: Cannot stat: No such file or directory
/bin/tar: mule-standalone-3.6.1/lib/boot/log4j-core-2.1.jar: Warning: Cannot stat: No such file or directory
/bin/tar: mule-standalone-3.6.1/lib/boot/log4j-slf4j-impl-2.1.jar: Warning: Cannot stat: No such file or directory
...
...
...

Looking into this I was able to get it to work with the following:

/bin/tar -C "share/" --extract -zf mule-standalone-3.6.1.tar.gz
dagwieers commented 8 years ago

I have fixed the confusing check_results output issue in #3307 as part of commit 8a1b0071bc845018fec97c7bb7ba474c299c34d2.

If you are using the unarchive module, I welcome feedback and wider test reviews before this hits a new release. Testing the new unarchive is easy:

mkdir -p ./library/
wget -O ./library/unarchive.py https://raw.githubusercontent.com/dagwieers/ansible-modules-core/idempotent-unarchive/files/unarchive.py
dagwieers commented 8 years ago

BTW The reason you see those confusing Warning: Cannot stat: No such file or directory messages is because the unarchive module (unfortunately) uses gtar --diff to establish whether there is a change between what's in the archive and what is on disk. However, if there is nothing on disk (yet) the output will report that for each individual file with the above statement.

Very confusing if you don't know exactly what is going on. (Whether it means the extract fails, was successful, or there is an expected change that doesn't require extraction).

The patch I proposed will remove the check_result from the output. Personally, I would only show that information in debug-mode (or very verbose mode). But that's v2 specific AFAIK.