search

goldmann / docker-squash

Docker image squashing tool
MIT License
831 stars 108 forks source link

docker-squash: I/O operation on closed file #202

Closed jeshra007 closed 1 year ago

jeshra007 commented 3 years ago

Hi,

I tried to perform a docker-squash image file size of 51GB and it failed at the end with the below error.

2021-03-24 12:13:07,748 root INFO Squashing file '/mnt/store/tmp/old/7ff6fd71c8b2f7dfd7f4d7a34cf1db5fcfa4c377365dbaf1627b73e120802428/layer.tar'... 2021-03-24 12:14:29,251 root ERROR I/O operation on closed file 2021-03-24 12:14:29,251 root ERROR Execution failed, consult logs above. If you think this is our fault, please file an issue: https://github.com/goldmann/docker-squash/issues, thanks!

I have tried docker-squash for the 55GB file which worked fine but it failed for another one. Please suggest to me anything that needs to be checked for this?

Thanks, Rajesh

jeshra007 commented 3 years ago

Adding the debug mode of error log:

2021-03-25 04:16:41,771 root DEBUG Found a symbolic link '/usr/lib64/security/pam_unix_session.so' which is already squashed, skipping 2021-03-25 04:16:41,772 root DEBUG Skipping file '/usr/bin/lz4c' because it is already squashed 2021-03-25 04:16:41,774 root DEBUG Cleaning up /mnt/store/tmp/ temporary directory 2021-03-25 04:16:51,928 root ERROR I/O operation on closed file Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/docker_squash/cli.py", line 87, in run from_layer=args.from_layer, tag=args.tag, output_path=args.output_path, tmp_dir=args.tmp_dir, development=args.development, cleanup=args.cleanup).run() File "/usr/lib/python2.7/site-packages/docker_squash/squash.py", line 61, in run return self.squash(image) File "/usr/lib/python2.7/site-packages/docker_squash/squash.py", line 91, in squash new_image_id=image.squash() File "/usr/lib/python2.7/site-packages/docker_squash/image.py", line 74, in squash ret = self._squash() File "/usr/lib/python2.7/site-packages/docker_squash/v2_image.py", line 40, in _squash self.layer_paths_to_move) File "/usr/lib/python2.7/site-packages/docker_squash/image.py", line 840, in _squash_layers squashed_files, added_symlinks) File "/usr/lib/python2.7/site-packages/docker_squash/image.py", line 639, in _add_file squashed_tar.addfile(member, content) File "/usr/lib64/python2.7/tarfile.py", line 2022, in addfile copyfileobj(fileobj, self.fileobj, tarinfo.size) File "/usr/lib64/python2.7/tarfile.py", line 278, in copyfileobj buf = src.read(remainder) File "/usr/lib64/python2.7/tarfile.py", line 819, in read buf += self.fileobj.read(size - len(buf)) File "/usr/lib64/python2.7/tarfile.py", line 737, in read return self.readnormal(size) File "/usr/lib64/python2.7/tarfile.py", line 744, in readnormal self.fileobj.seek(self.offset + self.position) ValueError: I/O operation on closed file 2021-03-25 04:16:51,936 root ERROR Execution failed, consult logs above. If you think this is our fault, please file an issue: https://github.com/goldmann/docker-squash/issues, thanks!

jeshra007 commented 3 years ago

The I/O operation on closed file error appears when I do squash for all 51 layers. It worked with a workaround for squashing 30 layers. But still squash needs to work for all layers without this exception.

Thanks, Rajesh