Truncated USS file downloaded

zowe / zowe-cli

Zowe CLI

Eclipse Public License 2.0

108 stars 85 forks source link

Truncated USS file downloaded #1170

Open plavjanik opened 2 years ago

plavjanik commented 2 years ago

I am downloading a binary file using the following command:

zowe zos-files dl uf /a/mydir/sca.jar -f sca.jar --binary

The download seems to be successful:

USS file downloaded successfully.
Destination: sca.jar

But the file is corrupted. It has been created using jar tool:

cd /usr/lpp/zosmf/installableApps
jar cvf /a/plape03/sca.jar izuCA.ear

But it cannot be opened using jar nor unzip on my computer.

When I have used zftp plugin to download it, I am getting a different size of the file:

-rw-r--r--   1 myuser  staff   9282652 Oct 19 13:24 sca.jar
-rw-r--r--   1 myuser  staff   9291928 Oct 19 11:59 sca3.jar

and the FTP transferred file is ok.

They seem to differ only at the end:

t1m0thyj commented 2 years ago

I've been able to reproduce this issue when downloading binary data sets or USS files thru z/OSMF.

The issue is not specific to Zowe CLI as cURL also truncates the end of the file. It only happens when the "Accept-Encoding: gzip" header is passed and the REST response is compressed.

We plan to file an issue with z/OSMF about this, and in the interim to disable gzip compression in Zowe CLI for binary downloads.

plavjanik commented 2 years ago

Thank you for working on a workaround and reporting it to IBM.

Would not it be possible to activate the workaround based on the z/OSMF version in case that only some versions have this issue?

t1m0thyj commented 1 year ago

Sorry for not posting an update on this issue in so long. We have an ongoing case open with the z/OSMF team about this, and they are looking into it.

We have reproduced the issue with the following steps:

Create a 4KB file with random contents: dd if=/dev/urandom of=testfile bs=1024 count=4
Upload the file to a sequential data set: zowe files upload ftds testfile <dsname> --binary
Download the file with cURL
- Uncompressed: curl -H "X-IBM-Data-Type: binary" "<zosmfUrl>/zosmf/restfiles/ds/<dsname>"
- Compressed: pass the "Accept-Encoding: gzip" header and --compressed option
Notice that the downloaded file is truncated only when gzip compression is enabled

I've added a "Blocked" label for now. Once the issue with z/OSMF is resolved, we can reenable gzip compression in Zowe CLI for binary downloads which should allow us to close this issue.

t1m0thyj commented 1 year ago

Would not it be possible to activate the workaround based on the z/OSMF version in case that only some versions have this issue?

I don't believe this is a version-specific bug. Gzip compression support is a fairly recent addition to z/OSMF so I think this bug is present in all versions of z/OSMF that support gzip compression.

github-actions[bot] commented 1 year ago

Thank you for creating a bug report. If you haven't already, please ensure you have provided steps to reproduce it and as much context as possible.