search

ucphhpc / migrid-sync

MiGrid workspace where master branch is kept strictly in sync with SF upstream svn repo. Any development or experiments should use a branch. You probably want to fork your own clone or work e.g. on the edge branch if you wish to contribute.
GNU General Public License v2.0
3 stars 4 forks source link

unpack zipped files fails #16

Closed ARiis-63 closed 1 year ago

ARiis-63 commented 1 year ago

Unpack zipped files does fail with utf-8 error. Like this from mig.log ...s@au.dk/temp/Stempel-ur-2022.xls, error: 'utf-8' codec can't decode byte 0xd0 in position 0: invalid continuation byte and other 'utf-8' errors {"deploy_version": "v1.4", "MIG_GIT_REV": "6c71d37679f6750faa288365e68f4bc055e953fc", "MIG_SVN_REV": "0000"}

jonasbardino commented 1 year ago

In my initial tests on bench.erda I can't reproduce the error, but perhaps it requires specific non-utf8 content or filenames inside the zip. Can you please provide a simple example zip file I can test with?

jonasbardino commented 1 year ago

Thanks for the zip example with non-utf8 content on email @ARiis-63 :)

The problem was another python3 string encoding issue because binary zip file contents (bytes) were attempted written as native strings and thus hit string conversion errors for such non-utf8 content. It should be fixed since experimental revision 6b5498e27ff46f4241775510ce47dac52f8f656c by explicitly opening the destination files in binary mode. Tested on bench.erda with rocky8 and python3 only.