search

fingltd / 4mc

4mc - splittable lz4 and zstd in hadoop/spark/flink
Other
108 stars 36 forks source link

Can not specify output file/folder name using command line tool (darwin) #19

Closed baaa closed 6 years ago

baaa commented 7 years ago

These commands are executed back to back on the same input file, it only works when I do not specify any output file at all

UK-HAV-M4NAG8WN:zstd-fast yucelm01$ 4mc -d split_0000.4mz /tmp/delete/split_0000.json
Unrecognized header : not a 4mc file
UK-HAV-M4NAG8WN:zstd-fast yucelm01$ 4mc -d split_0000.4mz /tmp/delete/split_0000
Unrecognized header : not a 4mc file
UK-HAV-M4NAG8WN:zstd-fast yucelm01$ 4mc -d split_0000.4mz /tmp/delete/split_0000.4mc
Unrecognized header : not a 4mc file
UK-HAV-M4NAG8WN:zstd-fast yucelm01$ 4mc -d split_0000.4mz /tmp/delete
Warning : /tmp/delete already exists
Overwrite ? (Y/N) : y
Cannot open output file: /tmp/delete
UK-HAV-M4NAG8WN:zstd-fast yucelm01$ 4mc -d split_0000.4mz
Decoding file split_0000 
Compression: ZSTD
Successfully decoded 32517745 bytes                                            
UK-HAV-M4NAG8WN:zstd-fast yucelm01$

edit: also it fails with the same error if I have a dot(.) in the filename anywhere like: 4mc -d split_0000.json.4mz

carlomedas commented 7 years ago

what if you do -z -d ?

baaa commented 7 years ago

both 4mc -z -d split_0000.4mz /tmp/delete/split_0000.json and 4mc -z -d split_0000.json.4mz /tmp/delete/split_0000.json works.

it even works without any extension at all (as it should) I guess the only trick is gotto know the compression type and provide -z flag, cheeers. I guess I will need to read the header and decide.

carlomedas commented 7 years ago

yes, it was implemented like that atm. it's not reading the header to understand the type. feel free to implement it and pull request, thanks in advance!