JuliaIO / Tar.jl

TAR files: create, list, extract them in pure Julia
MIT License
79 stars 19 forks source link

Extracting error #94

Closed henry2004y closed 3 years ago

henry2004y commented 3 years ago

Hi,

I found this package when I was having issue to do a cross-platform extracting for my package test data. I assume this package should be able to untar a file compressed by gzip. I have a sample data file and was trying to unzip it with

using Tar
Tar.extract("bulk_vlsv.tar")

but I got

ERROR: invalid octal digit: '\n'
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] read_header_int(::SubArray{UInt8,1,Array{UInt8,1},Tuple{UnitRange{Int64}},true}, ::Int64, ::Int64) at /home/hongyang/.julia/packages/Tar/DQaSa/src/extract.jl:534
 [3] read_standard_header(::IOStream; buf::Array{UInt8,1}, tee::Base.DevNull) at /home/hongyang/.julia/packages/Tar/DQaSa/src/extract.jl:489
 [4] #read_header#47 at /home/hongyang/.julia/packages/Tar/DQaSa/src/extract.jl:370 [inlined]
 [5] read_tarball(::Tar.var"#25#27"{Array{UInt8,1},Bool,IOStream,String}, ::Tar.var"#1#2", ::IOStream; buf::Array{UInt8,1}, skeleton::Base.DevNull) at /home/hongyang/.julia/packages/Tar/DQaSa/src/extract.jl:331
 [6] extract_tarball(::Function, ::IOStream, ::String; buf::Array{UInt8,1}, skeleton::Base.DevNull, copy_symlinks::Bool) at /home/hongyang/.julia/packages/Tar/DQaSa/src/extract.jl:57
 [7] (::Tar.var"#76#79"{String,IOStream,Tar.var"#1#2"})(::Base.DevNull) at /home/hongyang/.julia/packages/Tar/DQaSa/src/Tar.jl:201
 [8] arg_write(::Tar.var"#76#79"{String,IOStream,Tar.var"#1#2"}, ::Base.DevNull) at /home/hongyang/.julia/packages/ArgTools/4vlk9/src/ArgTools.jl:94
 [9] (::Tar.var"#75#78"{IOStream,Tar.var"#1#2"})(::String) at /home/hongyang/.julia/packages/Tar/DQaSa/src/Tar.jl:200
 [10] arg_mkdir(::Tar.var"#75#78"{IOStream,Tar.var"#1#2"}, ::Nothing) at /home/hongyang/.julia/packages/ArgTools/4vlk9/src/ArgTools.jl:145
 [11] #74 at /home/hongyang/.julia/packages/Tar/DQaSa/src/Tar.jl:196 [inlined]
 [12] open(::Tar.var"#74#77"{Tar.var"#1#2",Nothing}, ::String; kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at ./io.jl:325
 [13] open(::Function, ::String) at ./io.jl:323
 [14] arg_read at /home/hongyang/.julia/packages/ArgTools/4vlk9/src/ArgTools.jl:42 [inlined]
 [15] extract(::Function, ::String, ::Nothing; skeleton::Nothing, copy_symlinks::Nothing) at /home/hongyang/.julia/packages/Tar/DQaSa/src/Tar.jl:195
 [16] #extract#80 at /home/hongyang/.julia/packages/Tar/DQaSa/src/Tar.jl:217 [inlined]
 [17] extract at /home/hongyang/.julia/packages/Tar/DQaSa/src/Tar.jl:217 [inlined] (repeats 2 times)
 [18] top-level scope at REPL[3]:1

Any idea what's going on?

Attached test file: test file

giordano commented 3 years ago

I assume this package should be able to untar a file compressed by gzip

Wrong assumption

It is typical to compress tarballs when saving of transferring them. In the UNIX tradition of doing one thing and doing it well, the Tar package does not do any kind of compression and instead makes it easy to compose it's API with external compression tools. The simplest way to read a compressed archive is to use a command-line tool to decompress it. For example:

Tar.list(`gzcat $tarball`)
Tar.extract(`gzcat $tarball`)

Despite the simple extension .tar, your archive is compressed

% file bulk_vlsv.tar
bulk_vlsv.tar: gzip compressed data, last modified: Sat Jan 23 09:05:21 2021, from Unix, original size modulo 2^32 5171200
henry2004y commented 3 years ago

Sorry I am not familiar at all with the concepts here. So what should I do to fulfill my goal then?

On Linux/Mac, using the command line works well:

run(`tar -zxvf bulk_vlsv.tar`)

But how should I do this on Windows?

giordano commented 3 years ago

If you want to use external commands (like tar), you need to have them installed.

If you want to do everything with Julia, you can use for example CodecZlib.jl together with Tar.jl:

julia> using CodecZlib, Tar

julia> open("bulk_vlsv.tar") do io
           Tar.extract(GzipDecompressorStream(io))
       end
"/tmp/jl_mf7624"
henry2004y commented 3 years ago

Thanks!