Error: `[scan] Unable to scan target: gzip: invalid header`

[thekthuser]

When I run this after scanning 90+% or possibly 100% I get an uncountable (past my terminal's scrollback limit) number of lines with variations of:

[scan] Starting new target: Gzipfile @ byte 77406201856 in [/dev/sdb]
[scan] Unable to scan target: gzip: invalid header

followed by:

[COMPLETE]
panic: send on closed channel

goroutine 5 [running]:
panic(0x4f6d20, 0xc821a68000)
        /usr/lib/go-1.6/src/runtime/panic.go:481 +0x3e6
github.com/jakewins/findbtc/detector.scanGzipFiles(0xc82006c300, 0xc82006c360, 0xc82006c1e0)
        /home/user/go/src/github.com/jakewins/findbtc/detector/gzip_scanner.go:77 +0xb6
created by github.com/jakewins/findbtc/detector.Scan
        /home/user/go/src/github.com/jakewins/findbtc/detector/detector.go:123 +0x387
panic: send on closed channel

goroutine 4 [running]:
panic(0x4f6d20, 0xc821a66000)
        /usr/lib/go-1.6/src/runtime/panic.go:481 +0x3e6
github.com/jakewins/findbtc/detector.scanZipFiles(0xc82006c2a0, 0xc82006c300, 0xc82006c1e0)
        /home/user/go/src/github.com/jakewins/findbtc/detector/zip_scanner.go:106 +0xb6
created by github.com/jakewins/findbtc/detector.Scan
        /home/user/go/src/github.com/jakewins/findbtc/detector/detector.go:120 +0x351

Is there an argument I can add to get around this? Are the found (maybe just possible) wallets being saved somewhere I can find?

[thekthuser]

Should I be using a start-offset and how do I determine what that should be?

[jakewins]

@thekthuser I don't think start-offset stuff will help you here. I'm rather strapped for time, so I won't be able to help sort this in the short term; however, if someone comes along and is open to submitting a patch, I'm happy to review it.

There are two things to look at

• 1. (Easy) Remove the log message Unable to scan target for gzips; this is expected, it means it found something that looked like a gzip file, but the file was so broken it couldn't read it. Flooding the log with those messages is unhelpful
• 2. (Hard) Find source of the race condition bug you've bumped into where close happens before the system is finished. It could be that it'd be better to rewrite the scanner to just be single-threaded; I don't think much performance is gained from the goroutines in the code since the I/O of Go is so far from exhausting read performance of a modern disk