Panic errors when repeating `gowitness single`

[austinginder]

I'm attempting to batch collect images from hundreds of websites. Using gowitness file with threads seems to work fine. However for more control I'd like to use gowitness single as it's part of a larger bash script. However running multiple gowitness single commands or even many gowitness single command consecutively and I'm get panic errors as shown below.

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1357950]

goroutine 1 [running]:
github.com/tidwall/buntdb.(*Tx).lock(0xc0004f8440)
        /Users/austinginder/go/src/github.com/tidwall/buntdb/buntdb.go:1088 +0x30
github.com/tidwall/buntdb.(*DB).Begin(0x0, 0xc000126701, 0x115e9c5, 0xc0000881e0, 0xc0001420c0)
        /Users/austinginder/go/src/github.com/tidwall/buntdb/buntdb.go:1067 +0x61
github.com/tidwall/buntdb.(*DB).managed(0x0, 0x115c901, 0xc0001268b8, 0x0, 0x0)
        /Users/austinginder/go/src/github.com/tidwall/buntdb/buntdb.go:939 +0x40
github.com/tidwall/buntdb.(*DB).Update(0x0, 0xc0001268b8, 0x1, 0x1)
        /Users/austinginder/go/src/github.com/tidwall/buntdb/buntdb.go:984 +0x3a
github.com/sensepost/gowitness/storage.(*Storage).SetHTTPData(0x1929de8, 0xc00017e500)
        /Users/austinginder/go/src/github.com/sensepost/gowitness/storage/storage.go:53 +0x456
github.com/sensepost/gowitness/utils.ProcessURL(0xc00017e480, 0x192c0e0, 0x1929de8, 0x3c)
        /Users/austinginder/go/src/github.com/sensepost/gowitness/utils/processor.go:109 +0x1c99
github.com/sensepost/gowitness/cmd.glob..func5(0x19239c0, 0xc000088a20, 0x0, 0x6)
        /Users/austinginder/go/src/github.com/sensepost/gowitness/cmd/single.go:36 +0x8c
github.com/spf13/cobra.(*Command).execute(0x19239c0, 0xc000088960, 0x6, 0x6, 0x19239c0, 0xc000088960)
        /Users/austinginder/go/src/github.com/spf13/cobra/command.go:766 +0x2cc
github.com/spf13/cobra.(*Command).ExecuteC(0x1923500, 0xc000127f50, 0x14447ce, 0x1923500)
        /Users/austinginder/go/src/github.com/spf13/cobra/command.go:852 +0x2fd
github.com/spf13/cobra.(*Command).Execute(0x1923500, 0xc000092058, 0x0)
        /Users/austinginder/go/src/github.com/spf13/cobra/command.go:800 +0x2b
github.com/sensepost/gowitness/cmd.Execute()
        /Users/austinginder/go/src/github.com/sensepost/gowitness/cmd/root.go:93 +0x31
main.main()
        /Users/austinginder/go/src/github.com/sensepost/gowitness/main.go:6 +0x20

If helpful here is the format of the single command I'm using.

gowitness single --url="$url" --resolution="1200,750" --log-format=json --log-level=warn --timeout=60 --destination="/Users/austinginder/Desktop/Screenshots/"

[austinginder]

Actually I haven't been able to pinpoint the cause of this issue. It seems to periodically happen on both single and file commands. Sometimes I can crunch through hundreds of screenshots. Other times it gives an error after only a few screenshots.

[austinginder]

Ah... it appears DB related. If I delete the gowitness.db file it generates images just fine. If there is an existing gowitness.db file it results in the above errors sporadically.

[yaworsk]

In case others run into it, I also got similar panic errors but the result was using the -D flag without providing an actual DB name. I was just pointing to a directory.

[leonjza]

Hey, thanks for the report! I think this may be as a result of a race condition when using the buntdb storage which admittedly makes little sense in single mode. As the db is only used when generating reports, I think I should add a flag to just disable writing the db entirely in cases where it may not be needed.