panic: runtime error: invalid memory address or nil pointer dereference

[GoogleCodeExporter]

What steps will reproduce the problem?
1. start a 1 master 1 volume weed cluster
2. dd if=/dev/urandom bs=1024 count=200000 of=/tmp/data
3. weed upload /tmp/data
4. delete the file using curl (curl -X DELETE file_url)
5. cd into the weed storage folder
6. find the .dat file that was used to store the data (it should be 196MB)
7. weed export -dir . -volumeId THE_VOLUME_ID

What is the expected output? What do you see instead?
Expected output is a 0 length file listing.  Or maybe show me that the file is 
still there but has been marked as deleted.

Actual output:

shelly /storage/weed $ weed export -dir . -volumeId 5
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x474625]

goroutine 1 [running]:
runtime.panic(0x7232c0, 0xb47848)
    /home/chris/apps/go/src/pkg/runtime/panic.c:266 +0xb6
code.google.com/p/weed-fs/go/storage.(*Volume).readSuperBlock(0xc210050d90, 
0x7fb8d068be80, 0xc2100430c0)
    /home/chris/dev/workspace/home/gopath/src/code.google.com/p/weed-fs/go/storage/volume.go:153 +0x95
code.google.com/p/weed-fs/go/storage.(*Volume).load(0xc210050d90, 0xc210050d00, 
0x10, 0x41cb76)
    /home/chris/dev/workspace/home/gopath/src/code.google.com/p/weed-fs/go/storage/volume.go:82 +0x642
code.google.com/p/weed-fs/go/storage.loadVolumeWithoutIndex(0x7fff6ca90b7a, 
0x1, 0x781a60, 0x0, 0x7fb800000005, ...)
    /home/chris/dev/workspace/home/gopath/src/code.google.com/p/weed-fs/go/storage/volume.go:53 +0xc4
code.google.com/p/weed-fs/go/storage.ScanVolumeFile(0x7fff6ca90b7a, 0x1, 
0x781a60, 0x0, 0xc200000005, ...)
    /home/chris/dev/workspace/home/gopath/src/code.google.com/p/weed-fs/go/storage/volume.go:322 +0x81
main.runExport(0xb45ba0, 0xc21000a060, 0x0, 0x0, 0x0)
    /home/chris/dev/workspace/home/gopath/src/code.google.com/p/weed-fs/go/weed/export.go:115 +0x8da
main.main()
    /home/chris/dev/workspace/home/gopath/src/code.google.com/p/weed-fs/go/weed/weed.go:76 +0x50a

goroutine 3 [chan receive]:
code.google.com/p/weed-fs/go/glog.(*loggingT).flushDaemon(0xb4d220)
    /home/chris/dev/workspace/home/gopath/src/code.google.com/p/weed-fs/go/glog/glog.go:835 +0x50
created by code.google.com/p/weed-fs/go/glog.init·1
    /home/chris/dev/workspace/home/gopath/src/code.google.com/p/weed-fs/go/glog/glog.go:402 +0x276

goroutine 4 [syscall]:
runtime.goexit()
    /home/chris/apps/go/src/pkg/runtime/proc.c:1394

What version of the product are you using? On what operating system?
0.45
ubuntu 12.04

Please provide any additional information below.

Original issue reported on code.google.com by sverre.c...@gmail.com on 24 Jan 2014 at 1:51

[GoogleCodeExporter]

Can not reproduce it. Does it happen consistently or just one time? Can you 
reproduce it?

chris@clu-dt64:/tmp$ ./weed_0.45_linux_amd64/weed export -dir=. -volumeId=4
key=4,4e21899b6d4e Name=data1 Size=20480000 gzip=false mime=
chris@clu-dt64:/tmp$ 
chris@clu-dt64:/tmp$ 
chris@clu-dt64:/tmp$ curl -X DELETE http://localhost:8080/4,4e21899b6d4e
{"size":20480037}chris@clu-dt64:/tmp$ 
chris@clu-dt64:/tmp$ 
chris@clu-dt64:/tmp$ ./weed_0.45_linux_amd64/weed export -dir=. -volumeId=4
chris@clu-dt64:/tmp$ 

Original comment by chris...@gmail.com on 24 Jan 2014 at 6:40

[GoogleCodeExporter]

I suspect you got into this situation when the volume server is still running. 
The volume server periodically tries to compact volumes, which will likely 
happen since your case has a major deletion. When the volumes are compacting, 
the data file will change underneath, leading to your export error.

When doing export, you'd better shut volume server down.

Original comment by chris...@gmail.com on 24 Jan 2014 at 7:50

• Changed state: Invalid

[GoogleCodeExporter]

Thanks for the feedback Chris!!  Sorry for not getting back to you.  I receive 
no notifications from Google code for some reason...

In other news, I will take your advice on the volume server stuff.  I am sure 
that you are right.

Thanks!

Original comment by c...@memsql.com on 5 Feb 2014 at 6:08