[Compact issue] Offset overflow

fengshao0907 / weed-fs

Automatically exported from code.google.com/p/weed-fs

0 stars 0 forks source link

[Compact issue] Offset overflow #45

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago

You are using uint32(Maximum 4Gb) to store needle offset(Maximum 32Gb) when 
compacting.
Currently It is ok if the volume size is < 4gb
Change variable "offset" in ScanVolumeFile function to uint64 to fix the issue.

Original issue reported on code.google.com by hieu.hcmus@gmail.com on 19 Sep 2013 at 7:15

GoogleCodeExporter commented 9 years ago

This is by design. There is a up-to-8-bytes padding to use uint32 to represent 
32GB offset.

Original comment by chris...@gmail.com on 19 Sep 2013 at 7:54

Changed state: Invalid

GoogleCodeExporter commented 9 years ago

Yes, I know that.
You are using uint32(* 8 bytes padding = 32Gb  maximum) to represent the needle 
offset in memory.
When compacting, you read needles from dat file, the "offset" variable in 
ScanVolumeFile function is not divided by 8 --> so it should be uint64 not 
uint32

Please read the code carefully

Original comment by hieu.hcmus@gmail.com on 19 Sep 2013 at 8:06

GoogleCodeExporter commented 9 years ago

Read the code carefully or you can test by Println offset in ScanVolumeFile 
function when compacting volume(size > 4gb) :)

Original comment by hieu.hcmus@gmail.com on 19 Sep 2013 at 10:41

GoogleCodeExporter commented 9 years ago

Original comment by chris...@gmail.com on 19 Sep 2013 at 4:44

Changed state: Started

GoogleCodeExporter commented 9 years ago

Thanks for your detailed suggestions! Fix now is in v0.42

Original comment by chris...@gmail.com on 19 Sep 2013 at 7:27

Changed state: Fixed