Cannot memory map with offset under Win7

[ghost]

I tried to run the following mmap_array example and it doesn't work as it throws an error when calling mmap_array under Win7.

# Create a file for mmapping
# (you could alternatively use mmap_array to do this step, too)
A = rand(1:20, 5, 30)
s = open("/tmp/mmap.bin", "w+")
# We'll write the dimensions of the array as the first two Ints in the file
write(s, size(A,1))
write(s, size(A,2))
# Now write the data
write(s, A)
close(s)

# Test by reading it back in
s = open("/tmp/mmap.bin")   # default is read-only
m = read(s, Int)
n = read(s, Int)
A2 = mmap_array(Int, (m,n), s)

julia> A2 = mmap_array(Int, (m,n), s)
ERROR: could not create mapping view in mmap_array at mmap:145

[JeffBezanson]

I suspect on windows you need to call close before mmap_array. The example should be updated to reflect this.

[tkelman]

I've been noticing this in a few packages (https://github.com/timholy/HDF5.jl/issues/89 for example), so thanks for reducing it!

How would mmap'ing a closed file work?

julia> A2 = mmap_array(Int, (m,n), s)
ERROR: could not create mapping view
 in mmap_array at mmap.jl:139
julia> close(s)
julia> A2 = mmap_array(Int, (m,n), s)
ERROR: could not get handle for file to map
 in mmap_array at mmap.jl:122

[timholy]

Maybe if you insert the close statement prior to https://github.com/JuliaLang/julia/blob/master/base/mmap.jl#L138? I can't test this myself, unfortunately.

[tkelman]

Does not appear to fix things - different error though, ERROR: could not create file mapping. I'm happy to test any more ideas you can come up with.

[JeffBezanson]

Maybe windows doesn't support mmap if the file position is non-zero? Try seek(s,0) before mmap? You can pass an offset as an extra argument to mmap_array.

[tkelman]

That looks more promising. seek(s,0) works for this case - but non-zero offsets don't.

[timholy]

I wonder if any multiple of the pagesize works.

One could change the code to extract the current position, seek(s,0), and then offset the pointer passed to pointer_to_array.

[ghost]

I tested two things:

1. close(s) prior to mmap_array(): this did not work given it closed file handle throwing error at mmap.jl:125
2. removed reads for the dimensions (m & n): this didn't gave an error but gave a matrix with wrong dimensions

[tkelman]

@timholy's suggestion sounds worth trying. If I wasn't about to leave for a conference for the rest of the week I'd take a crack at it.

[tkelman]

Ah, looks like #6203 has a suggested implementation of something similar, I'll give the code there a try