memfile mapMem() offset for windows

[jlp765]

The windows call MapViewOfFileEx() has a memory mapping granularity (usually around 4K).

The combination of the high and low offsets must specify an offset within the file mapping. 
They must also match the memory allocation granularity of the system. 
That is, the offset must be a multiple of the allocation granularity. 
To obtain the memory allocation granularity of the system, use the GetSystemInfo function, 
which fills in the members of a SYSTEM_INFO structure.

Calls to mapMem for offsets less than the granularity should be mapped internally to offsets in the mem buffer of the MemFile object.

Currently it fails with

Traceback (most recent call last)
tmemfiles2.nim(41)       tmemfiles2
memfiles.nim(56)         mapMem
os.nim(151)              raiseOSError
Error: unhandled exception: The base address or the file offset specified does n
ot have the proper alignment.
 [OSError]

for the test

discard """
  file: "tmemfiles2.nim"
  disabled: true
  output: '''Full read size: 20
Half read size: 10 Data: Hello'''
"""
# doesn't work on windows. fmReadWrite doesn't create a file.
import memfiles, os
var
  mm, mm_full, mm_half: MemFile
  fn = "test.mmap"
  p: pointer

if fileExists(fn): removeFile(fn)

# Create a new file, data all zeros
mm = memfiles.open(fn, mode = fmReadWrite, newFileSize = 20)
mm.close()

# read, change
mm_full = memfiles.open(fn, mode = fmWrite, mappedSize = -1)
echo "Full read size: ",mm_full.size
p = mm_full.mapMem(fmReadWrite, 20, 0)
var p2 = cast[cstring](p)
p2[0] = 'H'
p2[1] = 'e'
p2[2] = 'l'
p2[3] = 'l'
p2[4] = 'o'
p2[5] = '\0'
mm_full.unmapMem(p, 20)
mm_full.close()

# read half, and verify data change
mm_half = memfiles.open(fn, mode = fmRead, mappedSize = 10)
echo "Half read size: ",mm_half.size, " Data: ", cast[cstring](mm_half.mem)
mm_half.close()

mm_full = memfiles.open(fn, mode = fmReadWrite, mappedSize = -1)
echo "Full read size: ",mm_full.size, " Read second half of file."
p = mm_full.mapMem(fmReadWrite, 10, 10)
p2[0] = 'H'
p2[1] = 'e'
p2[2] = 'l'
p2[3] = 'l'
p2[4] = 'o'
p2[5] = '\0'
mm_full.unmapMem(p, 10)
mm_half.close()

if fileExists(fn): removeFile(fn)

[Araq]

Is that really a bug though? Memory mapping of files is low level and we give you what the OS supports.

[jlp765]

Currently it is an undocumented feature (in the memFiles doco). If it is different on different OS, then that should also be documented.

There should at least be a function that provides (in an OS independent way) the min offset size that can be mapped.

However, it would be much more user friendly to provide the means to map any sized object.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think it is still a valid issue, write a comment below; otherwise it will be closed. Thank you for your contributions.