dmik
commented
6 years ago From what I see, there is actually nothing that would restrict supporting files >4G, all relevant structure members are 64 bits. The only significant problem is the way how we track dirty pages. We use 1 bit per each page. For example, for a file as big as 64 GB, it will end up in 2 MB of the dirty map array (64 1024 1024 * 1024 / 4096 (page size) / 8 (bits per byte)). So if the file to map is too big, we will simply end up returning ENOMEM because there won't be enough memory for the dirty map array.
Note that mapping more than 4G (in reality — much less, like 2G or so) is impossible anyway because our address space is 32 bits. This means that any call with the size more than 2 G will fail with ENOMEM anyway. However, it's still possible to read very big files (like 256 GB for instance) in 32 bits using much smaller window (e.g. 1 M). But given that we always create the dirty map for the entire file, this won't help in terms of a map file: a 256 GB file will require 8 MB of the dirty map array anyway, regardless of the window size.
This is done so because we share the dirty map and the mapped pages themselves between all mmap calls to a given file (to save memory). In theory, I can change the implementation so that it doesn't use actual space for unused buts but this will make the logic more complex so I will postpone it until we face a real use case. For now I will simply try remove the check that returns EINVAL.
Note that all the above (and this ticket itself) only applies to MAP_SHARED mappings (and only writable ones). MAP_PRIVATE and r/o MAP_SHARED ones don't write back to disk so they don't need a dirty map at all.
Currently, mapping regions of a file at offsets greater than 4G results in EOVERFLOW (comments say it's a technical overflow). This makes Qt 5 tests fail, see https://github.com/bitwiseworks/qtbase-os2/issues/39.