Mmap used in SnpArrays.jl currently allocates virtual memory as large as the bed file size. This normally would not be a serious issue, but when virtual memory is limited by external option (e.g., on Hoffman2), this seriously limits the possibility of running on large bed files. I think it is preferable to provide an option to run it without mmap, especially for GWAS-like applications. In particular, for GWAS-like applications, it will be relatively simple to create something like VariantIterator and ref_allele_dosage!() in BGEN.jl, without using mmap. A similar interface is also planned for PGEN.jl.
Mmap used in SnpArrays.jl currently allocates virtual memory as large as the bed file size. This normally would not be a serious issue, but when virtual memory is limited by external option (e.g., on Hoffman2), this seriously limits the possibility of running on large bed files. I think it is preferable to provide an option to run it without mmap, especially for GWAS-like applications. In particular, for GWAS-like applications, it will be relatively simple to create something like
VariantIteratorandref_allele_dosage!()in BGEN.jl, without using mmap. A similar interface is also planned for PGEN.jl.