search

JuliaLang / julia

The Julia Programming Language
https://julialang.org/
MIT License
45.63k stars 5.48k forks source link

mmap Union of bits types #26309

Open tpapp opened 6 years ago

tpapp commented 6 years ago

Now that arrays with Union of bitstypes elements have an optimized layout, it would be useful if one could mmap them. Alternatively, exporting the API that would allow the user to mmap the constituent flag and content vectors, then "assemble" the array, would also be as solution.

Cf

  1. a related issue on exporting the API: https://github.com/JuliaLang/julia/issues/24057
  2. discussion (I am opening this as a reminder, as suggested)
  3. I would use it in this package
Keno commented 6 years ago

The in memory representation of union bits types is not (and is not supposed to be) stable. However, you can achieve the same performance by an array view type of the appropriate element that uses the same logic (implemented in julia) and that'll keep working if we decide to change the representation.

tpapp commented 6 years ago

Revisiting this issue, I am wondering if the representation has stabilized yet. It would be nice to mmap eg Vector{Union{Missing,Int}} and similar.