evmar / retrowin32

windows emulator
https://evmar.github.io/retrowin32/
Apache License 2.0
578 stars 24 forks source link

Rethink `Mem` (cannot create a slice starting at memory offset 0) #26

Closed evmar closed 4 weeks ago

evmar commented 3 months ago

The Mem type is used to model "the x86 memory". In the x86-64 case that means it just conceptually is a slice from address 0 to 2^32. However, it cannot actually be a slice starting at address 0 because Rust doesn't allow this.

I originally introduced this Mem type for a few reasons:

However, a better approach (I think?) for (1) is to instead extend slices with these getters, via the memory::Extensions trait I (later) added. So one idea I have for fixing this in general is to say: Mem always only represents "all of memory", get rid of like mem.sub() which returns a sliced Mem subregion in favor of making that return a slice.

One thing to keep in mind is that for MMU reasons, we don't want to have code rely on creating "big" slices. E.g. if we had some code like mem.slice(ofs..) that returned a &[u8] then that would require all memory from ofs forward to be contiguous, which may not be true with an MMU. In particular this pattern comes up when:

  1. reading nul-terminated strings, because we don't know how long to slice until we start reading bytes
  2. PE handling, where many addresses are relative to "the image base"

You can grep the code for _todo() to see all the places this slicing is done. So I think my vague plan is to fix those, then get rid of the .as_slice methods in favor of making mem.sub() return a slice directly.

Finally, regarding the aliasing muts, I think using more .get_pod (copying from memory) and fewer .view_* (creating pointers into memory) methods will reduce aliasing and is generally a better approach because it also avoids alignment problems.

evmar commented 3 months ago

One big place where aliasing is a problem is if you grab some pointer into memory, mem.view::<Foo>(addr) and then later cause memory to grow (on web for example memory is backed by a Vec that can grow), that invalidates the pointer. However, with an MMU maybe we could handle growing memory differently. And on web it's possible we could put the x86 heap last in memory such that it can always grow without moving. I'm still not sure if it is a good idea to make it possible to invalidate pointers in this way, but so far retrowin32 (painfully) handles it.

evmar commented 4 weeks ago

This is fixed, we now use slices everywhere we refer to a subregion of memory.