kawamuray / wasmtime-java

Java or JVM-language binding for Wasmtime
Apache License 2.0
128 stars 29 forks source link

Implement memory.grow() #7

Closed SuperIceCN closed 3 years ago

SuperIceCN commented 3 years ago

Implement memory.grow() to allow expanding memories from java.

SuperIceCN commented 3 years ago

@kawamuray OK, done!

SuperIceCN commented 3 years ago

@kawamuray As grow method being implemented, the memory can grow really big and its size will be greater than 2147483648, which is the index limit of java.nio.ByteBuffer. When an 'u32' number which is greater than 2147483648 convert into an i32 number, it will become a negative number. It seems that we can't use a negative number as an index on java.nio.ByteBuffer. We will not able to read some parts of a memory then. Any solutions?

kawamuray commented 3 years ago

which is the index limit of java.nio.ByteBuffer

oh, that's really a good point. The only use cases I've ever tested didn't created more than 2GiB memory so I didn't realized that issue.

I think we have 2 possible solutions for that:

  1. Stop using ByteBuffer interface and introduce own "buffer" type that directly maps to host memory address space.
  2. Add another method ByteBuffer buffer(long offset) or ByteBuffer buffer(long offsetPages) (offset by address or by number of pages) that computes base address by the given offset.

I think the first option's pros is that we can use a single "buffer" object to access entire memory space, but it also has a big downside which that we will be disabled to use java APIs that often uses java.nio.ByteBuffer as the type to take a chunk of data. Also, mimicing whole APIs provided by ByteBuffer by ourselves is a big deal. On the other hand, the 2nd option has a little downside that users may have to manage multiple ByteBuffer instances to access the range of memory that they want to access, but we can keep using ByteBuffer which is very familiar API for many java developers and is compatible with many useful APIs. Creating a subclass of ByteBuffer which supports long version methods like position(long) isn't an option since it is nearly impossible to create its subclass by design.

So how about this:

  1. Add a new method ByteBuffer buffer(long offsetPages) that uses offset slice to call env.new_direct_byte_buffer() like env.new_direct_byte_buffer(&mut ptr[offset..])
  2. User code calls it with offset to obtain the space they want to access. For example, if they want to access last 500MiB portion out of 3.0GiB memory, they can do buffer = memory.buffer(2684354560 / WASM_PAGE_SIZE(=64k)).
SuperIceCN commented 3 years ago

@kawamuray Could you please merge this pull request? I think we can talk about the two solutions in another issue.

kawamuray commented 3 years ago

I think we can talk about the two solutions in another issue

ok let's do that.

kawamuray commented 3 years ago

LGTM, thanks!