search

huggingface / ratchet

A cross-platform browser ML framework.
https://ratchet.sh
MIT License
586 stars 31 forks source link

Minimum number of copies from `Uint8Array` to `wgpu::Buffer` #171

Open FL33TW00D opened 5 months ago

FL33TW00D commented 5 months ago

Currently in wgpu, there is no way to create a wgpu::Buffer directly from Uint8Array.

This means that our tensor data does something like this:

IndexedDB -> Uint8Array -> Vec<u8> ~~> Uint8Array -> WebGPU

Judging by this: https://developer.mozilla.org/en-US/docs/Web/API/GPUBuffer

I think the optimal is:

IndexedDB -> Uint8Array -> WebGPU

Reference: https://github.com/gfx-rs/wgpu/blob/trunk/wgpu/src/backend/webgpu.rs#L1821

sigma-andex commented 5 months ago

If we store the webgpu aligned Uint8Array, can't we just pass it through when loading?

FL33TW00D commented 5 months ago

@sigma-andex There is just no way to provide a Uint8Array to wgpu as of now, we need to add it

ikorin24 commented 3 months ago

Hi, I am also interested in this issue.

I found a method wgpu::BufferSlice::get_mapped_range_as_array_buffer. It seems to be available only in the webgpu feature. Would this be of any help to you?

FL33TW00D commented 3 months ago

wgpu::BufferSlice::get_mapped_range_as_array_buffer

@ikorin24 It looks like that method was added in this PR: https://github.com/gfx-rs/wgpu/pull/4042

It seems to help only in the other direction (i.e getting data from WebGPU land -> JS land with less copies). We need the other direction to be added!