gzuidhof / zarr.js

Javascript implementation of Zarr
https://guido.io/zarr.js
Apache License 2.0
133 stars 23 forks source link

Write zarr array to remote HTTPStore #108

Open oeway opened 3 years ago

oeway commented 3 years ago

Hi, I am trying to use the HTTPStore to write zarr array to remote servers (where I implemented the PUT method), however I couldn't find any example for doing it for HTTPStore.

This is what I have so far:

        const fetchOptions = { redirect: 'follow', headers: { 'Authorization': `Bearer ${token}` } };
        const supportedMethods = ['GET', 'PUT', 'HEAD']; // defaults
        const store = new zarr.HTTPStore(`https://my-website.io/123/mydata.zarr`, { fetchOptions, supportedMethods });
        const z = await zarr.zeros([1000, 1000], {chunks: [100, 100], dtype: '<i4', store});
        await z.set(null, 42);
        await z.set([0, null], zarr.NestedArray.arange(1000, "<i4")); 
        await z.set([null, 0], zarr.NestedArray.arange(1000, "<i4")); 
        console.log(await z.get([0, 0]));

I tested with this, but it failed at "fetcing https://my-website.io/123/mydata.zarr/.zarray" (when calling containsItem), I tried to set overwrite: true, but it said not implemented yet.

gzuidhof commented 3 years ago

Hi Wei,

You can find the implementation here: https://github.com/gzuidhof/zarr.js/blob/master/src/storage/httpStore.ts#L58-L68. I wouldn't be surprised if you were one of the first to write to a remote server using the HTTPStore so it's a bit of an uncharted territory (without any examples).

What fetch errors are you seeing (in the Network tab in your browser)? Is it the webserver rejecting the message, or is it something internal to zarr/httpstore going wrong?