gonum / hdf5

hdf5 is a wrapper for the HDF5 library
BSD 3-Clause "New" or "Revised" License
131 stars 33 forks source link

Don't have support for resize(H5Dset_extent) #53

Open zyc-sudo opened 5 years ago

zyc-sudo commented 5 years ago

Hi, I tried to find the wrapper for H5Dset_extent which resize the chunked datasets. I forked the repo to https://github.com/zyc-sudo/hdf5 and implemented the wrapper in h5d_dataset.go as func (s *Dataset) Resize(dims []uint)

I'm not sure if you guys already implemented it or not. If you already did, could you please tell me where it is? Or if you didn't let me know too so we can do pull request when I finish it to contribute a little to this project

kortschak commented 5 years ago

We have not implemented that. However, the implementation you have is not currently safe. What happens if someone calls s.Resize([]uint{})?

zyc-sudo commented 5 years ago

Thank you for your response. You are correct, it is not safe at all and I’m new to golang also. I was doing another project and need this feature so badly that I created it in a “I just want it to work” mind set. I’ll perfect that with proper error handling and input validation for sure. Thank you very much for your response and I’ll keep you posted when I actually have a better version of it. I hope I can contribute to this project when I implement a better version.

On Wed, Jul 24, 2019 at 7:39 PM Dan Kortschak notifications@github.com wrote:

We have not implemented that. However, the implementation you have is not currently safe. What happens if someone calls s.Resize([]uint{})?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/gonum/hdf5/issues/53?email_source=notifications&email_token=AEOWLFXWCDEGKMYAFGYMDNLQBDR4XA5CNFSM4IGSGZZ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2X5APA#issuecomment-514838588, or mute the thread https://github.com/notifications/unsubscribe-auth/AEOWLFT6BVMXRFY3ALC2I2LQBDR4XANCNFSM4IGSGZZQ .

kortschak commented 5 years ago

Yes, please do. I'll go through the code with comments with a view to merging this when I get some time.

zyc-sudo commented 5 years ago

Thanks!

On Wed, Jul 24, 2019 at 8:23 PM Dan Kortschak notifications@github.com wrote:

Yes, please do. I'll go through the code with comments with a view to merging this when I get some time.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/gonum/hdf5/issues/53?email_source=notifications&email_token=AEOWLFSREYDLJKMTEG2Z46TQBDW67A5CNFSM4IGSGZZ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2X7AQY#issuecomment-514846787, or mute the thread https://github.com/notifications/unsubscribe-auth/AEOWLFVP3LTYPMSP56IGMXTQBDW67ANCNFSM4IGSGZZQ .

mrwill84 commented 1 year ago

wow good project . Bros, it's merged ?

zyc-sudo commented 1 year ago

wow good project . Bros, it's merged ?

Nope. I think the project is stalled. If you want to use it, you can use my fork where I implemented it.

mrwill84 commented 1 year ago

@zyc-sudo thank you ;)

mrwill84 commented 1 year ago

@zyc-sudo man, since your repo can't open issue to support, could you take a quick look at ? it failed when I want to resize dim . https://gist.github.com/mrwill84/66f47b5e4dc2e9de5fdc45de69983e09

zyc-sudo commented 1 year ago

It needs a “max size” when you create the dataspace instead of nil. You put nil: hdf5.CreateSimpleDataspace(dims, nil) It should not be nil. Have a look at this file: https://github.com/zyc-sudo/hdf5/blob/master/h5d_dataset_test.go

There is a test code : func TestDatasetResize In this function you can see you need to specify the maximum size the dataset can be resized to:

https://github.com/zyc-sudo/hdf5/blob/036a2c6fedc9ec44f82aef381e8e6e1d3679391f/h5d_dataset_test.go#L225 Also the chunk size need to be set too:

https://github.com/zyc-sudo/hdf5/blob/036a2c6fedc9ec44f82aef381e8e6e1d3679391f/h5d_dataset_test.go#L236 I recall that only “chunked” dataset can be resized. But you can refer to the hdf5 C api to verify this. this test code has passed so I know if you follow it , it works

On Sun, Aug 27, 2023 at 11:49 PM zhao.qing @.***> wrote:

@zyc-sudo https://github.com/zyc-sudo man, since your repo can't open issue to support, could you take a quick look at ? it failed when I want to resize dim . https://gist.github.com/mrwill84/66f47b5e4dc2e9de5fdc45de69983e09

— Reply to this email directly, view it on GitHub https://github.com/gonum/hdf5/issues/53#issuecomment-1694961636, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEOWLFWVSYQ6E7EKG3WBOQDXXQIMVANCNFSM4IGSGZZQ . You are receiving this because you were mentioned.Message ID: @.***>

mrwill84 commented 1 year ago

thank you man