aldanor / hdf5-rust

HDF5 for Rust
https://docs.rs/hdf5
Apache License 2.0
310 stars 84 forks source link

Implement Send + Sync for VarLenUnicode, VarLenAscii, VarLenArray #269

Closed watsaig closed 9 months ago

watsaig commented 9 months ago

I am implementing a utility to write data to an HDF5 file in a thread, which works fine for some types but not others. For example, any type that includes VarLenUnicode, VarLenAscii or VarLenArray cannot be sent between threads.

Here is a minimal example:

use hdf5::types::VarLenUnicode;

#[derive(hdf5::H5Type)]
#[repr(C)]
struct CustomType {
    text: VarLenUnicode,
}

fn write_in_thread() {
    let file = hdf5::File::create("/tmp/dset.h5").unwrap();

    let data = vec![
        CustomType {
            text: VarLenUnicode::from_str("foo").unwrap(),
        },
        CustomType {
            text: VarLenUnicode::from_str("bar").unwrap(),
        },
    ];

    std::thread::spawn(move || {
        file.group("/")
            .unwrap()
            .new_dataset_builder()
            .with_data(&data)
            .create("dataset")
            .unwrap();
    });
}

Output (truncated):

error[E0277]: `*mut u8` cannot be sent between threads safely
   --> src\io\src\writer.rs:136:28
    |
136 |           std::thread::spawn(move || {
    |  _________------------------_^
    | |         |
    | |         required by a bound introduced by this call
137 | |             file.group("/")
138 | |                 .unwrap()
139 | |                 .new_dataset_builder()
...   |
142 | |                 .unwrap();
143 | |         });
    | |_________^ `*mut u8` cannot be sent between threads safely
    |
    = help: within `CustomType`, the trait `std::marker::Send` is not implemented for `*mut u8`
note: required because it appears within the type `VarLenUnicode`
   --> C:\Users\watsaig\.cargo\git\checkouts\hdf5-rust-c1072d7a2617f7d8\2e44e90\hdf5-types\src\string.rs:288:12
    |
288 | pub struct VarLenUnicode {
    |            ^^^^^^^^^^^^^
note: required because it appears within the type `CustomType`
   --> src\io\src\writer.rs:149:8
    |
149 | struct CustomType {
    |        ^^^^^^^^^^
    = note: required for `Unique<CustomType>` to implement `std::marker::Send`
note: required because it appears within the type `RawVec<CustomType>`

Is it possible to make these types thread-safe? I'm happy to help with a PR but not sure where to start.

mulimoen commented 9 months ago

It should be safe to implement Send for these types. The nomicon contains information on how to implement this trait. These trait impls should be added to string.rs and array.rs.