apache / arrow-rs

Official Rust implementation of Apache Arrow
https://arrow.apache.org/
Apache License 2.0
2.62k stars 802 forks source link

More trivial implementation of `Box<dyn AsyncArrowWriter>` and `Box<dyn AsyncArrowReader>` #6748

Closed ethe closed 1 day ago

ethe commented 3 days ago

In the current,

impl AsyncFileReader for Box<dyn AsyncFileReader> {
    ...
}

equals to

impl AsyncFileReader for Box<dyn AsyncFileReader + 'static> {
    ...
}

which 'static bound is not necessary, for all 'a, Box<dyn AsyncFileReader + 'a> should be an AsyncFileReader. I removed this bound to make it more trivial

jayzhan211 commented 3 days ago

I think adding read and write is not the same as static

ethe commented 3 days ago

It is not the same, but a superset of it. 'read and 'write are able to be 'static, and also make non-static for<'a>: Box<dyn AsyncArrowWriter + 'a> be an AsyncArrowWriter

jayzhan211 commented 2 days ago

I'm still not convinced why we need to add lifetime for it. 👀 In my understanding, we should start from the simple one without lifetime until we want to implement with reference that compiler starts to complain about it.

It would be clear if there is an example that compilation failed without lifetime

ethe commented 2 days ago

Considering this case:

pub struct ArrowReader<'reader, T> {
    reader: &'reader T,
}

impl<T: AsyncFileReader + Send + Sync> AsyncFileReader for ArrowReader<'_, T> {
    fn get_bytes(
        &mut self,
        range: std::ops::Range<usize>,
    ) -> std::pin::Pin<
        Box<dyn std::future::Future<Output = parquet::errors::Result<bytes::Bytes>> + Send + '_>,
    > {
        todo!()
    }

    fn get_metadata(
        &mut self,
    ) -> std::pin::Pin<
        Box<
            dyn std::future::Future<
                    Output = parquet::errors::Result<Arc<parquet::file::metadata::ParquetMetaData>>,
                > + Send
                + '_,
        >,
    > {
        todo!()
    }
}

fn test<'r, T: AsyncFileReader + Send + Sync>(
    file: ArrowReader<'r, T>,
) -> impl AsyncFileReader + 'r {
    Box::new(file) as Box<dyn AsyncFileReader>
}

test function would not be passed using current implementers, after merging this pr, it will

ethe commented 2 days ago

In my understanding, we should start from the simple one without lifetime

Actually it does have one lifetime in the current: 'static, lifetime is just elided rather than non-existent.

If you are still confused, there is a basic explanation about it: https://quinedot.github.io/rust-learning/dyn-elision-basic.html#impl-headers

Consider using implementations like the following if possible, as they are more general:


// Implemented for all lifetimes
impl Two for Box<dyn Trait + '_> {}

// Implemented for all lifetimes such that the inner lifetime is // at least as long as the outer lifetime impl Two for &(dyn Trait + '_) {}