meilisearch / heed

A fully typed LMDB wrapper with minimum overhead 🐦
https://docs.rs/heed
MIT License
606 stars 55 forks source link

Is it possible to get range working with slice types? #129

Open Palmik opened 2 years ago

Palmik commented 2 years ago

All the slice types define EItem = [T], since [T] is not Sized, RangeFrom<&[T]> (and others) do not implement RangeBounds. Is there some workaround that I am missing?

My use case is a simple key like [u8; N] or Vec<u8>, etc.

curquiza commented 2 years ago

Hello @Palmik I think @Kerollmops or @irevoire can answer this! They just went on Holiday, so sorry in advance for the delay in answering 🙏

Kerollmops commented 2 years ago

Hey @Palmik,

I understand that it is pretty complex to use, and we would like to rewrite a big part of heed and the BytesEncode/BytesDecode traits. In the meantime, you should be able to give a reference of your range type to make it work.

db.range(&([0, 1, 2]..[2, 3, 4]))?;

db.range(&(&vec1[..]..&vec2[..]))?;
mikedilger commented 11 months ago

I want a delete_prefix, but as there is none I'm trying to use delete_range() but have encountered this issue. This is on 0.20.0-alpha6. Here's a snippet:

// This is the prefix I want to delete
let start_key_slice: &[u8] = id.as_slice();

// Here I make a copy and increment the last byte, so I can make a range
let end_key = id;
end_key.0[end_key.len() - 1] += 1;
let end_key_slice: &[u8] = end_key.as_slice();

// Here I use a let binding to make an owned range so I can reference it
let range = start_key_slice .. end_key_slice;

// And this line gives the error
self.db_event_seen_on_relay()?.delete_range(txn, &range)?;

The error is:

error[E0277]: the size for values of type `[u8]` cannot be known at compilation time
    --> gossip-lib/src/storage/mod.rs:1231:62
     |
1231 |             self.db_event_seen_on_relay()?.delete_range(txn, &range)?;
     |                                            ------------      ^^^^^^ doesn't have a size known at compile-time
     |                                            |
     |                                            required by a bound introduced by this call
     |
     = help: the trait `Sized` is not implemented for `[u8]`
     = help: the following other types implement trait `RangeBounds<T>`:
               std::ops::Range<&T>
               std::ops::Range<T>
     = note: required for `std::ops::Range<&[u8]>` to implement `RangeBounds<[u8]>`
note: required by a bound in `heed::Database::<KC, DC, C>::delete_range`
    --> /home/mike/.cargo/git/checkouts/heed-cddcbc156c1cdd8e/8bfdf3b/heed/src/database.rs:2092:12
     |
2089 |     pub fn delete_range<'a, 'txn, R>(&self, txn: &'txn mut RwTxn, range: &'a R) -> Result<usize>
     |            ------------ required by a bound in this associated function
...
2092 |         R: RangeBounds<KC::EItem>,
     |            ^^^^^^^^^^^^^^^^^^^^^^ required by this bound in `Database::<KC, DC, C>::delete_range`

For more information about this error, try `rustc --explain E0277`.
error: could not compile `gossip-lib` (lib) due to previous error
Kerollmops commented 10 months ago

Hey @mikedilger 👋

I'm sorry for not getting back to you sooner. Here is a possibly better working version in which you use references for the slices in the range:

// This is the prefix I want to delete
let start_key_slice: &[u8] = id.as_slice();

// Here I make a copy and increment the last byte, so I can make a range
let end_key = id;
end_key.0[end_key.len() - 1] += 1;
let end_key_slice: &[u8] = end_key.as_slice();

// Here I use a let binding to make an owned range so I can reference it
let range = (&start_key_slice) .. (&end_key_slice);

// And this line gives the error
self.db_event_seen_on_relay()?.delete_range(txn, &range)?;
xyzshantaram commented 6 months ago

Hi all, I'm trying to implement the solution described above and I'm running into the same issue.

Here's what I have:

let start_key_slice: &[u8] = req.start.as_slice();
            let end_key_slice: &[u8] = req.end.as_slice();
            let range = (&start_key_slice)..(&end_key_slice);
            for entry in self
                .db
                .range(&txn, &range)
                .map_err(|e| Error::msg(e.to_string()))?
            {}

And here's the error:

error[E0277]: the trait bound `std::vec::Vec<u8>: heed::BytesDecode<'_>` is not satisfied
  --> src/lib.rs:61:26
   |
61 |               for entry in self
   |  __________________________^
62 | |                 .db
63 | |                 .range(&txn, &range)
64 | |                 .map_err(|e| Error::msg(e.to_string()))?
   | |________________________________________________________^ the trait `heed::BytesDecode<'_>` is not implemented for `std::vec::Vec<u8>`
   |
   = help: the following other types implement trait `heed::BytesDecode<'a>`:
             <heed::heed_types::CowSlice<T> as heed::BytesDecode<'a>>
             <heed::heed_types::CowType<T> as heed::BytesDecode<'a>>
             <heed::heed_types::OwnedSlice<T> as heed::BytesDecode<'a>>
             <heed::heed_types::OwnedType<T> as heed::BytesDecode<'a>>
             <heed::heed_types::Str as heed::BytesDecode<'a>>
             <heed::heed_types::UnalignedSlice<T> as heed::BytesDecode<'a>>
             <heed::heed_types::UnalignedType<T> as heed::BytesDecode<'a>>
             <heed::heed_types::Unit as heed::BytesDecode<'_>>
           and 4 others
   = note: required for `heed::RoRange<'_, std::vec::Vec<u8>, denokv_proto::KvValue>` to implement `std::iter::Iterator`
   = note: required for `heed::RoRange<'_, std::vec::Vec<u8>, denokv_proto::KvValue>` to implement `std::iter::IntoIterator`

I'm sure I'm missing something really basic but I'm not super sure what... any help would be appreciated. req.start and req.end are both Vec.

mikedilger commented 6 months ago

I'm using (and it works):

let range = ( Bound::Included(&*start_prefix), Bound::Excluded(&*end_prefix) );