softdevteam / vob

Vector of Bits
Other
15 stars 3 forks source link

a set() method that resizes the Vob on demand #60

Closed eadf closed 3 years ago

eadf commented 3 years ago

I got a problem where I need to keep track of a unknown, and ever growing, number of bits, so I need a way to (silently) grow the Vob when setting bits.

Something like this:

fn set_grow__or_another_name(&mut self, index: usize, value: bool) -> bool {
  if self.capacity() <= index {
    self.reserve(index-self.len()+1); // details need refinement, maybe max(50,+15% of current size)? 
  }
  self.set(index, value) // <- does not work as I assumed, does not fill in 'missing' bits
}

Or similar to how num-bigint::BigUInt.set_bit() does things (not fond of their automatic shrinking though. My problem always generates more bits, so a reduction is a total waste of time).

I know that re-allocating the data-store is expensive, but sometimes there is no way around it.

Could a method like that be something you would like to add to Vob?

thomwiggers commented 3 years ago

A note on reserve(): the underlying storage method, Vec, already doubles the capacity each time it reallocates. If there's enough capacity, it's a no-op. This function could probably be simplified to just first calling reserve() and then set().

Because Vob doesn't do a sparse storage, I wonder if having this kind of function might cause unexpected large consumption of memory if people start doing myvob.set_grow_or_whatever(100000, true). Meanwhile writing the wrapper is trivial...

eadf commented 3 years ago

Because Vob doesn't do a sparse storage, I wonder if having this kind of function might cause unexpected large consumption of memory if people start doing myvob.set_grow_or_whatever(100000, true).

Ppl do stupid things all the time (me included), they could just as well do myvob.reserve(100000) with the same result.

Meanwhile writing the wrapper is trivial...

Update: My naive set_grow__or_another_name() example does not work, set() does not fill in the missing 'in-between' bits. So writing an efficient wrapper will be a tad bit more complicated than just two lines of code.

ltratt commented 3 years ago

I can see both POVs. As @thomwiggers says, Vobs are fundamentally not sparse, so any suggestion of a "set" might cause people to treat them as a set, when they're not (and then memory will go bananas). But, that said, I can see that the general use case here has some merit -- but is it not mostly covered by Vob::resize?

eadf commented 3 years ago

Oh, I did not suggest a sparse set. Just that a variant of the method set() that could grow the container.

but is it not mostly covered by Vob::resize?

Oh, missed that. Thanks! Indeed it solves the whole issue.

// public domain
pub trait GrowingVob {
    fn fill(initial_size: usize) -> vob::Vob;
    fn set_grow(&mut self, bit: usize, state: bool) -> bool;
}

impl GrowingVob for vob::Vob {
    #[inline]
    fn fill(initial_size: usize) -> Self {
        let mut v = vob::Vob::new_with_storage_type(0);
        v.resize(initial_size, false);
        v
    }

    #[inline]
    fn set_grow(&mut self, bit: usize, state: bool) -> bool {
        if bit >= self.len() {
            // if the actual size of the Vob isn't critical, you can set this to bit + 64 or higher
            self.resize(bit + 1, false); 
        }
        self.set(bit, state)
    }
}

Feel free to include this as an optional, semi-dangerous, trait in Vob if you like.

Edit: added a creature-comfort method: fill(), generating a pre-filled Vob