ipfs-rust / ipfs-embed

A small embeddable ipfs implementation
352 stars 48 forks source link

ipfs-embed should cleanup its resources #64

Closed wngr closed 3 years ago

wngr commented 3 years ago

Seems ipfs-embed is spawning some background resources, which are not dropped if the main Ipfs handle is dropped.

Reproducer:

use std::time::Duration;

use ipfs_embed::{Config, Ipfs};
use libipld::{store::StoreParams, IpldCodec};

#[derive(Debug, Clone)]
struct Sp;

impl StoreParams for Sp {
    type Hashes = libipld::multihash::Code;
    type Codecs = IpldCodec;
    const MAX_BLOCK_SIZE: usize = 1024 * 1024 * 4;
}

#[async_std::main]
async fn main() -> anyhow::Result<()> {
    for _ in 1..256 {
        let config = Config::new(None, 1024 * 1024);
        let ipfs = Ipfs::<Sp>::new(config).await?;

        drop(ipfs)
    }

    std::thread::sleep(Duration::from_secs(300_000_000));

    Ok(())
}

One thing I noticed is that every time a new blocking-<nr> thread with the postfixed integer incremented is spawned (up to 500):

➜  ~ ps -e -T | grep 168268 | wc -l
270

There might be other resources that have not been properly released; for example I have seen an error message with concurrent access to the sqlite block store at one time.

Would be nice to have a future to release everything, which can be awaited. Another option would probably be to let users provide a custom executor, so that the runtime can be cleaned up by the user.