ipfs-rust / ipfs-embed

A small embeddable ipfs implementation
352 stars 48 forks source link

Transactional database ops #82

Closed rklaehn closed 3 years ago

rklaehn commented 3 years ago

Currently, we don't have a way to do transactional database opts. Which severely limits performance when interacting with stores that have paranoid transactional guarantees, such as sqlite in synchronous = normal or synchronous = full mode. But I do like paranoid transactional guarantees.

The ipfs sqlite block store does support transactions as of some time ago, so we should expose that in the public interface of ipfs-embed.

rklaehn commented 3 years ago

This excerpt from the actyx code base shows the necessity for this: We are doing something in n>=5 write transactions which could be done in 1. This is needlessly inefficient even in synchronous = off, and extremely inefficient in synchronous = normal or synchronous = full mode.

                            match store.ipfs().create_temp_pin() {
                                Ok(tmp) => {
                                    tracing::trace!("temp pin created");
                                    if let Err(err) = store.ipfs().temp_pin(&tmp, &root_update.root) {
                                        tracing::error!("{}", err);
                                    }
                                    tracing::trace!("temp pinned");
                                    for block in &root_update.blocks {
                                        if let Err(err) = store.ipfs().insert(block) {
                                            tracing::error!("{}", err);
                                        } else {
                                            tracing::trace!("{} written", display(**block));
                                        }
                                    }
                                    let source = if root_update.blocks.is_empty() {
                                        RootSource::SlowPath
                                    } else {
                                        RootSource::FastPath
                                    };
                                    match Link::try_from(root_update.root) {
                                        Ok(root) => store.update_root(root_update.stream, root, source),
                                        Err(err) => tracing::error!("failed to parse link {}", err),
                                    }
                                }
                                Err(err) => {
                                    tracing::error!("failed to create temp pin {}", err);
                                }
                            }
dvc94ch commented 3 years ago

Fixed in #86