p2panda / aquadoggo

Node for the p2panda network handling validation, storage, aggregation and replication
GNU Affero General Public License v3.0
69 stars 5 forks source link

`migrate` API to allow bootstrapping schema data on launch of node #597

Closed adzialocha closed 9 months ago

adzialocha commented 9 months ago
use aquadoggo::{Configuration, Node};
use p2panda_rs::identity::KeyPair;

let config = Configuration::default();
let key_pair = KeyPair::new();
let node = Node::start(key_pair, config).await;

let data = include_str!("schema.lock");
let lock_file: LockFile = toml::from_str(&data).expect("error parsing schema.lock file");
node.migrate(&lock_file).await;

Related discussion: https://github.com/p2panda/fishy/pull/11

adzialocha commented 9 months ago

Another idea could be to add the schema file into the Configuration struct:

use aquadoggo::{Configuration, Node};
use p2panda_rs::identity::KeyPair;

let config = Configuration {
  lock_file_path: "...",
  ...Default::default(),
};

let key_pair = KeyPair::new();
let node = Node::start(key_pair, config).await;

Then start is ready after migration took place which is a nice feature to have.

adzialocha commented 9 months ago

There's basically two usecases here:

  1. Using "low-level" migrate method: Nice for embedding and compiling aquadoggo manually into your software, it's up to you if you want to include the string into your code or load it from a file etc.
  2. Using Configuration: Gives a nice API for the cli, env vars and config.toml file for cases where you don't want to compile aquadoggo yourself (server setups, experiments, etc.)
sandreae commented 9 months ago

Really nice functionality to have access to programmatically, there's already a demand for it too.

With regards to the two use cases you outline above, both are very valid. I'm wondering if they could be combined in some way into a unified API which serves both cases. Basically the API from version 2) but under the hood we're using include_str! for those who want to compile. It means the lock file will increase binary size (maybe unexpectedly), but only if included, you can still opt to not include it and use fishy.

If we would rather pick one, then I'd lean towards 1) as we currently don't have any solution/tool for migrating data onto an embedded-compiled node. For cases covered by 2) I think it's normally possible to use fishy in any case.

I'm working on an embedded node in a tauri app, and for this I definitely need 1) 😄.

adzialocha commented 9 months ago

Good point, we already have fishy for that! Let's go for the embed case then