search

tecosaur / DataToolkit.jl

Reproducible, flexible, and convenient data management
https://tecosaur.github.io/DataToolkit.jl
87 stars 4 forks source link

Adding a dataset programmatically prompts for input #23

Open jfb-h opened 7 months ago

jfb-h commented 7 months ago

Adding a dataset via 

DataToolkitBase.add(DataToolkit.Dataset, name, Dict{String, Any}("description" => description), source)

still prompts for user input on description and attributes, which is probably not intended?

Btw., is spec::Dict{String, Any} the right place to put description info?

tecosaur commented 7 months ago

Yea, the DataToolkitBase "julia API" probably needs a bit more love that I've been able to give it so far.

It should definitely be able to operate noninteractively, and to answer your question spec is indeed the right place to put a description.

tecosaur commented 5 months ago

On this topic, if anybody has ideas for a nice Julia API and a willingness to dig into the codebase, I'd love to hear them.

tecosaur commented 3 months ago

I'm currently overhauling this, so interactive input is gated behind the (new) REPL package.

Here's what the WIP-API looks like:

collection = DataCollection()
dataset = create!(collection, DataSet, "name", "description" => desc, ...)
create!(dataset, DataStorage, :filesystem, "path" => "path/to/file.csv")
create!(dataset, DataLoader, :csv)

I'm wondering if for convenience it might also be worth having

collection = DataCollection()
dataset!(collection, "name", "description" => desc, ...)
storage!(dataset, :filesystem, "path" => "path/to/file.csv")
loader!(dataset, :csv)