Closed mihlema closed 1 year ago
Just to be 100% clear here, should we try and keep the archetypes as "minimal" as possible, meaning that they area meant for importing on top of existing DB:s and each other to serve as "templates" for doing different things in SpineOpt?
E.g. that the "archetype" for a deterministic
model stochastic structure would essentially just consist of a pre-made stochastic_structure
, stochastic_scenario
, stochastic_structure__stochastic_scenario
relationships (and maybe some parameters I'm forgetting)? This could then be imported on top of the SpineOpt template to provide a ready-made stochastic structure, that would then have to be connected to the desired parts of the model. (I guess I'd have to explain somewhere how the archetype stochastic structure is then actually connected to the rest of the model, but that would be in the documentation I guess).
Or is the purpose to be more like "tutorials", where I'd also provide ready-made model
, unit
, node
, or something to illustrate how the stochastic_structure
is connected to the rest of the model via the structural relationship classes?
My understanding is that they would be minimal and allow importation on top of an existing dataset.
I can see the value of the second option, however, this feels more like a sample system?
My understanding is that they would be minimal and allow importation on top of an existing dataset.
I prefer the "minimal" type myself, don't get me wrong. I just wanted to make sure we're trying to achieve the same thing. I'm not sure where we can easily include instructions for how to connect the archetype to the rest of the system, but I guess we'll figure something out.
I should be able to whip up archetypes for e.g. a deterministic
, branching
and converging
stochastic structures easily enough. I'd probably make them according to the example instructions in the Stochastic Framework
section of the documentation, so that I could point to them from there or the other way around.
FYI, it seems that due to recent changes to parameter value types, exporting and importing special parameter values from .json isn't working properly. I tried making example archetypes for a couple different stochastic structures, but importing them doesn't work properly atm due to special parameter types like Duration
. I've let @manuelma know about the issue, and he said he'll work on it alongside the other type-related stuff he's doing.
I'll try and let you guys know when the .json parameter values are working as intended.
Ok, @manuelma fixed the above issue in the .json export/import in spinedb_api
, so I've got my first working drafts for the stochastic structure "archetypes" in the WIP pull request #386. Hopefully, they can serve as a basis for further discussion.
It seems @manuelma already made a sort of "basic model" template for a workshop at KTH basic_spineopt_template.json
, so I'm guessing we can use that for the
Basic model with 1 temporal block, 1 stochastic scenario
part? Could maybe use some cleanup and documenting, but other than that, I think that's more or less what was expected? A basic template on which users can immediately start adding units
, connections
, nodes
, etc. without having to deal with the temporal and stochastic stuff.
I think we need to agree where we want to store the archetypes. Currently, the SpineOpt.jl root folder has the following sub-folders:
\data - Houses the `spineopt_template.json` (and @manuelma `basic_spineopt_template.json`)
\docs - Houses the docs environment and the source code for the documentation.
\examples - Houses some old `test_systemA5.jl` files.
\data - Houses the input data .sqlite for the `test_systemA4.jl`
\src - Contains the source code for the SpineOpt.jl module.
\test - Contains the unit testing environment and code.
I think the root \data
folder is unnecessary, as it currently only (really) houses the spineopt_template.json
. Also, the name is a bit misleading in my opinion, as it's not really about "data" but more about "data structure". I like the \examples
folder, as it's immediately obvious what it contains, and its potentially really helpful for new users to see. However, I think we could structure things a little nicer, e.g.
\examples - We could include e.g. the `basic_spineopt_template.json` directly under here.
\models - This is currently only for the `test_systemA5` stuff, if we want to preserve it. We can always include more example models here when needed, e.g. my stochastic test system?
\(potential sub-folders if needed)
\archetypes - This would be where the archetype .json files would live. Essentially a "library" for pre-made model pieces.
\(are sub-folders needed?)
I'm still uncertain what would be the best place for the spineopt_template.json
, as it's not really an "example". In my opinion, it's integral enough to SpineOpt to be included under \src
if necessary, but I think it's also even more important in the sense that it needs to be referred to by the users more often than the source code.
I would maybe just include spineopt_template.json
in the root folder for maximum visibility. Maybe even rename to something more descriptive, e.g. spineopt_data_structure.json
? Any thoughts, @mihlema, @manuelma, @DillonJ ?
I believe the best is to rename data
to templates
and host all the templates (aka archetypes) there. Typically, examples
is reserved to .jl scripts that demonstrate part of the code, and src
is reserved to .jl files too. It would be weird to have a JSON in any of these folders in my opinion.
I also don't like having spineopt_template.jl
in the root, but that's also just my opinion.
Ok, I've made the example archetypes for the stochastic structures, as well as documented them under a new section in the documentation in the pull request #386. I also reorganized the folder structure according to my proposition before, but that can be changed at a later date as desired. Check it out and comment if you have opinions.
The merge request should be good to go, if you think we should go with my proposed approach. (Although the unit tests don't pass due to some changed @mihlema made to the master
a couple of days ago?)
On a side note: if we're aiming to rack up a lot of archetypes eventually, it might be worth it to auto-generate the documentation similar to the concept reference. However, I don't think it's necessary at this stage.
I believe the best is to rename
data
totemplates
and host all the templates (aka archetypes) there. Typically,examples
is reserved to .jl scripts that demonstrate part of the code, andsrc
is reserved to .jl files too. It would be weird to have a JSON in any of these folders in my opinion.I also don't like having
spineopt_template.jl
in the root, but that's also just my opinion.
I feel like we should definitely keep at least the spineopt_template.jl
separate from the other archetypes, as it's way more fundamental for SpineOpt.jl (e.g. used for auto-generation of the documentation). Also, it doesn't really contain any "working example", just the basic structure.
I merged my stochastic archetype stuff into master
and tried to organise the files according to our discussion last Wednesday. I feel it's better to include it now than to wait until it gets forgotten.
Regardless, we can always shuffle the files around later if we feel like the file structure isn't ideal.
Closing due to stale status.
Closing because stale.
As discussed before, our initial step towards supporting archetypes will be to provide
.json
templates that can be imported with all necessary parameter relationships defined. This issue serves as a list of envisioned archetypes and responsibilities. Note that archetypes will also need to be documented in the online documentation #280, in general (how to use) and very briefly specific (what is each archetype for). Note that I have also added "model" archetypes (in contrast to "system" archetypes, which typically correspond to the physical system)General description on how to use archetypes @mihlema(I feel like this should be mentioned in the relevant parts of the documentation; e.g. description of pressure-driven gas transfer)System archetypes:
Model archetypes, t.b.d.:
I'll make an initial example for one of "my" archetypes, then others can swiftly follow.