spine-tools / Spine-Toolbox

Spine Toolbox is an open source Python package to manage data, scenarios and workflows for modelling and simulation. You can have your local workflow, but work as a team through version control and SQL databases.
https://www.tools-for-energy-system-modelling.org/
GNU Lesser General Public License v3.0
73 stars 17 forks source link

Update Importing and exporting data section in User Guide #306

Closed spine-o-bot closed 3 years ago

spine-o-bot commented 3 years ago

In GitLab by @PekkaSavolainen on Sep 4, 2019, 19:02

@pvper, @soininen, @manuelma Anybody want to update this section?

spine-o-bot commented 3 years ago

In GitLab by @manuelma on Sep 4, 2019, 20:40

Maybe we can work together on it. Say, I start and then @pvper checks if what I did is fine?

There are many use cases, especially when importing data, so I suggest we stick to the most simple ones for v0.3 and call it a first iteration...?

spine-o-bot commented 3 years ago

In GitLab by @soininen on Sep 5, 2019, 10:17

@manuelma Sounds like a plan! I don't think I know enough of the data import interfaces to write any documentation for them so if you don't mind I'll assign this issue to you.

spine-o-bot commented 3 years ago

In GitLab by @soininen on Sep 5, 2019, 10:17

assigned to @manuelma and unassigned @soininen

spine-o-bot commented 3 years ago

In GitLab by @PekkaSavolainen on Sep 5, 2019, 10:19

What I meant by this issue is that, just quickly go through the text in data_import_export.rst and remove everything that does not exist anymore. It's ok that something is not documented at this point but it's not good if the documentation talks about stuff that does not exist.

You can write the real tutorial/documentation later.

spine-o-bot commented 3 years ago

In GitLab by @PekkaSavolainen on Sep 5, 2019, 10:22

I'd like to get the 0.3 release out tomorrow so the first iteration should be what I wrote in previous comment.

spine-o-bot commented 3 years ago

In GitLab by @manuelma on Sep 5, 2019, 22:55

Will do.

spine-o-bot commented 3 years ago

In GitLab by @manuelma on Sep 5, 2019, 23:08

Looking at this, I realized that the whole datapackage editor thing is pretty outdated. We kinda took a very different direction for data import, with the Data Interface item and the mapping support from spinedb_api. It's harder for the datapackage editor to find it's place in this new setup. For this reason, I will just drop the section in the documentation that talks about the datapackage editor. I'd also suggest to go as far as to remove the Datapackage button from the DC properties for now, until we decide what to do. I remember @ererkka had good ideas about how to use the datapackage format to inject metadata to a DB, and that could be a good use-case.

spine-o-bot commented 3 years ago

In GitLab by @manuelma on Sep 5, 2019, 23:09

mentioned in commit 055b2cfe354b941acfc80928b7763a1d51376506

spine-o-bot commented 3 years ago

In GitLab by @manuelma on Sep 5, 2019, 23:10

@pvper can you please check if the Excel section in data_import_export.rst is up to date? I don't remember what we're doing with the time series...

spine-o-bot commented 3 years ago

In GitLab by @PekkaSavolainen on Sep 6, 2019, 10:36

The datapackage editor should be moved from the Data Connection to the Data Interface item.

spine-o-bot commented 3 years ago

In GitLab by @PekkaSavolainen on Sep 6, 2019, 11:02

But the datapackage editor is there and it has not been updated in a while, so the documentation in Datapackage section in data_import_export.rst should be valid? What if we just take the Datapackage stuff that was removed in commit 055b2cf and add it to spine_datapackage_editor.rst. And add a notice on top of the section that says This section is a work in progress or something.

Or is the Datapackage section so out-dated that we should just remove it?

spine-o-bot commented 3 years ago

In GitLab by @manuelma on Sep 6, 2019, 15:11

The documentation talks about how to use the Datapackage editor to import data from CSV into a Spine Database. That should still work, but I don't think we want users to do that anymore, we want them to use the Data Interface item for that. So the Datapackage editor is almost pointless at this point, that's why I suggested to remove it and allow us some time to take a decision on what to do with it. For example we can use it to visualize datapackages, edit data and metadata, and stuff like that.

spine-o-bot commented 3 years ago

In GitLab by @manuelma on Sep 6, 2019, 15:37

mentioned in commit 4a61dce299fa44516e9824d2dcc0f515959edd1d

spine-o-bot commented 3 years ago

In GitLab by @PekkaSavolainen on Sep 6, 2019, 15:42

Fine.

spine-o-bot commented 3 years ago

In GitLab by @PekkaSavolainen on Sep 6, 2019, 15:42

closed