Data representation temporal structure

[spine-o-bot]

In GitLab by @Poncelet on Jan 30, 2019, 10:41

This issue aims to describe the data representation of the temporal structure.

This issue is related to issue #17 , but in this issue only the representation of the temporal structure in the data is discussed.

This issue is mainly aimed to discuss Manuel's current proposal (and follow-up proposals), but also discussions regarding interfaces between that data structure and other representations (e.g., temporal blocks, spatio-temporal blocks) could be discussed here.

[spine-o-bot]

In GitLab by @manuelma on Jan 30, 2019, 12:19

From #17:

However, how would we see time_slice_groups working out if time_slices are not objects?

Maybe we can create temporal_level_group, which would essentially be like grouping time slices (remember, the time_step parameter holds time steps for a temporal level, so everything we do with temporal levels can translate into time steps 'effortlessly')

[spine-o-bot]

In GitLab by @Poncelet on Jan 30, 2019, 12:29

Maybe we can create temporal_level_group, which would essentially be like grouping time slices (remember, the time_step parameter holds time steps for a temporal level, so everything we do with temporal levels can translate into time steps 'effortlessly')

But if I understand correctly, then we would only be able to group all time slices of a specific temporal level with all time slices of another temporal level?

[spine-o-bot]

In GitLab by @manuelma on Jan 30, 2019, 12:30

But if I understand correctly, then we would only be able to group all time slices of a specific temporal level with all time slices of another temporal level?

Unfortunately, yes. If you want finer granularity you need to define more temporal levels holding only the time steps you want to group.

[spine-o-bot]

In GitLab by @Poncelet on Jan 30, 2019, 12:38

Unfortunately, yes. If you want finer granularity you need to define more temporal levels holding only the time steps you want to group.

Yes, we need finer granularity. And then a temporal level is no longer a temporal level but a temporal block (as there are now sequential temporal blocks) - something in addition to the temporal level.

And if a temporal block becomes very small, it becomes a time step. Then we have arrived in a situation where a time step is an object...

[spine-o-bot]

In GitLab by @manuelma on Jan 30, 2019, 12:45

Then we have arrived in a situation where a time step is an object...

So this is nice, right? A temporal block can have a single time step if needed. But that doesn't mean that all time steps should be represented as a temporal block (I mean, an individual object) in my opinion.

Also, it looks like we're gonna need these 'sequential' temporal blocks within the same temporal level, what do you think?

[spine-o-bot]

In GitLab by @Poncelet on Jan 30, 2019, 13:02

I'm not sure whether I think it is nice or not at this point...

What I think that the time_slice_group thing has shown is that not having time slices as objects might make the data representation of the data structure less robust. We had not thought about time_slice_groups before, and suddenly we need to change our data design...

Just to get an overview, what we are now discussing would mean we would need:

• temporal levels, each having a number of time slices (parameter of the temporal level object)
• temporal blocks: object containing a group of time slices within the same temporal level
• time_slice_group: an object that groups the time slices corresponding to different temporal blocks.

By having time slices as objects, we would be able to remove the temporal block (there would be no difference between a temporal block and a time_slice_group anymore).

The process of creating a time_slice_group also is more complex now (first need to create temporal_blocks, than time_slice_group). Similarly, the unpacking of which time slices correspond to a time_slice_group object will become more complex.

And what is not clear to me yet is how will we specify which time slices correspond to a temporal block?

[spine-o-bot]

In GitLab by @manuelma on Jan 30, 2019, 13:25

I still think there's a huge benefit from not storing every single time step as an individual object. In case we need to isolate some time steps, eg, to create an arbitrary group, we can still do it using temporal blocks having a single time step. Then of course we need to specify which temporal block comes after each other, but can you imagine doing this with individual time slices? We would need a relationship between time slice 1 and 2, 2 and 3, and so on until time slice 99999 and 100000!

[spine-o-bot]

In GitLab by @Poncelet on Jan 30, 2019, 13:42

I'm not advocating for having time slices as individual objects (at least, not yet). I'm just trying to keep an open mind and get an overview of the implications doing it as well as not doing it.

When we do not have time slices as objects, there are still some issues which need to be overcome to enable the capabilities we want in the model.

In contrast, when having time slices as objects, it is straightforward to enable everything we want to do. The only concerns I have noticed in the discussion relate to convenience (and possibly database size).

At this point, I'm not sure what would yield the better final solution: stretching the data representation without time slices as objects to enable everything we want (and hoping that it then also enables things we might want in a later stage), or trying to find solutions to make it more convenient to have time slices as objects.

We would need a relationship between time slice 1 and 2, 2 and 3, and so on until time slice 99999 and 100000!

I have a question related to this remark (and a similar one earlier from @juha_k ). The representation of time in the database is something which only relates to Spine Model. Spine Model is an optimization model, likely to be used predominantly for modeling energy systems. In that context, I am not aware of models anywhere near having so many time steps. I do understand that time series data can be quite long. So here the question: do we need to have time slice objects for all time steps spanned by our data, or only what is used in the optimization?

[spine-o-bot]

In GitLab by @jkiviluo on Jan 31, 2019, 08:06

On the temporal_block vs. time_slice_group. I think the time_slice_group is redundant. Temporal_block could define things in relation to the fundamental time interval of the model. E.g. the model has a fundamental time interval of 5 minutes. Then we can define a temporal_block 1h_from_h7_to_h24 with following parameters:

• first_time_step: 12 * 7 + 1
• time_steps: [Symbol(\"1h\", i) for i in 7:24]
• duration: [12 12 ... 12]

We need to establish a convention whether we have t0000 or t0001 as the first time step. Above it assumes that time starts from t0001. However, t0000 can be useful for initial values like storage content and online status.

And when one needs to establish vertical weights for e.g. investment model, I would do it through temporal_block relationships:

• 1year__winter --> weight: 1
• winter__wi_weekend --> weight: 2
• winter__wi_weekday --> weight: 5
• wi_weekday__wi_wd_daytime --> weight: 12
• wi_weekday__wi_wd_nighttime --> weight: 12

This defines a vertical structure only when it's needed and I think it's actually a more correct place for the weight, since it tells the weight in relation to the upper level. It will also mean that the vertical structure can be directly shown in our Graph view.

Temporal_level is not really needed either. The less terms referring to similar things we have, the easier it will be to comprehend. Some terms are of course necessary, but I don't currently find temporal_level to be such.

[spine-o-bot]

In GitLab by @manuelma on Jan 31, 2019, 12:40

Temporal_block could define things in relation to the fundamental time interval of the model

Do we want to have this fundamental time interval thing? To me it seems unnecessary. Temporal blocks would define time steps (just a set of names, e.g., :h1, ..., :h24) with their corresponding duration, nothing more.

We need to establish a convention whether we have t0000 or t0001 as the first time step.

t0 for holding initial conditions seems good.

And when one needs to establish vertical weights for e.g. investment model, I would do it through temporal_block relationships

Seems good. In my proposal wiki I put the weight on the temporal block object, rather than in the relationship, but it can go there as well.

[spine-o-bot]

In GitLab by @Poncelet on Jan 31, 2019, 15:45

After today's call with @juha_k and @manuelma : have agreed on the data representation and model representation of the temporal structure as described in these slides -> Data_representation_temporal_structure.

Jump_all_out would not require any new functionality.

Based on the temporal_block objects, and the optional parent_temporal_block__child_temporal_block relationships, a dedicated script within Spine model will generate the set of time slices and the sets containing the tuples of time slices required for the Spine model (variable generation, equations).

Planned next steps include:

• developing the scripts generating the required sets
• set-up a test case (probably test case 2 with varying compositions of temporal blocks)

[spine-o-bot]

In GitLab by @Poncelet on May 9, 2019, 13:38

Seems to be working nicely.

Currently open issues are:

• how to represent looping structure in the data (see Issue https://gitlab.vtt.fi/spine/data/issues/44 )
• how to determine when to have investment variables

The last one will likely get its own issue at some point. Closing this issue now

[spine-o-bot]

In GitLab by @Poncelet on May 9, 2019, 13:38

closed