Include ability to handle subscripts

[pbreach]

Need to be able to:

• Get and set entire subscripted variables as well as subranges
• Ability to pass Python functions for entire subscripted variables and subranges

Modifying subscripted model components can be dealt with by adding model component as function which will get values from the subscripted model component.

Setting model components can be dealt with by a including ability to pass multidimensional lists in VenPy.__setitem__() (maybe numpy support would be a good idea to have in this case). Will need a way to distinguish between setting look-up tables and subscripted variables.

values will likely need to be obtain using the Vensim DLL command, vensim_get_vecval.

[pbreach]

Seems like for setting subscripts only 2 subranges can be specified at a time, as is the case in Vensim. Also, will need to find a way to ensure the passed Python list has the correct number of dimensions for the Vensim model variable.

Handling of passed Python functions will be no different than previous syntax. It looks like this will be fairly straightforward to implement.

[pbreach]

Setting and getting should support up to 8 dimensions (Vensim limit). A mix of subranges and subrange elements should be able to be specified.

Two subscript ranges can be set at one time using the SIMULATE>SETVAL vensim command, and only one fully subscripted value can be retrieved at a time with the Vensim DLL (subranges won't work for getting values).

In the case of setting, this means that behind-the-scenes many SIMULATE>SETVAL commands will need to happen in the case where a list with more than two dimension is being passed.

For getting, a work-around to this could be to get all subscript combinations for the subscript ranges being set via vensim_get_varattrib. These combinations can be retrieved by getting all offsets via vensim_get_varoff, and then passing them to vensim_get_vecvals. The result will need to be reshaped to the appropriate dimensions of the subscripted variable. Implementing this for the general case could be a little bit trickier than originally thought.

[tonyvsuk]

I'm not sure what you are trying to do here.

SIMULATE>SETVAL is really for setting individual constants/gaming variables (and it can also be used to set any parameter no matter how many subscripts it has, it's not limited to two). There is no mechanism to set an entire array which is what I think you are trying to do. Anyone with that amount of data will usually store it in Excel/database/CIN file/other and import it, calling SIMULATE>SETVAL hundreds of times would be too inefficient.

vensim_get_vecvals is no use here. It is used to retrieve values from a run, not to be able to set values.

Hope this helps.

[pbreach]

@tonyvsuk You're right it will probably be slow if there are many subscript ranges, especially more than 2. I thought SIMULATE>SETVAL was limited to setting two subranges per call?

For example, say range1 and range2 represent subscript ranges and elem1 is a an element of a range3. I have a variable named constant[range1,range2,range3]. How would you set all three subranges? My take would be to use something like SIMULATE>SETVAL|"constant[range1,range2,elem1]=1,2,3; 4,5,6; 7,8,9;" and that this would have to be repeated for every element in range3. Is there a better way?

vensim_get_vecvals is to get the subscripted variable values either before or during a GAME simulation.

the idea is to have a general way to setting and getting subscripted values.

This will allow for values to be set prior to simulation, but more importantly, will be able to 'replace' Vensim equations (only GAME type variables) with Python functions. This will get the values into Python, pass to any function(s) and set the result during a GAME simulation after a specified amount of time steps. It's not particularly fast with large amounts of subscripts but I've seen it done before.

[tonyvsuk]

SETVAL does not work in the way you are trying to implement. It will only set one variable at a time.

The following will not work (or at least, I've never tried it, and would not recommend using it). SIMULATE>SETVAL|"constant[range1,range2,elem1]=1,2,3; 4,5,6; 7,8,9;

You would need to call it nine times for each of the values 1,2,3; 4,5,6; 7,8,9;

Forget vensim_get_vecvals. You cannot use it for setting any values, it is for retrieval of values only. All this does is allow you to get values during simulation. Useful if you are running a simulation and want to update a GUI at each time step, you could use this to get the x/y values for an animated graph.

To set gaming vars, you need to use SIMULATE>SETVAL and call it once for each subscript combination.

Hope this helps.

[pbreach]

Ouch, okay thanks Tony. Setting all combinations is at least a little more general than two subranges at a time. Will test for speed in case it will be an issue.

As mentioned above, vensim_get_vecvals is to be used for getting the values (not setting), although the Python syntax will be similar.

i.e. for only 2 subscripts model['constant[range1, range2]'] will be the syntax to get the values via vensim_get_vecvals, while model['constant[range1, range2]'] = [[1, 2, 3], [4, 5, 6], [7, 8, 9]] will be the syntax to set the values via SIMULATE>SETVAL|... (for all combinations of subscripts), where model is a VenPy object. Both of these examples should be able to extend to more dimensions if performance is somewhat reasonable.

Thanks again for the guidance.

[tonyvsuk]

I don't understand your question.

It would really help if you detailed what you are trying to do, then I could suggest the correct things for you to use. I can understand writing a Python wrapper for the Vensim DLL, but I just don't see what you are trying to do.

It might also be an idea for you to look at one the DLL examples as they have all calls to the Vensim DLL. If you have Excel, there is an example that reads variables, sets them, retrieves values etc.

[pbreach]

That's okay, it wasn't really a question. It will help when I have some documentation up for the project. What you've mentioned I think will be the way to go for getting and setting subscripted variables.

For non-subscripted variables this project currently supports:

• Getting and setting model constants
• Running models and retrieving output
• 'Replacing' Vensim equations with Python functions during game simulations

All I am trying to do is extend this to subscripted variables.

[tonyvsuk]

Treat subscripted variables exactly the same way as constants, but with the subscript element names instead. The exact same calls are used to set them.

What functions does Python have that Vensim does not out of interest? If it's something useful, we'll add it in to Vensim itself (not many requests for new functions have been made for a long time now).

[pbreach]

Exactly, the API developed here will just allow the user to pass say a multidimensional array instead of setting each element in a loop.

One thing that would be useful to have in Vensim are 2D lookup tables. We use these routinely for representing the relationship between reservoir outflows as a function of gate position and reservoir elevation curves.

Most of the other Python functions that my group is using in conjunction with the Vensim DLL are really domain specific things like genetic algorithms for optimization, or various statistical models found in statsmodels or other scientific / statistical packages being actively developed in Python.

[pbreach]

Okay so this has been addressed in 848618420a36a03b1cf03bfc831cbd08ceb49fb3. I am basically just providing a syntax to set and get subscripts with arrays (which essentially is just getting each element individually with vensim_get_val.

See newly added documentation for details.