search

SDXorg / pysd

System Dynamics Modeling in Python
http://pysd.readthedocs.org/
MIT License
367 stars 88 forks source link

XMILE Conformance Clause 2: XMILE Simulator #119

Closed quipa closed 6 years ago

quipa commented 7 years ago

Hello @JamesPHoughton ,

Does pysd aim to be a XMILE simulator as specified in the conformance clause 2, heading 7.2? If so which points of the Base-level (7.3.1) and Optional Conformance (7.3.2-7.3.8) does it currently fulfill and which does the project aim to fulfill in future? Think conformance with clause 2 would probably make translation of XMILE files quite straightforward and simplify the implementation of issue #112.

Cheers, Maxim

JamesPHoughton commented 7 years ago

Hey @quipa , I haven't given it too much thought - I've just built features as I've needed them or as people asked for them. I suppose it would make sense to describe PySD as an aspiring XMILE simulator, although not one that shoots for all of the pieces that could be included in that description. I've always thought of PySD as something that would be used in conjunction with a model development environment, not for constructing models, but for modifying them and exploring their behavior. In that case, parts of the XMILE spec which deal with construction or interaction with the visual model (sliders, plots, basically anything in sections 5, 6) might not need to be included. On the other hand, anything related to model behavior (functions, arrays, variable limits, sections 3 and 4) would want to be on the development pathway. If down the road PySD was used as a back-end for an open-source model development environment, or something where model creation was a major component, then we'd have to work out a new breakdown.

Here's some quick thoughts about what exists, doesnt, and whether I or anyone I know is working on them:

7.3.1 Base-Level Conformance

To conform to the XMILE base level, an implementation:

  1. MUST adhere to the model assumptions (Section 3) yup

  2. MUST support the simulation rules of each base functionality object (Section 3.1 and all subsections) In place, although groups are basically non-relevant in this context

  3. MUST support Euler’s method and Runge-Kutta 4 (Section 3.4.1) Currently not supporting RK4, I broke what was a previously working implementation, and would like to fix it. Either need to write a new integrator, or figure out how to best integrate new stateful objects structure into the numpy rk4.

  4. MUST support the full range of built-in functions (Section 3.5 and all subsections) I believe this to be the case, but haven't gone through the xmile spec to be sure

7.3.2 Optional Conveyor Conformance

Not implemented at all - but probably should be at some point in the future. I think the dev order here should be to get subscripts/arrays working properly, then get exponential delays/smooths working appropriately with subscripts, and then extend the delay machinery to be timestep-aware. This has the potential to be really expensive computationally, so we may want to work out how to get some performance gains before diving into it too deeply.

7.3.3 Optional Queue Conformance

Not implemented. I'm not sure I understand the difference with 7.3.2

7.3.4 Optional Array Conformance

To support the optional array features of XMILE, an implementation:

  1. MUST support dimensions (Section 2.5) Basic functionality in place

  2. MUST generate one value for each array element (Section 3.7.1) Yes

  3. MUST support subscripts and subscript expressions (Section 3.7.1) I think so?

  4. MUST support array operators (Section 3.7.1.1) Not yet, but in progress

  5. MUST support array slicing (Section 3.7.1.2) Not yet, but in progress

  6. MUST support the full list of array built-in functions (Section 3.7.1.3) Not yet, but in progress

7.3.5 Optional Submodel Conformance

To support the optional submodel features of XMILE, an implementation:

  1. MUST support the basic run behavior, i.e., use the same simulation specifications as the whole-model (Section 3.7.4). I'm not sure. In vensim, I think all the submodels would be grouped together. If that's a problem, we'd probably want to use the macro machinery to break the model into different python files/scopes.

7.3.6 Optional Macro Conformance

Macros include three levels: a base level, a recursive level, and an options-filter level.

7.3.6.1 Base Macro Conformance

To support the optional macro features of XMILE, an implementation:

  1. MUST support all macro features but recursive macros and option filters (Section 3.6.1 and Sections 4.8.1-4.8.3) I think this is in place, although it is under-tested

7.3.6.2 Recursive Macro Conformance

To support the optional recursive macro features of XMILE, an implementation:

  1. MUST implement recursive macros (Section 3.6.1 and Sections 4.8.1-4.8.3) I think the existing infrastructure should handle this, although it isn't tested at all

7.3.6.3 Option-Filter Macro Conformance

To support the optional option-filter macro features of XMILE, an implementation:

  1. MUST implement option filter macros (Section 3.6.2 and 4.8.4) No, I haven't encountered this. If someone asks, we might think about it.

7.3.7 Optional Event-Poster Conformance

To support the optional event poster features of XMILE, an implementation:

  1. MUST implement simulation events and their options (Sections 3.4.2 and 4.1.2) Not supported. No plans to at the moment, but if someone desperately wants it, we might consider it.

7.3.8 Optional Inputs Conformance

No plans to support.

Best, James

quipa commented 7 years ago

Hey @JamesPHoughton!

Thanks for looking into the sections and describing pysd's current XMILE simulator conformance ;)!

I totally agree with keeping pysd as a backend XMILE simulator, leaving model diagramming and widgets for a frontend program (by the way know of any interesting open source python possibilities for frontend?), although think construction (through code, with no frontend info) is an interesting feature and think pysd can already be used in this way by accessing pysd.functions directly, right? (for example the Stateful class). One suggestion on using pysd for model construction would be to use XMILE terms so for example pysd.functions.Stateful class would be a Stock class.

As for queues and conveyors I think the main difference is that a queue only processes one item per time step while conveyor handles all items that entered at the same time step. Have a look at Stella documentation for the differences of these elements

Another interesting thing of the spec is that along with standard functions it allows for vendor/user specific functions through a namespace (in 3.2.2.3), so for example something like vensim.log10() and stella.log10().

Cheers, Max

quipa commented 7 years ago

Hey @JamesPHoughton

Looking at 2.1 XMILE on namespaces and the functions.py, I've realised that this script includes what would be the XMILE variables and functions namespaces. From the scripts name and initial docstring it seems to be dealing only with the functions (sensu XMILE), when in fact it deals with model structure and elements (Variables XMILE namespace).

Think it could be useful to adopt this same distinction that XMILE does in the code/project structure itself. Not sure if this should just be limited to changing the scripts name and docstring or something more dramatic such as splitting these aspects into functions.py and model.py, but as XMILE spec requires interaction between these namespaces (in the sense that names should be unique in BOTH namespaces) that might complicate things (although I'm sure there is way to do it in python).

Cheers, Max

JamesPHoughton commented 7 years ago

Hey @quipa, You've got a point that it may make more sense to use XMILE terms for things than vensim terms. The 'Integ' object would correspond to 'Stock'. The stateful is just a parent class for stocks, delays, initials, other things that have to keep something from time to time. It's helpful for the integrator to be able to identify everything it needs to deal with as a subclass of that parent class. (See functions lines 276-278)

The functions namespace originally just held the pieces like if_then_else and xidz, before there were so many different types of stateful objects (I hadn't implemented independent delays or initial functions, etc) and has slowly acquired more pieces, to the point where it has almost everything. It would make sense to refactor the parent class and macro and model classes into their own file, and make the functions namespace correspond more closely to the vensim function set, or pointers to a shared function set. I'll think about that in some more detail. In the meantime, it would certainly make sense to make an xmile file or xmile_functions that would be imported as an xmile namespace into models that need it.

I'll see if I can clean up the functions, and fix some of the documentation there...

JamesPHoughton commented 6 years ago

I'm going to close this, considering our progress in https://github.com/JamesPHoughton/pysd/commit/b1d29d62abaa58517327da26bfe63131295edff5. If there is anything that needs to be extracted into its own issue, lets do that and keep things explicit.