general use of injectables

[fscottfoti]

@jiffyclub Matt - got a question for you. I've taken to using injectables as a way to store general state in the simulation. In this specific case I'm talking about the submarket_ratios (price shifters) that are used in supply and demand, but I use them for other data too - things that aren't appropriate as tables. First question, does this jive with your idea of UrbanSim - to store state as injectables? It's pretty handy, and works well.

If the answer to the first is yes, I have two follow up questions. First, I don't always inject the injectables - I mean sometimes I use them in a module rather than in a "sim.model" - in other words, I often use sim.get_injectable(). When I do this though, and the injectable is a function, the actual function gets returned. Is this something that should be fixed? I mean, I don't have autocall=False set or anything so I would sort of expect it to call the function and actually return the value - e.g. isn't this how get_table works?

Second, I sometimes check to see if state is already set and have to do it this way if key in sim.list_injectables(): - I wonder if it would be appropriate to pass an optional second parameter which is what gets returned if the key does not currently exist, similar to .get on a dictionary. So instead of needing to check if it exists or not, you just call sim.get_injectable(key, None) or something similar. Maybe this would require doing the same on the other "get" methods, or maybe not. Thoughts?

[jiffyclub]

We should update get_injectable so it evaluates functions and returns their results.

I'd be interested to take a closer look at how you're setting things up and what your goals are. You're not doing anything wrong per se, but anytime someone starts to rely on mutable, global state I like to see what the options are. That could be a sign something about the simulation framework isn't serving you well.

Ideally I think we'd want to avoid having much code at the module level because the simulation framework can't manage or track that. (As in when we made the table_source thing for lazy loading of tables.) So maybe let's have a look at exactly what you're trying to accomplish and see if injectables are the right thing.

[fscottfoti]

Good point - I've started keeping the shared default models in its own repo called urbansim_defaults, at least for now. So you can search the repo for add_injectable or get_injectable - unfortunately github hasn't updated the search index for 21 hours so you have to search the individual files using ctrl^f.

https://github.com/synthicity/urbansim_defaults/blob/master/urbansim_defaults/utils.py

Anyway, I seems to use injectables for 2-3 different things.

• Most common is small bits of simulation state - like run number, uid, hdf stores, and so on.
• Second is to store medium size json data structures that aren't data frames. I now have a settings object that I use to configure quite a bit of the simulation (and which enables the shared use of urbansim_defaults between bayarea and semcog and other regions).
• Third is I keep some debugging information around, like the new_buildings data frame, which keeps all buildings that have been added - this is what I pass to the website for viewing. This maybe should just be a table but I don't really consider it "official" simulation data so I'm a bit reticent to add it as a table or maybe other models could begin referring to it.
• I also keep the shifters as a pd.Series as an injectable.

All-in-all I guess I just use injectables as the key-value store for quite a bit of general state.

One other related (but not directly) question. I want to start pegging each commit of bayarea_urbansim to a certain urbansim version, a certain urbansim_defaults version, and maybe a certain pandana version. So that you could run python setup.py install in bayarea_urbansim and it would install the dependencies but not actually install anything. Is a degenerate setup.py the best way to do that?

[jiffyclub]

For your last question, a requirements file is probably the way to go: https://pip.readthedocs.org/en/1.1/requirements.html. If you have a pip-requirements.txt file you use it like pip install -r pip-requirements.txt. I think conda supports files with the same format, so you could also have conda install --file conda-requirements.txt.

[fscottfoti]

Yes, that's exactly what I need - thx.

[jiffyclub]

Looking over urbansim_defaults/utils.py: for those functions at least I'd push hard to make them purely functional, e.g. they don't read from or write to global state. I think anything they need they can get as inputs from an officially registered sim.model. And some of the functions there look like they could actually be models, like write_simulation_output and write_parcel_output.

If you really need a global key value store for things that will never be injected you don't necessarily have to use the injectables feature. You can, for example, set variables straight on the sim module itself:

In [6]: import numpy as np

In [7]: np.foo = 'foo'

In [8]: np.foo
Out[8]: 'foo'

In [9]: np.MYGLOBALS = {}

In [10]: np.MYGLOBALS['bar'] = 'bar'

In [11]: np.MYGLOBALS
Out[11]: {'bar': 'bar'}

But I'd try to avoid that sort of thing as much as practically possible. My conception of simulations is something like "put all real work in models, which may take advantage of functional-style utilities".

[tbuckl]

happy to see this conversation about injectables here! this was a small hurdle for me.

i've been trying to understand how one might, for example, use urbansim's hedonic estimations with real estate data in washington, dc. i started by looking at the sanfran urbansim repo. it took me a moment to understand that data used for the simulation was added to the simulation using add_injectable.

[jiffyclub]

Tom, have you looked over the simulation docs? http://synthicity.github.io/urbansim/sim/index.html Please, let us know if there is info missing from those.

[tbuckl]

I had not. Thank you, this looks helpful.

[fscottfoti]

This is solved primarily by the following PR

https://github.com/synthicity/urbansim_defaults/pull/3

I'm not storing state as injectables unless absolutely required.