Updated HLCM calibrated coefficients

[janowicz]

This PR contains updated calibrated HLCM coefficients.

@Eh2406 When you get a chance, could you please try a simulation run with the code/configs in this PR? This branch should be independently runnable- but please let me know if you encounter any issues.

[Eh2406]

Running as is gets errores about pandana. Removing the calls to .init_pois and .reserve_num_graphs gets an error:

/home/da/anaconda2/bin/python /home/da/semcog_urbansim/Simulation.py
using seed 271828
./runs/run4085.h5
Running step 'refiner'
Time to execute step 'refiner': 17.97 s
Running step 'build_networks'
Time to execute step 'build_networks': 58.67 s
Running step 'neighborhood_vars'
Computing accessibility variables
Computing log_sum_residential_units
Removed 8 rows because they contain missing values
Computing log_sum_nonresidential_units
Removed 8 rows because they contain missing values
Computing ave_lot_sqft
Removed 8 rows because they contain missing values
Traceback (most recent call last):
  File "/home/da/semcog_urbansim/Simulation.py", line 26, in <module>
    ["increase_property_values"])  # Hack to make more feasibility
  File "/home/da/anaconda2/lib/python2.7/site-packages/orca/orca.py", line 1987, in run
    step()
  File "/home/da/anaconda2/lib/python2.7/site-packages/orca/orca.py", line 792, in __call__
    return self._func(**kwargs)
  File "/home/da/semcog_urbansim/models.py", line 1011, in neighborhood_vars
    nodes = networks.from_yaml(orca.get_injectable('net_walk'), "networks_walk.yaml")
  File "/home/da/udst/urbansim/urbansim/utils/networks.py", line 57, in from_yaml
    nodes[name] = net.aggregate(radius, type=agg, decay=decay)
  File "/home/da/anaconda2/lib/python2.7/site-packages/pandana/network.py", line 336, in aggregate
    return pd.Series(res, index=self.node_ids)
  File "/home/da/anaconda2/lib/python2.7/site-packages/pandas/core/series.py", line 250, in __init__
    data = SingleBlockManager(data, index, fastpath=True)
  File "/home/da/anaconda2/lib/python2.7/site-packages/pandas/core/internals.py", line 4117, in __init__
    fastpath=True)
  File "/home/da/anaconda2/lib/python2.7/site-packages/pandas/core/internals.py", line 2719, in make_block
    return klass(values, ndim=ndim, fastpath=fastpath, placement=placement)
  File "/home/da/anaconda2/lib/python2.7/site-packages/pandas/core/internals.py", line 115, in __init__
    len(self.mgr_locs)))
ValueError: Wrong number of items passed 0, placement implies 270914
Closing remaining open files:./data/run4032_school_v2_baseyear.h5...done./data/semcog_networks.h5...done

Process finished with exit code 1

We tried to rebase onto the code from #24 our attempt is in this branch but we get an odd error:

Running step 'feasibility'
...
Disaggregating prop_residential_buildings to parcels from zones
standardizing zones_logsum_pop_low_income on parcels
standardizing zones_transit_jobs_30min on parcels
standardizing zones_taz on parcels
Traceback (most recent call last):
  File "/home/da/semcog_urbansim/Simulation.py", line 67, in <module>
    compress=True)
  File "/home/da/anaconda2/lib/python2.7/site-packages/orca/orca.py", line 1987, in run
    step()
  File "/home/da/anaconda2/lib/python2.7/site-packages/orca/orca.py", line 792, in __call__
    return self._func(**kwargs)
  File "/home/da/semcog_urbansim/models.py", line 854, in feasibility
    parcel_custom_callback=parcel_custom_callback
  File "/home/da/udst/urbansim_parcels/urbansim_parcels/utils.py", line 773, in run_feasibility
    else parcels.to_frame())
  File "/home/da/anaconda2/lib/python2.7/site-packages/orca/orca.py", line 230, in to_frame
    df[name] = col()
  File "/home/da/anaconda2/lib/python2.7/site-packages/orca/orca.py", line 634, in __call__
    col = self._func(**kwargs)
  File "/home/da/semcog_urbansim/variables/variables_zone.py", line 417, in column_func
    return standardize(orca.get_table(table_name)[column_to_s])
  File "/home/da/semcog_urbansim/variables/variables_zone.py", line 398, in standardize
    return (series - series.mean()) / series.std()
  File "/home/da/anaconda2/lib/python2.7/site-packages/pandas/core/ops.py", line 721, in wrapper
    result = wrap_results(safe_na_op(lvalues, rvalues))
  File "/home/da/anaconda2/lib/python2.7/site-packages/pandas/core/ops.py", line 692, in safe_na_op
    lambda x: op(x, rvalues))
  File "pandas/_libs/algos_common_helper.pxi", line 1212, in pandas._libs.algos.arrmap_object
  File "/home/da/anaconda2/lib/python2.7/site-packages/pandas/core/ops.py", line 692, in <lambda>
    lambda x: op(x, rvalues))
TypeError: ufunc 'subtract' did not contain a loop with signature matching types dtype('S32') dtype('S32') dtype('S32')
Closing remaining open files:./data/run4032_school_v2_baseyear.h5...done./data/semcog_networks.h5...done

Process finished with exit code 1

Help would be appreciated.

[Eh2406]

Why does feasibility need that column? Looks to me like it doesn't https://github.com/SEMCOG/urbansim_parcels/blob/master/urbansim_parcels/utils.py#L773 just calls parcels.to_frame(), is there a way that could be more precise about what column it needs? (also is there a public repo we can point to instead of that private pre release version?)

[janowicz]

@Eh2406 Ah ok. On that line of urbansim_parcels/utils.py that you reference above, could you please replace the parcels.to_frame() with the following? parcels.to_frame(parcels.local_columns + ['max_far', 'max_height', 'max_dua', 'parcel_size', 'land_cost', 'ave_unit_size'])

There's no general way to get the columns needed yet, so will need to replace parcels.to_frame() with the the above to avoid calculating all parcel-level variables, which should usually be avoided since it slows things down. I made the above change in my local copy of urbansim_parcels but forgot to mention! Sorry about that. The list of variable names above are the particular parcel-level columns that the SEMCOG model actually needs in the proforma.

Regarding the pandana error you get above- I'm not sure why that's happening- this PR didn't change the way we are interacting with pandana I think, or am I missing something?

The urbansim_parcels repo is now public here: https://github.com/urbansim/urbansim_parcels

I just committed the above parcels.to_frame() change to the following branch of the repo, which you may find it helpful to utilize: https://github.com/urbansim/urbansim_parcels/tree/semcog-ej-test/urbansim_parcels

Please note, however, that if you move to the latest code in urbansim_parcels, there may be other changes we need to make to the SEMCOG model to make sure we're calling functions with their latest arguments. For example, the following line in the scheduled _development_events model step should be changed from: all_buildings = parcel_utils.merge_buildings(b, sched_dev[b.columns], False, max_id) to: all_buildings = parcel_utils.merge_buildings(b, sched_dev[b.columns], False) since merge_buildings function takes different arguments now compared to before.

I made the various changes I reference in this comment to my branch of the semcog_urbansim model: https://github.com/janowicz/semcog_urbansim/tree/hlcm-calib

Which you can run in conjunction with the semcog-ej-test branch of urbansim/urbansim_parcels: https://github.com/urbansim/urbansim_parcels/tree/semcog-ej-test