Some of the pandas DataFrames within HDF5 files with urbansim-related data (tables of parcels, buildings, etc) have index names (i.e. df.index.name) that are saved as binary data (bytes type) rather than strings. Since bytes and str is handled the same way in Python 2.7, it wasn't a problem. In Python 3, they're treated separately.
We've decided not to add code to orca table registration functions that convert these names. Instead, folks using Python 3 with data that has these quirks should just update their datasets. When we release the version of UrbanSim that has python 3 compatibility, we should include a note about this in the release notes.
We can include a code snippet to do this conversion in HDF5 files, something like:
import numpy
import os
import pandas as pd
new_store = pd.HDFStore('path/to/newstore.h5', mode='w')
with pd.HDFStore('path/to/oldstore.h5') as store:
for tablename in store.keys():
table = store[tablename]
if table.index.name and type(table.index.name) == numpy.bytes_:
table.index.name = table.index.name.decode()
new_store.put(tablename, table, format='t')
new_store.close()
Some of the pandas DataFrames within HDF5 files with urbansim-related data (tables of parcels, buildings, etc) have index names (i.e.
df.index.name
) that are saved as binary data (bytes
type) rather than strings. Sincebytes
andstr
is handled the same way in Python 2.7, it wasn't a problem. In Python 3, they're treated separately.We've decided not to add code to orca table registration functions that convert these names. Instead, folks using Python 3 with data that has these quirks should just update their datasets. When we release the version of UrbanSim that has python 3 compatibility, we should include a note about this in the release notes.
We can include a code snippet to do this conversion in HDF5 files, something like: