End use: edit fucntions available for example_use_data.py

[epogrebnyak]

https://github.com/epogrebnyak/rosstat-kep-data/blob/master/example_use_data.py

[epogrebnyak]

@Pastafarianist: can you check error at https://github.com/epogrebnyak/rosstat-kep-data/blob/master/example_use_data.py#L20-L22 ?

[dniku]

For me, a version freshly checked-out from the repository crashes even earlier:

Traceback (most recent call last):
  File ".../rosstat-kep-data/example_use_data.py", line 13, in <module>
    ts1 = get_ts('SOC_WAGE_rub','a', 1999)
  File ".../rosstat-kep-data/kep/query/end_user.py", line 22, in get_ts
    return df[label] 
  File "/usr/lib64/python3.4/site-packages/pandas/core/frame.py", line 1969, in __getitem__
    return self._getitem_column(key)
  File "/usr/lib64/python3.4/site-packages/pandas/core/frame.py", line 1976, in _getitem_column
    return self._get_item_cache(key)
  File "/usr/lib64/python3.4/site-packages/pandas/core/generic.py", line 1091, in _get_item_cache
    values = self._data.get(item)
  File "/usr/lib64/python3.4/site-packages/pandas/core/internals.py", line 3211, in get
    loc = self.items.get_loc(item)
  File "/usr/lib64/python3.4/site-packages/pandas/core/index.py", line 1759, in get_loc
    return self._engine.get_loc(key)
  File "pandas/index.pyx", line 137, in pandas.index.IndexEngine.get_loc (pandas/index.c:3979)
  File "pandas/index.pyx", line 157, in pandas.index.IndexEngine.get_loc (pandas/index.c:3843)
  File "pandas/hashtable.pyx", line 668, in pandas.hashtable.PyObjectHashTable.get_item (pandas/hashtable.c:12265)
  File "pandas/hashtable.pyx", line 676, in pandas.hashtable.PyObjectHashTable.get_item (pandas/hashtable.c:12216)
KeyError: 'SOC_WAGE_rub'

Re-running kep_import.py fixed the problem. I wonder what caused it in the first place. Does it work for you with the current repository state?

[dniku]

This one was trivial. end_user.py contained start_date instead of end_date. This goes back at least to https://github.com/epogrebnyak/rosstat-kep-data/commit/c63b8116a83838e9b64dec54dd6bdca4a2b2c68f#diff-bb1dc03167f0db5d223aeb0b014fef1fR69, and I haven't hunted it down further.

[epogrebnyak]

As for KeyError: 'SOC_WAGE_rub' - it is not a big problem, at least it is traceable. Tests happen to overwrite database, until restored by proper import, like kep_import.py. Addressed in issue #75 (split db files), also may split kep.update() to kep.import and kep.output methods and call update method in example_use_data.py

[epogrebnyak]

See three todos in https://github.com/epogrebnyak/rosstat-kep-data/blob/master/example_use_data.py#L5-L15

[epogrebnyak]

Also - maybe get_dfa(), get_dfm(), get_dfm() may accept arguements.