ebmdatalab / openprescribing

A Django app providing a REST API and dashboards for the HSCIC's GP prescribing data
https://openprescribing.net
MIT License
97 stars 26 forks source link

KeyError triggered when new data imported but new MatrixStore not loaded #3090

Open evansd opened 3 years ago

evansd commented 3 years ago

This occurred on the All England page (https://openprescribing.net/national/england/) when trying to calculate PPU savings for a date for which there was no data in the current MatrixStore file. Restarting the app so it picks up the new MatrixStore file fixes this, but this isn't great behaviour.

Possibly we want something similar to #3046 but for missing dates, rather than missing organisations.

Relevant Slack error here: https://sentry.io/organizations/ebm-datalab/issues/2408412944/?project=1254683

Traceback:

KeyError: '2021-03-01'
  File "django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "django/core/handlers/base.py", line 115, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "django/core/handlers/base.py", line 113, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "frontend/views/views.py", line 80, in wrapper
    return view_function(request, *args, **kwargs)
  File "frontend/views/views.py", line 364, in all_england
    ppu_savings = get_total_savings_for_org(str(date), "all_standard_practices", None)
  File "frontend/price_per_unit/savings.py", line 111, in get_total_savings_for_org
    totals = get_total_savings_for_org_type(
  File "matrixstore/cachelib.py", line 31, in wrapper
    result = func(*args, **kwargs)
  File "frontend/price_per_unit/savings.py", line 150, in get_total_savings_for_org_type
    quantities, net_costs = get_quantities_and_net_costs_at_date(
  File "matrixstore/cachelib.py", line 31, in wrapper
    result = func(*args, **kwargs)
  File "frontend/price_per_unit/savings.py", line 203, in get_quantities_and_net_costs_at_date
    date_column = db.date_offsets[date]
richiecroker commented 1 year ago

@evansd is this still a problem?

evansd commented 1 year ago

Yes