search

quintel / etengine

Calculation engine for the Energy Transition Model
https://energytransitionmodel.com/
MIT License
15 stars 7 forks source link

Incorrect start value prevents input caching. #418

Closed antw closed 12 years ago

antw commented 12 years ago

Check full details on Airbrake: http://quintel.airbrake.io/errors/51977894?auth_token=1bcd54c40e8de6f973d490870c215bd9e3068daf

[PROJECT_ROOT]/app/models/api/v3/input_presenter.rb:32:in `as_json'
[GEM_ROOT]/gems/activesupport-3.2.7/lib/active_support/json/encoding.rb:55:in `block in as_json'
[GEM_ROOT]/gems/activesupport-3.2.7/lib/active_support/json/encoding.rb:77:in `check_for_circular_references'
[GEM_ROOT]/gems/activesupport-3.2.7/lib/active_support/json/encoding.rb:54:in `as_json'
[GEM_ROOT]/gems/activesupport-3.2.7/lib/active_support/json/encoding.rb:242:in `block in as_json'
[GEM_ROOT]/gems/activesupport-3.2.7/lib/active_support/json/encoding.rb:242:in `each'
[GEM_ROOT]/gems/activesupport-3.2.7/lib/active_support/json/encoding.rb:242:in `map'
[GEM_ROOT]/gems/activesupport-3.2.7/lib/active_support/json/encoding.rb:242:in `as_json'
[GEM_ROOT]/gems/activesupport-3.2.7/lib/active_support/json/encoding.rb:47:in `block in encode'
[GEM_ROOT]/gems/activesupport-3.2.7/lib/active_support/json/encoding.rb:77:in `check_for_circular_references'
[GEM_ROOT]/gems/activesupport-3.2.7/lib/active_support/json/encoding.rb:46:in `encode'
[GEM_ROOT]/gems/activesupport-3.2.7/lib/active_support/json/encoding.rb:31:in `encode'
[GEM_ROOT]/gems/activesupport-3.2.7/lib/active_support/core_ext/object/to_json.rb:16:in `to_json'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_controller/metal/renderers.rb:93:in `block in <module:Renderers>'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_controller/metal/renderers.rb:35:in `block in _handle_render_options'
/home/ubuntu/.rbenv/versions/1.9.3-p0/lib/ruby/1.9.1/set.rb:222:in `block in each'
/home/ubuntu/.rbenv/versions/1.9.3-p0/lib/ruby/1.9.1/set.rb:222:in `each_key'
/home/ubuntu/.rbenv/versions/1.9.3-p0/lib/ruby/1.9.1/set.rb:222:in `each'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_controller/metal/renderers.rb:32:in `_handle_render_options'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_controller/metal/renderers.rb:28:in `render_to_body'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_controller/metal/compatibility.rb:50:in `render_to_body'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/abstract_controller/rendering.rb:88:in `render'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_controller/metal/rendering.rb:16:in `render'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_controller/metal/instrumentation.rb:40:in `block (2 levels) in render'
[GEM_ROOT]/gems/activesupport-3.2.7/lib/active_support/core_ext/benchmark.rb:5:in `block in ms'
/home/ubuntu/.rbenv/versions/1.9.3-p0/lib/ruby/1.9.1/benchmark.rb:295:in `realtime'
[GEM_ROOT]/gems/activesupport-3.2.7/lib/active_support/core_ext/benchmark.rb:5:in `ms'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_controller/metal/instrumentation.rb:40:in `block in render'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_controller/metal/instrumentation.rb:83:in `cleanup_view_runtime'
[GEM_ROOT]/gems/activerecord-3.2.7/lib/active_record/railties/controller_runtime.rb:24:in `cleanup_view_runtime'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_controller/metal/instrumentation.rb:39:in `render'
[PROJECT_ROOT]/app/controllers/api/v3/inputs_controller.rb:16:in `index'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/abstract_controller/base.rb:167:in `process_action'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_controller/metal/rendering.rb:10:in `process_action'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/abstract_controller/callbacks.rb:18:in `block in process_action'
[GEM_ROOT]/gems/activesupport-3.2.7/lib/active_support/callbacks.rb:447:in `_run__4232226798151136555__process_action__1794076562627330525__callbacks'
[GEM_ROOT]/gems/activesupport-3.2.7/lib/active_support/callbacks.rb:405:in `__run_callback'
[GEM_ROOT]/gems/activesupport-3.2.7/lib/active_support/callbacks.rb:385:in `_run_process_action_callbacks'
[GEM_ROOT]/gems/activesupport-3.2.7/lib/active_support/callbacks.rb:81:in `run_callbacks'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/abstract_controller/callbacks.rb:17:in `process_action'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_controller/metal/rescue.rb:29:in `process_action'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_controller/metal/instrumentation.rb:30:in `block in process_action'
[GEM_ROOT]/gems/activesupport-3.2.7/lib/active_support/notifications.rb:123:in `block in instrument'
[GEM_ROOT]/gems/activesupport-3.2.7/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
[GEM_ROOT]/gems/activesupport-3.2.7/lib/active_support/notifications.rb:123:in `instrument'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_controller/metal/instrumentation.rb:29:in `process_action'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_controller/metal/params_wrapper.rb:207:in `process_action'
[GEM_ROOT]/gems/activerecord-3.2.7/lib/active_record/railties/controller_runtime.rb:18:in `process_action'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/abstract_controller/base.rb:121:in `process'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/abstract_controller/rendering.rb:45:in `process'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_controller/metal.rb:203:in `dispatch'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_controller/metal/rack_delegation.rb:14:in `dispatch'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_controller/metal.rb:246:in `block in action'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_dispatch/routing/route_set.rb:73:in `call'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_dispatch/routing/route_set.rb:73:in `dispatch'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_dispatch/routing/route_set.rb:36:in `call'
[GEM_ROOT]/gems/journey-1.0.4/lib/journey/router.rb:68:in `block in call'
[GEM_ROOT]/gems/journey-1.0.4/lib/journey/router.rb:56:in `each'
[GEM_ROOT]/gems/journey-1.0.4/lib/journey/router.rb:56:in `call'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_dispatch/routing/route_set.rb:600:in `call'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_dispatch/middleware/best_standards_support.rb:17:in `call'
[GEM_ROOT]/gems/rack-1.4.1/lib/rack/etag.rb:23:in `call'
[GEM_ROOT]/gems/rack-1.4.1/lib/rack/conditionalget.rb:25:in `call'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_dispatch/middleware/head.rb:14:in `call'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_dispatch/middleware/params_parser.rb:21:in `call'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_dispatch/middleware/flash.rb:242:in `call'
[GEM_ROOT]/gems/rack-1.4.1/lib/rack/session/abstract/id.rb:205:in `context'
[GEM_ROOT]/gems/rack-1.4.1/lib/rack/session/abstract/id.rb:200:in `call'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_dispatch/middleware/cookies.rb:338:in `call'
[GEM_ROOT]/gems/activerecord-3.2.7/lib/active_record/query_cache.rb:64:in `call'
[GEM_ROOT]/gems/activerecord-3.2.7/lib/active_record/connection_adapters/abstract/connection_pool.rb:473:in `call'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_dispatch/middleware/callbacks.rb:28:in `block in call'
[GEM_ROOT]/gems/activesupport-3.2.7/lib/active_support/callbacks.rb:405:in `_run__4124807551419855207__call__2815038947148084763__callbacks'
[GEM_ROOT]/gems/activesupport-3.2.7/lib/active_support/callbacks.rb:405:in `__run_callback'
[GEM_ROOT]/gems/activesupport-3.2.7/lib/active_support/callbacks.rb:385:in `_run_call_callbacks'
[GEM_ROOT]/gems/activesupport-3.2.7/lib/active_support/callbacks.rb:81:in `run_callbacks'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_dispatch/middleware/callbacks.rb:27:in `call'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_dispatch/middleware/remote_ip.rb:31:in `call'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_dispatch/middleware/debug_exceptions.rb:16:in `call'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_dispatch/middleware/show_exceptions.rb:56:in `call'
[GEM_ROOT]/gems/railties-3.2.7/lib/rails/rack/logger.rb:26:in `call_app'
[GEM_ROOT]/gems/railties-3.2.7/lib/rails/rack/logger.rb:16:in `call'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_dispatch/middleware/request_id.rb:22:in `call'
[GEM_ROOT]/gems/rack-1.4.1/lib/rack/methodoverride.rb:21:in `call'
[GEM_ROOT]/gems/rack-1.4.1/lib/rack/runtime.rb:17:in `call'
[GEM_ROOT]/gems/rack-1.4.1/lib/rack/lock.rb:15:in `call'
[GEM_ROOT]/gems/actionpack-3.2.7/lib/action_dispatch/middleware/static.rb:62:in `call'
[GEM_ROOT]/gems/rack-cache-1.2/lib/rack/cache/context.rb:136:in `forward'
[GEM_ROOT]/gems/rack-cache-1.2/lib/rack/cache/context.rb:245:in `fetch'
[GEM_ROOT]/gems/rack-cache-1.2/lib/rack/cache/context.rb:185:in `lookup'
[GEM_ROOT]/gems/rack-cache-1.2/lib/rack/cache/context.rb:66:in `call!'
[GEM_ROOT]/gems/rack-cache-1.2/lib/rack/cache/context.rb:51:in `call'
[GEM_ROOT]/gems/railties-3.2.7/lib/rails/engine.rb:479:in `call'
[GEM_ROOT]/gems/railties-3.2.7/lib/rails/application.rb:220:in `call'
[GEM_ROOT]/gems/railties-3.2.7/lib/rails/railtie/configurable.rb:30:in `method_missing'
[GEM_ROOT]/gems/rack-cors-0.2.7/lib/rack/cors.rb:51:in `call'
[GEM_ROOT]/gems/unicorn-4.3.1/lib/unicorn/http_server.rb:531:in `process_client'
[GEM_ROOT]/gems/unicorn-4.3.1/lib/unicorn/http_server.rb:606:in `worker_loop'
[GEM_ROOT]/gems/unicorn-4.3.1/lib/unicorn/http_server.rb:487:in `spawn_missing_workers'
[GEM_ROOT]/gems/unicorn-4.3.1/lib/unicorn/http_server.rb:137:in `start'
[GEM_ROOT]/gems/unicorn-4.3.1/bin/unicorn:121:in `<top (required)>'
/u/apps/etengine/current/bin/unicorn:16:in `load'
/u/apps/etengine/current/bin/unicorn:16:in `<main>'
antw commented 12 years ago

Gents,

I'm about to deploy a hotfix for this... it is caused by a failure to calculate the start value for the households_water_heater_heatpump_air_water_electricity_share input whose start value does not return a Numeric, but instead an array containing two converters.

@dennisschoenmakers I'll change the caching to be more accommodating of such errors (and notify us), but would someone look into the start value GQL on ETsource?

Input.get(597).start_value_for(
  Scenario.new(area_code: 'nl', end_year: 2050).gql)

# => [ <Converter households_useful_demand_for_hot_water_after_solar_heater_and_add_on>,
#      <Converter households_water_heater_heatpump_air_water_electricity>,
#      ... ]
dennisquintel commented 12 years ago

Wow! Good catch. That's very weird indeed!

@ChaelKruip: could you correct this start value asap?

antw commented 12 years ago

Unfortunately other inputs appear to be returning non-numeric start, min, or max values also. The complete list:

By "array-wrapped value" I mean that the GQL returns a value, but it does so inside an array:

Input.get('number_of_micro_chp_fixed').max_value_for(gql)
# => [7349500.0]

Would it be possible to adjust these queries to just return the numeric value? It would avoid the need for having to run extra checks when generating the cache.

For comparison:

A troublesome query, used as a max value:

GQL: Q(potential_electricity_production_of_offshore_wind)
=> [27278.424657534255]

An unrelated, randomly-selected query:

GQL: Q(co2_emissions_of_final_demand_excluding_imported_electricity)
=> 163359325067.51855
antw commented 12 years ago

Okay, last update. :smile:

I think it is possible to remove the wrapped array. The mw_of_offshore input uses the potential_electricity_production_of_offshore_wind query to create the start value. This query looks like this:

V(
  PRODUCT(
   DIVIDE(AREA(onshore_suitable_for_wind),V(energy_power_wind_turbine_offshore,land_use_per_unit)),
   V(energy_power_wind_turbine_offshore,"typical_electricity_production_capacity * full_load_hours / 8760")
  )
)

It appears simply removing the V() wrapper gets rid of the array:

PRODUCT(
  DIVIDE(AREA(onshore_suitable_for_wind),V(energy_power_wind_turbine_offshore,land_use_per_unit)),
  V(energy_power_wind_turbine_offshore,"typical_electricity_production_capacity * full_load_hours / 8760")
)
dennisquintel commented 12 years ago

@antw: you are allowed to make the necessary adjustments yourself. @ChaelKruip: could you help Anthony out here and there if necessary?

This is also a good clean up of the input gqueries. We should be more strict on returning numeric values.

ChaelKruip commented 12 years ago

Fixed with commits a2056fe266de1710f74d717503bf5f9d827fc0b8 and bff4f8d2b958e603fa8096219b0dfd95f9bacceb