Concept: Developing an evaluation model to assess prevention measures

[chisato-t]

Prevention measures evaluation.docx

Aim of the project

to develop a comprehensive and systematic model (system of performance indicators) to assess the prevention measures in the rescue services. The specific sub-goals are:

• to give an assessment of preventive activities as a whole;
• e.g. an “overall index”, which assesses whether the right things and in the right amount are done
• a problem: the aggregation and weighting of the prevention components
• to give an international comparison;
• the value of the “index” should be comparable between countries, therefore it should be generalizable across countries, since different countries do different activities for prevention
• to give an assessment of the trends;
• the value of the index has to be found over the years, so it must be ensured that the input data is (or will be in the future) available annually;
• to give an assessment to specific preventive activities;
• the model must be sufficiently detailed to be able to conclude what activities and in what proportion should be changed to ensure the optimum outcome (in other words: what to change to maximize the outcome);
• the components of the index should also have a meaning / interpretation on their own.
• to give an assessment to the activities across target groups (school children, elderly people etc, who would be more vulnerable)

Hints

• Potentials
  • Input-Output model (e.g. in private sector how is the impact of big marketing campaigns assessed)
  • linear programming (simplex method)
  • ratio analysis (e.g. a matrix of outputs/costs; outcomes/costs; outcomes/outputs etc.)
• Note
  • Keep activity: create new issues to keep search words, interesting articles and papers with citation (Why? How can we apply them to our project?)

[chisato-t]

Procedure:

1. Identify indicator which should be measurable (inputs, activities, outputs, outcomes) --> Logic Model of policy evalutaion would be applicable?
   • 1. Document analysis (if and how have the indicators been described before in development plans, service cards and so on. Also the experience of our partners);
   • 2. Expert opinion (2-3 opinions from each partner country);
   • 3. Conducting a workshop (an expert from every country participates and as a result a list of indicators is developed).
2. Prevention activities should be covered in details and in the same framework an input-output model can be developed for every single type of activity
3. Indicators should be weighted by AHP or Ranking/Grouping
4. Aggregation phase. Can get an interpretations. -->LP problem
5. Assessment as the index is the basis for benchmark.

[TarmoPuolokainen]

Here are some background materials (and our questionnaire):

background.pdf

http://www.paasteamet.ee/dotAsset/0e0aa3b2-c02a-4f1f-a838-f73d184c4156.pdf

prevention activities questionnaire_091015.docx

Paper 1: https://gupea.ub.gu.se/bitstream/2077/2939/1/gupea_2077_2939_1.pdf

[TarmoPuolokainen]

I tried to explain the task a bit. In case you have any questions, please write and we can discuss this together :) task.docx

[TarmoPuolokainen]

Here's the feedback we have received so far. questionary_germany.docx questionary_latvia.docx questionary_sweden.docx Civil Protection Act 101227_sweden.pdf questionary_denmark.docx

[chisato-t]

Hi, thank you for providing lots of papers. I thought the Henrik's paper was the best that we could replicate after all. Deciding input output variables based on the questionnaires and using the ordered probit model. However, as I see the questionnaires, there are some gaps in the way of thinking among countries. (For example, the most crucial part, what is the goal of prevention activities: major answered fewer victims and less property damaged, but the Denmark answered the change of citizen's behavior. Those would be adjusted by AHP to be evaluated?) My question is maybe there would be a country specific variable. For instance, the citizen's personality would affect its prevention activities. Then in that case, could we integrate it? In the paper, he used only fire service dummy alpha.