Added scripts used in WMT evaluations

[cfedermann]

Back from Stockholm, will be back in action tomorrow -- have a reviewing deadline tonight :|

Marathon finished :)

On Mon, Jul 16, 2012 at 6:00 PM, Matt Post < reply@reply.github.com

wrote:

You can merge this Pull Request by running:

git pull https://github.com/mjpost/Appraise master

Or you can view, comment on it, or merge it online at:

https://github.com/cfedermann/Appraise/pull/20

-- Commit Summary --

• added script to analyze three-way ranking task results

-- File Changes --

A scripts/summarize_three_way_ranking.py (69)

-- Patch Links --

https://github.com/cfedermann/Appraise/pull/20.patch https://github.com/cfedermann/Appraise/pull/20.diff

---

Reply to this email directly or view it on GitHub: https://github.com/cfedermann/Appraise/pull/20

[mjpost]

Sure, no problem. Congrats on the (real) marathon!

On Jul 16, 2012, at 12:18 PM, Christian Federmann wrote:

Back from Stockholm, will be back in action tomorrow -- have a reviewing deadline tonight :|

Marathon finished :)

On Mon, Jul 16, 2012 at 6:00 PM, Matt Post < reply@reply.github.com

wrote:

You can merge this Pull Request by running:

git pull https://github.com/mjpost/Appraise master

Or you can view, comment on it, or merge it online at:

https://github.com/cfedermann/Appraise/pull/20

-- Commit Summary --

• added script to analyze three-way ranking task results

-- File Changes --

A scripts/summarize_three_way_ranking.py (69)

-- Patch Links --

https://github.com/cfedermann/Appraise/pull/20.patch https://github.com/cfedermann/Appraise/pull/20.diff

---

Reply to this email directly or view it on GitHub: https://github.com/cfedermann/Appraise/pull/20

---

Reply to this email directly or view it on GitHub: https://github.com/cfedermann/Appraise/pull/20#issuecomment-7010861

[cfedermann]

Hi again, I'm thinking of putting a similar summary feature into the status view for tasks... Will keep you posted :)

[mjpost]

That would be very useful!

On Jul 19, 2012, at 5:43 AM, Christian Federmann wrote:

Hi again, I'm thinking of putting a similar summary feature into the status view for tasks... Will keep you posted :)

---

Reply to this email directly or view it on GitHub: https://github.com/cfedermann/Appraise/pull/20#issuecomment-7093905

[cfedermann]

Hi Matt, support for basic counts on raw_result classes is added with 018d0b9 -- I mailed you a screenshot.

Hope it works on your machine :)

[mjpost]

For posterity, can you provide a link to this page, assuming one has a server running at localhost:8000/appraise?

On Jul 19, 2012, at 11:48 AM, Christian Federmann wrote:

Hi Matt, support for basic counts on raw_result classes is added with 018d0b9 -- I mailed you a screenshot.

Hope it works on your machine :)

---

Reply to this email directly or view it on GitHub: https://github.com/cfedermann/Appraise/pull/20#issuecomment-7102101

[cfedermann]

You need to have staff privileges or be a superuser to get the following navbar:

Clicking on "Status" gives you an overview of the completion status for all tasks. You can click any of the tasks to see "task-specific" status information which includes the raw_result counts mentioned above.

The "Status" page is available here: http://localhost:8000/appraise/status/

[mjpost]

I see now, I didn't realize it was linked from the non-admin pages, so I was poking around in there.

Thanks.

On Jul 19, 2012, at 12:57 PM, Christian Federmann wrote:

You need to have staff privileges or be a superuser to get the following navbar:

Clicking on "Status" gives you an overview of the completion status for all tasks. You can click any of the tasks to see "task-specific" status information which includes the raw_result counts mentioned above.

The "Status" page is available here: http://localhost:8000/appraise/status/

---

Reply to this email directly or view it on GitHub: https://github.com/cfedermann/Appraise/pull/20#issuecomment-7104123

[mjpost]

I updated this pull request to contain the files needed for generating MTurk batches.