search

CogComp / open-eval

An open source evaluation framework for developing NLP systems
8 stars 5 forks source link

Allow user to view logs #83

Open joshuacamp opened 8 years ago

ryannk commented 8 years ago

The user as in the admin of the core system correct?

danyaljj commented 8 years ago

The solver should be able to send a log per each instance being solved; We will intercept these logs in the openEval sides, save them to DB, and show them to the user.

ryannk commented 8 years ago

Do we want to do this for every instance. I know we talked about only storing the first 100 results. Should we do this for the logs too?

ryannk commented 8 years ago

Also, do we want the logs to be formatted in any way, or will they just be plain text

danyaljj commented 8 years ago

Yeah for each instance. I think for now we are discussing the possibility of passing output logs from learner. We can decide on how to save (and what/where to save) later. json should be fine.

ryannk commented 8 years ago

Will annotator be updated to include the info to be logged? i.e. will Annotator.addView(TextAnnotation) return an object that has everything to be logged