Phone-home capability for error reporting

githubtraining / looking-glass-action

provides feedback to learners based on the outcome of their validation lab grader workflow

MIT License

35 stars 10 forks source link

Phone-home capability for error reporting #1

Open brianamarie opened 3 years ago

brianamarie commented 3 years ago

As a lab author, I would be interested in a "phone-home" capability. I'd like to track the success/failure rates of grading and troubleshooting.

It would be interesting and useful to know:

Is a lab causing more errors than expected? This could be a signal that the lab needs improving.
What is the rate of errors/failure that is ideal for learning? (Not too easy, not too hard?)

cc @hectorsector @mattdavis0351

mattdavis0351 commented 3 years ago

@brianamarie I think this is a very reasonable long term goal for the Looking Glass. A major hurdle will be getting infrastructure that supports the aggregation of this data. Maybe we can get some Azure storage or form some relationship to the OD and learning content data.

jonlipsky commented 3 years ago

@mattdavis0351 When you are ready to tackle this, let me know and we can setup a call with @ethanis and @dwhathaway who have been working on adding telemetry/analytics to Valet. We can mirror their setup/approach on the server side.

mattdavis0351 commented 3 years ago

@jonlipsky Will do. We will probably begin working towards this after we do the LL course lift in to this format. So we are a few weeks away from beginning the work related here. We'd also love for this information to tie back to our other metrics in Looker if at all possible.

I'll be sure to ping you before diving into this so I don't go off the deep end in the wrong direction.