Investigate data collection for outreach and metrics purposes

[ghost]

The testdaybot should be able to collect certain information for the purposes of reaching out to event attendees and to generate metrics so we can make improvements to the program.

Some initial ideas include:

• IRC names
• Bugs
• Tests
• Tasks
• Check-ins

A couple of requirements:

• This information should only be queryable by event organizers.
• This information should be opt-in for event participants
• A participant should be able to query the bot to scrub data collected on them
• A participant should be able to opt-out of data collection for their activity
• All data should be stored privately or in some way that is not publicly accessible
• No data should be kept indefinitely - we'll need to determine a reasonable time limit

[galgeek]

Test Day Start and End should be saved with metrics, as well as the etherpad.

[whimboo]

PR #40 has been merged. Keeping this issue open for the remaining items.

[galgeek]

Hi @ashughes1!

Could you please expand on and clarify requirements 2 and 4, "information should be opt-in..." and "participant should be able to opt-out..."? Maybe there's a relevant Mozilla privacy policy that I could review?

Thank you!

[ghost]

I mean that the Testdaybot should not be collecting information about people without those people knowing about it. Maybe for every new participant who joins the event we could send a backchannel message to them to notify them what information Testdaybot routinely collects and to tell them a command they can send to Testdaybot if they want it to stop.

[galgeek]

Hi @ashughes1!

Is it ok to ask people to opt out each test day, or should we remember that from test day to test day?

[ghost]

I think it should default to prompt people each testday as long as someone has not opted in/out. In other words, a user will be prompted each and every testday they join until they decide opt-in or opt-out. Perhaps keep a list of opted in/out IRC nicks and check against that list before prompting a channel joiner.

[whimboo]

Given that remembering those opted-out people would require to store data to disk, I wonder if we could move this to a separate issue. It's not ideal without it, but not a P1 blocker.

[ghost]

Given that remembering those opted-out people would require to store data to disk, I wonder if we could move this to a separate issue. It's not ideal without it, but not a P1 blocker.

That's fine but I do not want the code here merged to production until we can provide the facility to opt-out due to privacy concerns.

[whimboo]

PR #46 adds the opt-out and will be landed soon. It will keep the list of opted-out users as long as the bot runs and is not quit or crashes.

[galgeek]

Hi, @ashughes1 and @whimboo!

I'll have today's comments addressed shortly, then will work on saving to disk across bot restarts.

In the meantime, yesterday's WIP is running in #testdaybotTest. It might be helpful to take a look. You could start with :help.

[whimboo]

Is this issue done now given that PR #46 has been landed? Or what's left to do?

[galgeek]

Hi, @whimboo and @ashughes1!

For the data currently collected, maybe we want to provide it to organizers as something other than IRC messages, e.g., a file. (I've begun some code for that in PR #48.) Or should that be a separate issue?

[whimboo]

For now IRC should be fine. If metrics need to be happen, we might have to push into a database. But that's clearly outside of the scope of this issue.

[whimboo]

By the latest merge it looks like that all which was requested here is fixed. @ashughes1 please file a new issue if something is still missing or needs an enhancement. Thanks.