Gather data on first join interactions

[SomeKittens]

I propose we actually study the welcome message to see what impact, if any, it has on the chat room.

Starting in January (Holiday season may wreck the curve), Caprica should record every new person joining (noting the point of transcript where they started). We can track if they said anything/asked to ask/etc. This silent recording should go on for a full week.

Then we set the room message to the welcome message for a week. This was the suggested replacement for the welcome ping, and I'd like to see if there is any data to support the claim.

Then, we turn the welcome message back on (though it doesn't have to be Caprica that sends it). Same recording.

Data I'm interested in (only for users with > 20 rep): Does the welcome message increase both the quantity and quality of interaction? Quality of interaction defined as:

• Does not ask to ask
• No question 'dump and runs'

Will there be more visits to the help page?

Will more people stick around/return?

[ryankinal]

Good concept. :+1:

[FirstWhack]

I still agree with this. Although I don't honestly think you'll see the data you expect, but that's the whole point of it. +1

[ralt]

I'll ask @balpha if he likes the idea. I don't want to put back the automatic welcome message without his agreement, even if it's just for a week.

[benjamingr]

Can we please only collect the data anonymously (or at least dispose of the userids/names once it's collected)? I don't like the idea of tracking users.

[rlemon]

We already track certain info, and this could all be mined from the transcript anyways (well everything but them dodging out after a ping). On Dec 6, 2013 6:36 PM, "benjamingr" notifications@github.com wrote:

Can we please only collect the data anonymously (or at least dispose of the userids/names once it's collected)? I don't like the idea of tracking users.

— Reply to this email directly or view it on GitHubhttps://github.com/Zirak/SO-ChatBot/issues/145#issuecomment-30039292 .

[SomeKittens]

@benjamingr Absolutely, though I would like to track rep to see the results in different categories.

[caub]

apart from regular users, people come to ask specific questions (follow ups or not from SO) or general advices in programming. If you meant to get data about which topic are asked, about if the discussion become interesting (probably hard to quantify, ~number of users involve, length of discussion, stars), it's great, those kind of data could be useful for SO as well

[rlemon]

Okay, so I've been thinking about this since the entire hoopla started. There is often a room owner in the room when these messages do happen (or someone can see them) - I suggest a simple command that just notes the time / message ID of the initial ask to ask or ask if anyone is around. I have been using a userscript to add a one click welcome button to the page and wouldn't be opposed to using this to track the metrics on my server. It would only be useful if every room owner / regular / bot maintainer also installed the button and used it when people asked to ask or asked if anyone was around. Example: http://chat.stackoverflow.com/transcript/message/13560058#13560058

[shea-sollars]

Although I don't fully understand the need for these particular statistics/data, other than it's better to have it than not to have it, I do have some ideas for data collection.

We could have the /welcome command itself do some statistics logging, as well as add a listener for anything that remotely looks like an ask to ask question. Then have Caprica send that to a remote server, where maintainers can view, moderate, and verify collected data. When handling (statistics) data of this nature, I think it's imperative that we do not rely on localStorage for this.

Perhaps a whole new command layer specifically for statistics handling, that can be attached to any command we want, and can be used over and over again?

[FirstWhack]

I wouldn't see a problem with specifically compiling a list of arrogant/over the top ignorant users.

On Dec 6, 2013, at 6:36 PM, benjamingr notifications@github.com wrote:

Can we please only collect the data anonymously (or at least dispose of the userids/names once it's collected)? I don't like the idea of tracking users.

— Reply to this email directly or view it on GitHub.

[FirstWhack]

They're bad, and they should feel bad. :P

On Dec 6, 2013, at 6:36 PM, benjamingr notifications@github.com wrote:

Can we please only collect the data anonymously (or at least dispose of the userids/names once it's collected)? I don't like the idea of tracking users.

— Reply to this email directly or view it on GitHub.