search

CommonsBuild / coordination

The Coordination repo issues serve to coordinate all the work across the Token Engineering Commons (TEC)
11 stars 4 forks source link

Monitor Discord new members, are people still getting stuck or is it getting better? #671

Closed tamarandom closed 2 years ago

tamarandom commented 3 years ago

📋 Anything to add?

The onboarding journey had bad metrics. There were some steps taken to improve that (see #499).

Are they working?

Criteria for success

Note: There is a stop-gap measure of @danelsuga, as part of her responsibilities as onboarding coordinator, DM'ing those that get stuck on the "green check box" step. We will know more over the next sprint about whether that is working or not. When we checked earlier this week, 7 of the last 10 people that landed in the server did not have the "acknowledge CC" role and so are not able to see or join any channels.

⏰ Urgency/Due Date/Blockers?

❓ Why is it important?

🎉 Subtasks

🤼‍ Reviewer

@danelsuga

🔗 Work doc - inspirational links

danelsuga commented 3 years ago

I contacted @liviade1 and @JuankBell today to see if we could modify the message to make it clearer to new joiners that they will not have access to anything if they don't green-check the message. Perhaps if we make it supremely evident right from the start, that might help alleviate the problem.

danelsuga commented 3 years ago

Note also that of all the new joiners whom I contacted, none of them at this point have returned to green-check the message.

Vyvy-vi commented 3 years ago

The edits to the messages have been made. Hopefully, that should help emphasize that the reaction is necessary for getting access. I think it would be more easily classified if we know how many of the users are actually human. The site made in #499, hasn't been adopted as of yet, however, once it is, we could be more sure about how many members are bots and how many humans(and in future, no new bots could infiltrate the server)

tamarandom commented 3 years ago

Oh no! We don't have a champion for this. @Vyvy-vi would this be something you can take on next Sprint? I imagine it will remain a repeating issue for the next few sprints while we observe the results of changes. @danelsuga is looking at when new members are stuck (i.e., not checking the green box) and reaching out to them directly. A few of us are also manually assigning the role when we see someone we know personally that is in that limbo state. I did that this week for Tyler from BlockScience.

For the site made in #499, are there the next steps to roll that out? I don't see them in the issue itself.

tamarandom commented 3 years ago

@tamarandom to find a champion. (Hope it can be you @Vyvy-vi )

Vyvy-vi commented 3 years ago

Yes, I can take this next Sprint. The discord invite site is useable, the next steps would be -

tamarandom commented 3 years ago

Hero!

Vyvy-vi commented 3 years ago

Yes, I can take this next Sprint. The discord invite site is useable, the next steps would be -

  • Delete all invites to the server and use https://discord.tecommons.org/ for all discord invitations. (*note- we might need to discuss with Comms/respective WGs about editing links to the calls. We might need to tweak this a little bit to ensure that this can link to the discord voice channels too, but without the invite)
  • Enforce prevention of discord invite making permissions

However, I don't know if we want to move forward with using the site

Vyvy-vi commented 3 years ago

I've looked a bit into the server onboarding process, and here's some takeaways-

  1. Maybe we should have a hybrid approach, where we can use both normal discord links and the discord.tecommons.org website. I'll have to change the website workflow a bit and tweak the bot(store the user ids of the people that filled captcha, and are verified as "Human"). Whenever, anyone enters the server, if their id is not on the list, the bot removes them from the server, and DMs them a message about how to fill captcha on the site, and redirects them to discord. This might help us in inviting people directly to calls.

  2. Maybe, we should add a pre-text to invites about the onboarding process. -> GO to discord.tecommons.org -> Verify you are human by filling captcha -> Open the server and read the rules and policies, accept it by reacting with a ✅ -> Go to the call

  3. A large share of people, that haven't acknowledged that message, fall into 4 categories:

    • The ones that joined before the bot was set up(not a lot of people(5-6?), and they do have access to the server since the Acknowledged CC role was added to them manually)
    • The bots (not sure about the exact distribution here, but atleast 30 bots or more) I'm not sure how we filter these out and remove them. I think most probably we can kick all of the ones that didn't respond to Katy during onboarding, and send them a message stating, they can join back in via the site. (might ensure that the humans join)
    • The ones that didn't go through the onboarding process.
    • The ones that weren't able to go through the onboarding process. I think we need some community feedback from people that just joined. I'm wondering if maybe we can get some feedback from people that got stuck, who just joined about what can be done better, and on which step did they get stuck.

  4. Add an extra message that the bot DMs to people when they enter the server, which tells them about the onboarding process, and on which message and policies to read and react with ✅ on. This message would be sent, whenever someone enters the server. I think this should serve the purpose.

  5. When we move to using the captcha site, maybe we can tweak the server a bit so that people can get access through one source rather than needing 2 step-process? (verify you're human at https://discord.tecommons.org and acknowledge CC in the server on the onboarding channel message) I could probably add that Acknowledged CC policies and rules on the site. While someone fills the captcha, they can accept that on the site itself. (If this is something that we would like to look into)

  6. Any timeline for when the site would be refactored for these new changes- Most probably 2 days or less(depending on my availability) Major changes would be-

    • Storing user ids of people that filled captcha
    • Tweak the bot to check if a person's id is in the list, whenever someone joins the server
    • Have a discussion on the data. This would log the user IDs(and due to some other sourcecred configs, it would store some data for that), so I'll most probably make a Privacy Policy - I think maybe it would be fine if I make it, but it would be best that it's run through and approved by someone from the TEC(legal?), just to be sure that it abides by TEC's values/policies

Should we host a Sync call for discussing next steps? (Q) from whom should I ask with regard to the onboarding process and what time would work for people concerned about this?)

danelsuga commented 3 years ago

Thank you, @Vyvy-vi, for all of this great information. I really like your solutions, and I'm totally down for a sync. I think that's a super idea. Timeframe availability can be flexible especially if it's you and I, since I am CET and you are India. Ideally we could do your early afternoon/my morning. It depends on who wants to join. I am in 100%. Let's get feedback from @tamarandom.

Vyvy-vi commented 3 years ago

*Though my personal time zone sorta matches UTC +1, so timing shouldn't be an issue. I'm most available from 6.30pm CET to 10.30pm CET and can be available 1.30pm - 3.30pm CET

tamarandom commented 2 years ago

Closing this issue now! Thanks for everything @Vyvy-vi