Review Attendee Survey Form and Email for WordPress Events including WordCamp

[devinmaeztri]

As we want to automate the attendee survey for all WordPress events including WordCamp, we want to also improve the form and email to increase the response rate.

This is the GDocs of the current email and form to review. https://docs.google.com/document/d/1467XImh4NsgEfN4xz4yLyqj5dyWas1y92S-dbdj-Ge4/edit#heading=h.w7jdbu9dnn0a

Please also add your input as suggestions or comments. Thank you!

[dorsvenabili]

I've reviewed the texts, questions and answers and all of them feel relevant to all types of WP Events and/or WordCamps. Thank you!

[ciudadanoB]

As stated in the other issue, I'm sharing some feedback here regarding the attendee survey.

I think it should follow the same logic as the Camptix ticket form. There are some questions that are mandatory and organizers can't edit/delete them (first time attending? allergies? accessibility needs? code of conduct?) but we can translate them through GlotPress via Meta > WordCamp.org project. Those would be the ones we want to gather as a global project to monitor the health of the events.

But also (as in the tickets), organizers can add their own questions so they get the info that will help them to craft better events in the future. If we lose that (or if we make attendees answer two different surveys) they won't like it, for sure.

If we create a new activity for the event, a new space, if we experiment with some kind of content, we really need to know the opinion of the attendees, and that would never fit in a fully-automated survey.

Also, organizers need a way to visualize all these data easily. With Crowdsignal there's no problem as it takes care of making the reports and graphics, but now we're going steps back with this system as far as I understand 🤷🏻‍♂️ And for that reason I would also leave the flagships events out of this survey, because with such a big amount of answers it can be a real problem to handle the feedback this way (and they always send their own surveys).

[dorsvenabili]

Based on @ciudadanoB's feedback and @malgrauk's feedback here: https://github.com/WordPress/wordcamp.org/issues/1344#issuecomment-2222575751

I think we should not send this survey to our flagship events.

With Crowdsignal there's no problem as it takes care of making the reports and graphics, but now we're going steps back with this system as far as I understand

What about sending an automated email after-event to all WP Events/WordCamps (not flagships) with a link to a standard Crowdsignal survey? It would provide a full analysis and graphics that we don't have in our internal system.

cc/ @naokomc

[StevenDufresne]

What about sending an automated email after-event to all WP Events/WordCamps (not flagships) with a link to a standard Crowdsignal survey? It would provide a full analysis and graphics that we don't have in our internal system.

How many flagship events are there in the year? Do we need to automate it?

[dorsvenabili]

@StevenDufresne there are 3 flagship events per year: WC Asia, WC Europe and WCUS.

[peiraisotta]

Jumping in because I'm doing some research to gather information on how we can standardize all our post-event surveys.

I'm currently looking into a tool called Alchemer, that was used for the 2023 Annual Survey.

It looks like, depending what plan we have, we can integrate Alchemer with Google Sheet to have real-time updated spreadsheet, same feature available on CrowdSignal, and then we could share the link with the organizers (or whoever needs it). I'm waiting to get the login details so I can investigate more.

[malgrauk]

@peiraisotta Just wondering if it's worth suggesting that we use a tool which we already have available - I think on scale a separate tool could become expensive?

Due to restrictions with the WordCamp system, and to avoid large numbers of emails needing manual work, this year we've created surveys and forms in CrowdSignal for a wide range of items for WCEU (from applications for sponsors/volunteers/etc, registering for childcare or contributor day - and even specific ones capturing dietary requirements such as for the social event) and these all automatically connect to Google Sheets directly which the relevant internal team can view. CrowdSignal is part of Automattic and therefore no cost involved at all (at point of purchase) as we were otherwise looking at a pricey subscription based on the number of surveys and functionality we needed :)

If there are standard questions we can include in our flagship event surveys (as we agreed above any global surveys would not go to flagships to avoid duplication - see here: https://github.com/WordPress/wordcamp.org/issues/1344#issuecomment-2222575751) we can share these back with raw data if this would help with any large scale analysis?

[peiraisotta]

@malgrauk I agree on using a tool already available. The main issue we've faced with CrowdSignal is the lack of a quick multilingual option, which Alchemer provides. For local WordCamps, we'd love to have the same survey available in multiple languages and be able to collect all responses together. When we conducted the Meetup Surveys in 2022 in 14 languages using CrowdSignal, the translation process after collecting the responses and before analyzing them was very time-consuming. Automattic paid for an Alchemer license that was used for the 2023 Annual Survey, and it seems that we could use that license for event surveys as well. I'm still in the early stages of investigating this and will share my findings, along with the pros and cons, next week so we can discuss it further.

[peiraisotta]

From what I've been learning about Alchemer, it looks like the license already paid would be enough to cover WordCamp and Event post-event surveys. Like CrowdSignal, we could connect it to Google Sheets and share the real-time results link with organizers or anyone who needs to monitor them. The main blocker with CrowdSignal was the impossibility of providing a multilingual solution, which Alchemer provides. Questions can be translated and uploaded on Alchemer, and then respondents simply choose the language they would like to take the survey in from a drop-down list, with all responses stored in a single database (an option we don't have with CrowdSignal).

[peiraisotta]

More information about Alchemer. (Thank you @DanSoschin for providing them)

Accessibility

• Alchemer has a checklist you can follow to better ensure compatibility with screen readers and other accessibility tools.
• A comprehensive resource center for the creation of accessible surveys, including an automated tool that checks for issues and provides an actionable list of things to address.

Privacy & GDPR

• Robust policies in place concerning GDPR, privacy, and controls for transmitting data in encrypted manners. Survey responses can be anonymized as well.

For those who desire more information, here are two links:

• https://help.alchemer.com/help/gdpr-readiness-kit
• https://help.alchemer.com/help/alchemer-security-overview

Additional perk

• Sentiment analysis on free text responses

[dorsvenabili]

Thank you for the research, @peiraisotta!

Based on this discussion and extra info. I think we should go ahead with using Alchemer for creating a standard survey for all WP Events/WordCamps that can be available in as many languages as the community wants and that can be sent automatically after the events. Such automatic email should not be used for flagship events, which will include our standard questions in their customized and longer surveys as flagship events are more complex and require longer surveys.

Please react with 👍 if you agree with this path forward, thank you! :)

[peiraisotta]

Update: there is a chance that we can have multilingual surveys on CrowdSignal. I'm exploring it right now and will get back to you.

[malgrauk]

Agree with the approach above @dorsvenabili to use for non-flagships (but that they share relevant data back for standardised questions) but also note that there is a possibility of using CS where we'd hold/own the data as @peiraisotta mentions! 😄

[DanSoschin]

I believe that CrowdSignal supports translating UI strings but not survey strings.

I would recommend keeping things as simple as possible - eliminate steps and processes.

If more than one person is using Alchemer, we'll need to purchase additional licenses, so I am not confident it is a suitable solution for this distributed type of work being proposed here.

I might recommend looking at some plugins that could be added directly to the WordCamp site for running surveys. I'd keep the set of questions standardized so you can use a single survey for all events.

[peiraisotta]

@StevenDufresne @pkevan we're moving on with Jotform for attendee survey for WordCamps and WP Events (former next-gen).

Should I open a new issue to request to stop sending the currently automated survey emails to WP Events (former next-gen) attendees? We'll want them to fill out the same survey we've created for WordCamps.

FYI, I've created this issue to request a redirect from the older attendee survey link to a new one.

Please, let me know if you need any additional info from me. Thanks a lot for your help!

[pkevan]

Should I open a new issue to request to stop sending the currently automated survey emails to WP Events (former next-gen) attendees? We'll want them to fill out the same survey we've created for WordCamps.

Yes, probably best to avoid any confusion. It looks like it's only activated as a plugin on Event sites, and the data will still be available through the Feedback section.

[peiraisotta]

Thank you, @pkevan . I created this new issue: https://github.com/WordPress/wordcamp.org/issues/1363