For mapathons, would be great to set and track a shared goal in the missions

[jonfroehlich]

To support mapathons, would be great to set a goal for the mapathon and to have a tracker show progress towards this goal

[misaugstad]

@shokiami would love to hear some ideas for what this might look like on the website. Where would these goals be visible? Who would be able to see it? How would they be created? What would it look like?

[shokiami]

@misaugstad @jonfroehlich

Wow, this is a dope idea! I am going to use this post as an opportunity to rant a little. I just wanted to note that, when reading this issue, my mind immediately went to trying to find a solution that best cultivates productivity during mapathons, so everything I talk about is geared towards making that happen. I know this is probably a little extra, so please don't hesitate to let me know if this was way more than you imagined for this PR!

The first thing that came to mind was how we can implement these goals in a controlled fashion. What makes sense to me is treating mapathons like a round of a game. For example, in Among Us, a round of a game consists of all of the teammates working together to complete all of the given tasks (well, before they get killed by the imposters). Similarly, a mapathon could be an event hosted on the website itself during which participants work together to try to reach a common goal. This not only makes it easier to measure progress but also (hopefully) helps gamify the experience. Everything following is based on this design, and again, please let me know if you were envisioning something different!

So the first thing to address is what the start of a mapathon would look like. Who would be able to host a mapathon? Would it be select users or all users? I feel like if it was select users, we could limit this to just admins. Although I personally don't see the downside to opening this up to all users. When a user hosts a mapathon, they would select a specific region (or not?), and then they would receive a key/code that they could share with others (like in Among Us or Kahoot) so that they can join. Or do you think that these mapathons could be a public event where anyone could find a local mapathon and join? I personally feel that the private version would ensure that everyone is communicating (attending a mapathon over a call) and therefore would actually foster a more productive environment.

But most importantly, what even are the goals going to be? Should it be the same goal every time? Or should the goals ever change? Going back to this idea of implementing controlled goals, I feel that the gist of the goals should never change, but rather the bar to be met. What I mean by that is instead of trying to hit x every time, the goal should be to do better than before or to place higher on some sort of leaderboard (discussed later) like most games. Also, I feel that adding a time constraint would help make the goal more measurable, inspire efficiency, and also further gamify the mapathon. Here are some example goal formats:

• Get x done in as little time as possible.
• Get as much done as possible in x time.

Although time is easy to keep track of, how do we measure "how much we got done"? Is audited distance enough? Maybe the number of labels placed? Ideally, I feel that we use a combination of the two to calculate a score for the overall productivity of a given mapathon, as just using audited distance could lead to poorly labeled roads, and just using label count could lead to low ground coverage... although I don't know exactly what the equation to calculate the score would look like. During auditing, all users would be able to see the team score along with a time so that they can keep track of how they are doing on their current goal. Another question this begs is would this score be per person or cumulative. I feel it makes sense for it to be per person so larger teams don't have an advantage over smaller teams. However, if it is cumulative, maybe we should add a maximum participant cap. On the flip side though, if we do per person, maybe we should add a minimum participant cap so a mapathon can't be a single individual going ham. An extra thought I had that is less crucial but might be worth thinking about is considering the quality of auditing when calculating a score (that way people can't meet their goal by simply placing random labels everywhere and spamming the "move forward" button). For example, in the last portion of the mapathon, there could be time reserved specifically for validating each others' labels - a sort of peer review. Or maybe we can implement a less accurate method to estimate the general quality of labeling such as checking the number of labels per distance (too low likely means poor auditing) or measuring the variance in each individual's labeling (low variance in label type, severity, and/or tags could indicate a lack of thoroughness).

Finally, what happens when the mapathon ends? Going off of this idea of treating a mapathon like a round of a game, we could use this as a time to display results. The first thing we could show is a map to all of the users visualizing the impact they made. Next, for the grand reveal, the users would see their final score relative to previous mapathons in a sort of leaderboard. This is ultimately what would incentivize teams to work harder during mapathons. Also, another fun addition that could help promote some healthy competition could be a private leaderboard of the participants in that mapathon showing the top contributors. Rewards are a great way to motivate people to reach a goal, and these visuals and leaderboards are the best methods I can think of to do this, but if you have any other ideas, please let me know!

One extra thought I had which is completely irrelevant but I feel would make the mapathons more interactive and suuuper fun would be making auditing multiplayer somehow. How cool would it be where, as you are placing labels, you see your friend's user name jumping around you placing labels alongside you! Again, this is just something I thought of while typing all of this.

Rant over. Thanks so much for reading this lol 😅 sorry it was so long!

[misaugstad]

Who would be able to host a mapathon? Would it be select users or all users? I feel like if it was select users, we could limit this to just admins. Although I personally don't see the downside to opening this up to all users.

I think that the potential downside depends on what all changes on the website during a mapathon. Is it advertised on the landing page? Is it just on a user's dashboard? Does being part of a mapathon affect the behavior of the website in other ways? If it is just something that you see on your user dashboard, then I don't see the harm. But we should be careful if other things are changed as well. For example, if they can give their mapathon a name and it goes on the landing page, we need to be careful about that.

they would receive a key/code that they could share with others (like in Among Us or Kahoot) so that they can join. Or do you think that these mapathons could be a public event where anyone could find a local mapathon and join? I personally feel that the private version would ensure that everyone is communicating (attending a mapathon over a call) and therefore would actually foster a more productive environment.

I think that we should make them fully public with no key for now. If there is enough traffic on the site, then those who host mapathons may wish that the contributions could be restricted to people with a key. But I don't see us having that much traffic right now. If it becomes an issue, it's a feature that we could add, but I don't think we need it right away. I think the downsides of it are that users might forgot to put in the key/code, or might not know about it if they aren't on the call with everyone else, for example. Someone might start contributing and then realize they forgot to put in the key... Now do we need to write code that can go back and retroactively add their contributions to the mapathon after the fact?

I think this adds complexity, and the benefits to having private mapathons aren't worth it given our user base right now.

But most importantly, what even are the goals going to be? Should it be the same goal every time? Or should the goals ever change?

I was imagining there being options for goals. Some options I was considering:

1. Complete X miles in Y time
2. Complete neighborhood X in Y time
3. Complete X validations in Y time
4. Complete X miles and Y validations in Z time

I also like your idea of completing as much as possible in X time. I think that also lends itself to a sort of "mapathon week".

instead of trying to hit x every time, the goal should be to do better than before or to place higher on some sort of leaderboard (discussed later) like most games.

I like this idea too. From my experience at the Open Columbus mapathons, I feel like this would be motivating to people.

Although time is easy to keep track of, how do we measure "how much we got done"? Is audited distance enough? Maybe the number of labels placed? Ideally, I feel that we use a combination of the two to calculate a score for the overall productivity of a given mapathon, as just using audited distance could lead to poorly labeled roads, and just using label count could lead to low ground coverage...

I think to start, we should keep it simple. And that also has the benefit of users fully understanding what their goals are. I think that users expect something like distance as a goal. We might get better data if we use label count as a goal? But I would start with just distance for now.

During auditing, all users would be able to see the team score along with a time so that they can keep track of how they are doing on their current goal.

Agreed. Would love to see some mocks of what this might look like.

Another question this begs is would this score be per person or cumulative. I feel it makes sense for it to be per person so larger teams don't have an advantage over smaller teams. However, if it is cumulative, maybe we should add a maximum participant cap. On the flip side though, if we do per person, maybe we should add a minimum participant cap so a mapathon can't be a single individual going ham.

At least to start, I think that we should start with something simple and do a cumulative total with no participant cap. If other metrics become useful, we can adjust things in the future, but I don't see this being an issue right now.

An extra thought I had that is less crucial but might be worth thinking about is considering the quality of auditing when calculating a score (that way people can't meet their goal by simply placing random labels everywhere and spamming the "move forward" button). For example, in the last portion of the mapathon, there could be time reserved specifically for validating each others' labels - a sort of peer review. Or maybe we can implement a less accurate method to estimate the general quality of labeling such as checking the number of labels per distance (too low likely means poor auditing) or measuring the variance in each individual's labeling (low variance in label type, severity, and/or tags could indicate a lack of thoroughness).

Determining data quality is an ongoing issue that I don't think we can solve during the creation of mapathon events. One potential deterrent to adding low quality data might be to add in an accuracy number for the mapathon as a whole in the mapathon leaderboard?

Finally, what happens when the mapathon ends? Going off of this idea of treating a mapathon like a round of a game, we could use this as a time to display results. The first thing we could show is a map to all of the users visualizing the impact they made. Next, for the grand reveal, the users would see their final score relative to previous mapathons in a sort of leaderboard. This is ultimately what would incentivize teams to work harder during mapathons.

I like a lot of this, but I worry about implementation cost vs the benefits. I like the idea of the visualization of the contributions. Maybe we could make an endpoint that shows these results (I like the idea of the map visualizing the streets audited and labels placed), and then potentially have a list of mapathons that a user participated in on a user's dashboard. Entries in that list could hyperlink to this results page.

We could also have a mapathon leaderboard always visible. This way users can see their group climbing in the rankings during the mapathon. And they could check it whenever they wanted afterwards.

Also, another fun addition that could help promote some healthy competition could be a private leaderboard of the participants in that mapathon showing the top contributors.

I think this could go on that dedicated results page.

Rewards are a great way to motivate people to reach a goal, and these visuals and leaderboards are the best methods I can think of to do this, but if you have any other ideas, please let me know!

We could add more badges to the user dashboard! They could have badges for participating in mapathons! Maybe even a badge for being one of the top finishers in a mapathon.

One extra thought I had which is completely irrelevant but I feel would make the mapathons more interactive and suuuper fun would be making auditing multiplayer somehow. How cool would it be where, as you are placing labels, you see your friend's user name jumping around you placing labels alongside you! Again, this is just something I thought of while typing all of this.

I do think this is really fun! My only issue with it really is that people are generally pretty spread out. You would cross paths with someone else during a mapathon infrequently, which I think makes the difficulty of implementation not worth it :(

Rant over. Thanks so much for reading this lol sweat_smile sorry it was so long!

Thanks so much for sharing all your thoughts!! There was a ton of good stuff in there!! Let me know if you have followup stuff you want to go over. Otherwise I would love to see some mocks for some of these things!

[shokiami]

@misaugstad Hey Mikey, I am currently in the process of designing some mock-ups, and I was wondering what exactly you meant by mapathons being advertised on the user dashboard. Do you mean that all currently occurring mapathons show up on the user dashboard in a list? And were you envisioning that users could freely join from that page as well? Also, I was trying to draft the leaderboard functionality, and I realize it's kinda difficult if we allow for goal options. For example, can you really compare a team that audited x miles in z time and a team that validated y labels in z time? I think that for MVP it might be better to choose a single goal to start. How do you feel about the "Complete X miles and Y validations in Z time" goal? Or is there another goal that you think would cultivate more productivity? Please let me know. Thanks!

[misaugstad]

I was wondering what exactly you meant by mapathons being advertised on the user dashboard. Do you mean that all currently occurring mapathons show up on the user dashboard in a list? And were you envisioning that users could freely join from that page as well?

When I wrote that, I was only thinking about showing the mapathons that you are now (or were) a part of. Not that I'm against people joining from the user dashboard. We need to put a way to join somewhere. So I don't have a strong feeling either way right now.

Also, I was trying to draft the leaderboard functionality, and I realize it's kinda difficult if we allow for goal options. For example, can you really compare a team that audited x miles in z time and a team that validated y labels in z time? I think that for MVP it might be better to choose a single goal to start. How do you feel about the "Complete X miles and Y validations in Z time" goal?

I think that right now, the flexibility in which goal you can choose is probably more important than being able to have a leaderboard of mapathons.

[jonfroehlich]

I haven't read this full thread but the most important thing, imo, simply would be:

• How to add an organizational affiliation to your profile
• Create and display a leaderboard for organizations (rather than individuals). I propose simply adding this to the leaderboard page

Just doing this, would be huge.

I'm not sure adding in specific goal tracking, etc. is worth the dev effort as long as we do the above (because then folks can set whatever goals they want, as long as we're tracking/displaying them in the leaderboard).

[misaugstad]

I kind of like the idea of having an organizational affiliation (though people should be able to have multiple). This could also make it easier to email users that are part of an organization when there is an upcoming mapathon hosted by that organization.

Although people can set their own goals, I think that having a progress bar towards their goal(s) is really motivating (and is what this issue was originally about).

I do think that things were getting a little bit broad, and we should start a bit smaller, though; thanks for reminding us to start simple @jonfroehlich :grin:

• Maybe we start out with 3 potential goal options: X miles, X validations, or finish a neighborhood.
• I don't really have an opinion on whether or not we just add this to the leaderboard page. I think that a separate page that is meant just for that mapathon (containing a leaderboard and a progress bar for the goal) is also reasonable and not too much dev work
• I'm not really sure yet how we would want organizations to look... Would only be organizations that could create mapathons? How would someone create an organization? Do we need to approve them?

These are just some initial thoughts. I'm down to keep talking through ideas with both of you @shokiami and @jonfroehlich

[shokiami]

@misaugstad I realize I don't even fully know what we are referring to by "organizations". Could you please provide me with a few examples of organizations? Also, have mapathons always been held with members from a single organization? Or is there ever mixing?

[jonfroehlich]

Organization can be anything we want from a company to a non-profit to 2-3 people who just want to form their own team.

IMO, we simply have a blank to fill out in the user profile called "Team name" or something like that and we just do string matching to group folks together.

Jon

On Wed, Feb 10, 2021 at 12:31 AM Shosuke Kiami notifications@github.com wrote:

@misaugstad https://github.com/misaugstad I realize I don't even fully know what we are referring to by "organizations". Could you please provide me with a few examples of organizations? Also, have mapathons always been held with members from a single organization? Or is there ever mixing?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ProjectSidewalk/SidewalkWebpage/issues/2338#issuecomment-776535347, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAML55ISMZLWWN2ULLWKEBDS6I77PANCNFSM4S5Z2FTQ .

[misaugstad]

@jonfroehlich If we do something like that, I'm not sure what we do for people who attend mapathons for multiple organizations (e.g., myself).

[jonfroehlich]

But don't you have per-city profiles? And you can always change your organization. I also think you're an edge case.

On Thu, Feb 11, 2021 at 12:45 PM Mikey Saugstad notifications@github.com wrote:

@jonfroehlich https://github.com/jonfroehlich If we do something like that, I'm not sure what we do for people who attend mapathons for multiple organizations (e.g., myself).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ProjectSidewalk/SidewalkWebpage/issues/2338#issuecomment-777777882, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAML55NLUE4TSMTDI6UARU3S6Q6VPANCNFSM4S5Z2FTQ .

-- Jon E. Froehlich https://jonfroehlich.github.io/ (he/him https://www.mypronouns.org/he-him) | @jonfroehlich https://twitter.com/jonfroehlich Associate Professor, Allen School of Computer Science & Engineering Core Faculty, Urban Design and Planning, College of Built Environments Director, Makeability Lab https://makeabilitylab.cs.washington.edu/ | Associate Director, CREATE http://create.uw.edu/ University of Washington Help make sidewalks more accessible: http://projectsidewalk.io

[misaugstad]

That's fair

[shokiami]

@misaugstad @jonfroehlich

Sorry, I am still a little confused about a few things:

1. Could anyone host a mapathon then? And if someone starts a mapathon, do other users from the same organization automatically get a link to join the mapathon? Where does this link/button appear? Do offline users get notified?

2. Are you sure we want users to be able to freely join and change organizations? Shouldn't there be some sort of invitation mechanism like Slack or GitHub? But at the same time, I feel like this is another layer of complexity I feel we shouldn't be worrying about right now.

3. Jon, you were mentioning adding a leaderboard of organizations to the leaderboard page, and Mikey, I was talking to you earlier about how it is hard to have a single leaderboard if there are multiple different goal options. I'm lost on what this is going to look like. Should there be a leaderboard for each goal option? Or maybe there could be a singular leaderboard that takes all three goals into consideration when ranking teams.

4. Also, Mikey, I was confused about the goals you suggested. You said, "X miles, X validations, or finish a neighborhood." Do you mean in the shortest time possible? Also, do you think this would rush users during auditing and potentially lead to poor quality data?

Please let me know what you think or if anything I said didn't make sense. Thanks!

[jonfroehlich]

I didn't have a specific frame of reference for mapathons in my comments, so perhaps my thoughts should be in a different issue. I know from talking to partners that they want leaderboards not just for individuals but for organizations. This is particularly important as we begin partnering with schools and other community organizations.

In those cases, I don't think "choosing a shared goal" is important. Instead, I think it's important to:

• Allow users to quickly and simply type in or otherwise select their organization
• See their organization's performance on the leaderboard. At a minimum, this can be just like the individual leaderboard but the left column has organization rather than an individual's username

I consider these things more important than specific "mapathon" support because these features are more broadly applicable. In addition, if we started with the above, I imagine it could be extended to have specific mapathon support with goal settings.

[jonfroehlich]

Wheelmap has a nice UI to show team performance during/after mapathons:

Screenshot from a panel I participated on with Wheelmap at Intra-American Development Bank