Keep Marathon Match and Rapid Development Match forums open after challenge completes

[jmgasper]

In an effort for people to be able to learn from completed challenges, we are going to keep challenge forums open for these challenge types:

• Marathon Match
• Rapid Development Match

Once the challenge completes for these, we will:

• Leave the challenge forum unarchived
• Move the challenge from a private group to a publicly accessible parent category

[jmgasper]

@atelomycterus - I have a question out to Topcoder about how they want to handle the MM / RDM forums. If we can create the forums as public initially, in a public category, that'll make life easier, instead of having them be private and then moving them to public later.

[atelomycterus]

@jmgasper I think we can manage it as you have written (without moving, only change permissions). Need to clarify some requiremnts related to MM/RDM challenges to estimate the scope.

Permissions

The Marathon forums is public.
Need to manage permissions for nested ones:

• Private group created with custom permission by default, only participants have access
• a challenge is completed = > unset custom permission. The parent permissions (public) is used. TODO: Need to check what happens in Vanilla if custom permissions are unset. Does Vanilla remove all permissions from DB or change a flag only.

Forums

The top level looks like this:

• General(public)
• News (public)
• Learning (public)
• Challenges Forums (private) -- Development Forums --- Challenge1 --- Challenge2 -- Design Forums --- Challenge3
  -- Data Science Forums --- Challenge4
• Marathons Forums (public)

What about nested categories of Marathons Forums? Are there any nested subcategories like (Development Forums, etc)? Or does the structure look like this? Marathons Forums (public) -- challenge 1 (public,completed) -- challenge 2 (private,the challenge is in progress) -- challenge 3 (private, the challenge is in progress)

Challenge forum processor

Create all entities for MM/RDM challenges. Need to clarify what entities should be created for a MM/RDM challenge. For a challenge from Development track, the challenge forum processor creates entities from a template

• a private group,
• ‘Code Documents’, ‘Code Questions’ and ‘General Questions’ and one default discussion in each category.

For a challenge from Design track, it creates only one group and 3 discussions.

What structure (nested categories, discussions) should be created for MM/RDM challenges?

Are there specific Topcoder roles for MM/RDM challenges?

Vanilla changes

Need more details about the structure of MM/RDM challenges. There are some custom changes for ‘Challenge forums’ in model, view and controllers. Are there any differences between challenges from Dev/Design/etc tracks and MM/RDM challenges?

Thanks!

[jmgasper]

@atelomycterus - Thanks for this. I'll be in touch with specifics in a day or two.

[jmgasper]

@atelomycterus - Still haven't resolved this exactly with Topcoder. I'll keep you posted thanks.

[jmgasper]

@atelomycterus - Ok, here's what we're going to do:

1. I'm going to create a new Fun Challenges root
   2. Underneath Fun Challenges, I will create Rapid Development Matches (RDMs)
   3. Underneath Fun Challenges, I will create Marathon Matches (MMs)
2. The challenge forums will be public underneath each of those, so we'll need to build out the ability to have those be immediately public, when the challenge is create. Note that for these, since the challenges will be public, we won't have any group control on them, and we'll need to update the API and / or the challenge forum processor to know the difference between a regular challenge and a MM / RDM.

For your questions:

1. I think this clarifies the permissions questions. All forums will always be public - no archiving, unarchiving, etc...
2. If we can follow the same template format we use for the private challenges, that will be good and allow us to be flexible to future needs.
3. I don't think we need anything different for these - we can build in some code to the challenge forum processor to ignore member registration / permission granting for MM / RDM challenge types, since everyone will have access at all time

One more thing to think of:

• We'll need a way to migrate existing / past challenge forums from their private groups into the public areas.

[atelomycterus]

@jmgasper 1) Yes, if all forums always are public, it'll be easier to manage it. 2) Yes, sure. We can re-use it and follow the same template format. 3) Yes, we'll update the challenge forum processor. Could you provide payloads of MM/RDM challenges?

Some questions about existing / past challenge forums

1. Do we need to migrate any existing / past challenge forums from old Topcoder php forum (which existed before Vanilla forum)?

2. Migrating existing / past challenge forums in Vanilla: a. What permissions do users have to existing / past challenge forums? Can users add discusions/write comments if it's made in public? Or is there public and read-only access? Should we show some details from history (who participated in the challenge, etc.) ?

b. How many challenges should we migrate in Vanilla? 10+,100+, 300+ or more ? This can be done in different ways. For example, building in code/script using internal Vanilla API/methods or using Vanilla REST API.

Let me know if I should start working on the challenge forum processor. Perhaps some other tasks or the scope is not agreed with the Topocder team yet.

Thanks!

[jmgasper]

@atelomycterus - For your questions:

1. For now, we won't be migrating old Jive forums
2. They will be fully public, so users will be able to post to ask questions, etc.. . We won't show any history at this time

We will probably be migrating 100-200 challenges.

All of this has been agreed upon, so feel free to start, thanks!

[atelomycterus]

@jmgasper I'm working on it. Keep you updated.

[atelomycterus]

@jmgasper Please apply PR-https://github.com/topcoder-platform/challenge-forum-processor/pull/110. MM/RDM challenge has the 'Marathon Match'/Rapid Development Match' tag. For example, "tags":[ "Automated Testing", "Medium", "Rapid Development Match" ]. So if it has the Marathon Match/Rapid Development Match tag => a MM/RDM challenge will be created. The MM/RDM challenge is unarchived by default.

Configuration DEV/PROD

I've created all nested categories manually in DEV. 1) Marathon Matches (MMs)/Rapid Development Matches (RDMs) should be "Nested". Use the hard-coded category url code ('mms-forums' / 'rdms-forums') 3) Create all nested categories (Development Forums, etc). You can find out all url codes in template.json. For example, for RDMs:

Let's test creating MM/RD challenges in DEV.

[jmgasper]

@atelomycterus - Thanks, I've deployed this in dev.

Two issues:

1. I'm seeing this in the logs:

2. We're going to soon be updating marathon matches to their own challenge "type", so can we add an additional check for that as well? This would be for determining if a challenge is a marathon match. We'll either look at the tags, like we do now, or if the challenge is "type": "Marathon Match", or "type": "Rapid Development Match",, then we'll create the public forum in the appropriate spot

[atelomycterus]

@jmgasper Please apply PR-https://github.com/topcoder-platform/challenge-forum-processor/pull/111/.

1. I tested it in my local env and coudn't reproduce it. Your screenshot shows that the challange.action.resource.create event had been received. It looks like there were no tags because the challnge forum processor tried to create a regular challenge with a private group. It was failed because I made an issue. This issue has been fixed in this PR. Could you try to create a MM challenge after deploying and provide access to the logs? I'll investigate it futher.
2. Implemented.

Thanks!

[jmgasper]

@atelomycterus - Sorry for the delay. We ran into performance issues in dev that took awhile to track down. I'm still working on some updates to mitigate those - it seems to stem from the 10's of thousands of test challenges we have in our dev environment.

I have worked around things to the point where we should be able to better test this now with QA, thanks!

[jmgasper]

@atelomycterus - I'm not seeing this working in dev (challenges.topcoder-dev.com).

I'm using user TCConnCopilot / Appirio123

When I create a development challenge, with the Rapid Development Match tag, I'm seeing this in the logs:

My expectation is that we do not need a group for this, because we're just creating a new, public forum in Fun Challenges / Rapid Development Matches (RDMs). There's no need for a group, I don't think.

In addition, even though the challenge is tagged as Rapid Development Match, and that is being recognised by the challenge forum processor, I'm still seeing the challenge under Challenge Forums, instead of the public category:

https://vanilla.topcoder-dev.com/categories/f022eb10-179b-4b38-a65a-d1e419147679

Do I need to tweak anything for the setup here?

[jmgasper]

@atelomycterus - One more thing - when launching in the new Competitive Programming track, I see this in the logs and the forums aren't created. Can we just reuse the same template we have for when we detect the Marathon Match or Rapid Development Match tags to cut down on redundancy?

[atelomycterus]

@jmgasper Please apply PR - https://github.com/topcoder-platform/challenge-forum-processor/pull/112/.

Issues

1. with the Rapid Development Match

Yes, I see it didn't work as expected. It might have been created with a group because the first challenge event didn't have any tags. Tags might be added in the next event ("challenge.notification.update" topic). I tried to check logs in AWS but I couldn't connect to it. My Topcoder AWS key/secret was expired.

I need access to logs to analyze all challenge bus events. Could you provide new AWS key/secret? Or upload log items as a log file. It helps to find an issue out quickly.

2. 'Competitive Programming' was added in template.json. I've configured it in Dev. How it looks in my local env:

3) Events from resource topics ('challenge.action.resource.XXX') are processed correctly for MM/RDM challneges. In short, we ignore them.

Thanks!

[jmgasper]

1. I'll see if we can get you a new AWS key / secret, but I doubt that's going to be doable in the short term. I'll upload new logs as we test, thanks.

For the tags, they will only be added when the challenge is updated from NEW to DRAFT - they won't be on the initial creation, but part of the update.

Here's an example of a tagged one I tried today that went into the challenge forums instead of the public spot:

https://vanilla.topcoder-dev.com/categories/0cc24b2f-6ccf-4994-8232-6ff318ec8bb3

3. Ok, cool
4. Sounds good, thanks!

[jmgasper]

Log is attached. Note that I'm working with Topcoder to address the Gateway timeout seen. We have way too many test challenges in dev and they're hampering things a bit :(

log-events-viewer-result.csv

[jmgasper]

I created this one and it went to the correct spot: https://vanilla.topcoder-dev.com/categories/fce0127f-dc13-47c5-886a-5e1c4028214c

Note that the Fun Challenges on the home page aren't expandable for some reason? Do we need to tweak the setup of the navigation tree to fix that?

[atelomycterus]

@jmgasper Thanks for logs. Need to clarify some requirements.

1. The issue with 'JUSTIN TEST RDM FORUM 3' (https://vanilla.topcoder-dev.com/categories/0cc24b2f-6ccf-4994-8232-6ff318ec8bb3) The challenge forum processor received the first event ('challenge.notification.create') . The payload has the following attributes/properties type=challenge, tags = [] , status=New . It looks like a regular challenge. As a result , a group was created and this challenge went into the challenge forums instead of the public spot.

The challenge forum processor creates all Vanila entities: a. the 'challenge.notification.create' event b. the 'challenge.notification.update' event if there's no Vanilla group yet + the challenge's status is active.

Next, consider both cases

a. We can't manage it for RDM/MM challenge if (type="challenge", tags are empty , status="New"). It'll create a reqular challenge. If type="Marathon Match" or "type" = "Rapid Development Match" => No issues. It works as expected. This one was created and it went to the correct spot: https://vanilla.topcoder-dev.com/categories/fce0127f-dc13-47c5-886a-5e1c4028214c , type="Marathon Match"

b. I thought about skipping 'challenge.notification.create' event. But I checked log file and the tags 'Rapid Development Match' was only set in the second update event. So we can't just skip 'challenge.notification.create' events and build all entities on the first update event. We'll face again the same issue. Moreover, it also affects regular challenges (there are several 'challenge.action.resource.create' events processed after creating events).

It would be correct to create a challange with the right challenge type in the first step. I think we need to discuss it with Topcoder team.

2) it was timeout and discissons were not created for 'JUSTIN TEST MM FORUM 5'. Errors in logs: 1678312297097, { Error: cannot POST /api/v2/categories?accesstoken=va.HNbQh52LEsnvn6KQgJCmS10Zv-jAJ9M.xzFBcg.3yMpRqr (504) 1678312297097, at Response.toError (/challenge-forum-processor/node_modules/superagent/lib/node/response.js:98:13)

3) All public forums don't have nested subcategories, just discussions. So we need to tweak the main page. The Fun Challenges has the subcategories. Are you ok, if the subcategories are shown like this ?

Thanks!

[jmgasper]

@atelomycterus -

1. Thanks for that. Let me circle up with Topcoder and get back to you.
2. Yeah, I'm hoping we get someone to scale up our CPU / memory on the dev instance today
3. Yeah, let's do that, thanks!

[jmgasper]

@atelomycterus - After talking with Topcoder, we can leave out the processing for the tags now and just keep the processing for the challenge types / tracks being added, which is currently working, thanks. Can you look at the display issue in number 3 please? I think that'll get us over the line on this one.

[atelomycterus]

@jmgasper Please apply PR-https://github.com/topcoder-platform/forums/pull/698. I'll configure Fun challenge permisions in DEV. Thanks! From my local env:

[jmgasper]

@atelomycterus - Thanks - seems to be working. It's in QA right now.