Feedback from @thomasyu888 to be considered in docs redesign noted in PR #643
I think challenge docs and actually many things in synapse can be documented in two categories. Documentation of the building blocks, and then how we utilize those blocks. So the blocks would be:
challenge Projects
Wikis
Teams
Evaluation queues
Submissions
Access Requirements
Discussion
Type of challenges (Model to data...)
Then the utilization would be two different approaches
Participation
Administration
After reviewing the docs, I feel like the two challenge docs and communities in synapse are really how we can utilize all the components in the rest of the doc site. I almost feel like there should just be a "how we maximize synapse" section which includes the two challenge docs.
I tried my best to basically cut down these two docs to just use what synapse offers on a very high level and not get too complicated. What I mean by that is anyone who creates an evaluation queue can do syn.getSubmissionBundles(evalqueueid) to grab all the submissions in a received state and interact with it.
Still, the above docs do not actually tell users how we internally run challenges. Whether or not this should live in the doc site, we could have internal documentation for all of the challenge roles and what they do. For example:
Challenge infrastructure role, responsible for:
creating/maintaining infra that interacts with evaluation queues
creating challenge site
monitor discussion form
Common ways that sage employee in this role would create / maintain infra:
Documentation for challengeutils
Documentation for simple scoring harness
Documentation for synapse workflow orchestrator.
We could maybe even have: for super advanced synapse users:
Advanced challenge infrastructure ( I say advance because we string together many different synapse API calls)
Feedback from @thomasyu888 to be considered in docs redesign noted in PR #643
Then the utilization would be two different approaches
After reviewing the docs, I feel like the two challenge docs and communities in synapse are really how we can utilize all the components in the rest of the doc site. I almost feel like there should just be a "how we maximize synapse" section which includes the two challenge docs.
I tried my best to basically cut down these two docs to just use what synapse offers on a very high level and not get too complicated. What I mean by that is anyone who creates an evaluation queue can do syn.getSubmissionBundles(evalqueueid) to grab all the submissions in a received state and interact with it.
Still, the above docs do not actually tell users how we internally run challenges. Whether or not this should live in the doc site, we could have internal documentation for all of the challenge roles and what they do. For example:
Challenge infrastructure role, responsible for:
Common ways that sage employee in this role would create / maintain infra:
We could maybe even have: for super advanced synapse users:
Advanced challenge infrastructure ( I say advance because we string together many different synapse API calls)