Add doc about synapse Submission/SubmissionStatus objects

[thomasyu888]

Should this doc live in evaluation queues?

[kelshmo]

@thomasyu888 I think this belongs in Evaluation Queue since it is specific to queue submission.

Check out our proposed docs re-design. We talk about what to do with Challenges in our meetings a lot - it would be great to get your take on how to improve the high level architecture of Challenge documentation.

[thomasyu888]

I think challenge docs and actually many things in synapse can be documented in two categories. Documentation of the building blocks, and then how we utilize those blocks. So the blocks would be:

• challenge Projects
• Wikis
• Teams
• Evaluation queues
• Submissions
• Access Requirements
• Discussion
• Type of challenges (Model to data...)

Then the utilization would be two different approaches

• Participation
• Administration

After reviewing the docs, I feel like the two challenge docs and communities in synapse are really how we can utilize all the components in the rest of the doc site. I almost feel like there should just be a "how we maximize synapse" section which includes the two challenge docs.

I tried my best to basically cut down these two docs to just use what synapse offers on a very high level and not get too complicated. What I mean by that is anyone who creates an evaluation queue can do syn.getSubmissionBundles(evalqueueid) to grab all the submissions in a received state and interact with it.

Still, the above docs do not actually tell users how we internally run challenges. Whether or not this should live in the doc site, we could have internal documentation for all of the challenge roles and what they do. For example:

Challenge infrastructure role, responsible for:

• creating/maintaining infra that interacts with evaluation queues
• creating challenge site
• monitor discussion form

Common ways that sage employee in this role would create / maintain infra:

• Documentation for challengeutils
• Documentation for simple scoring harness
• Documentation for synapse workflow orchestrator.

We could maybe even have: for super advanced synapse users:

Advanced challenge infrastructure ( I say advance because we string together many different synapse API calls)

[thomasyu888]

Perhaps the sections that string together all the synapse components really tells people "why they should use synapse"

[kelshmo]

@thomasyu888 can you resolve the merge conflicts? I assume you want to preserve the web example but I'm not sure.

[kdaily]

@thomasyu888 you just got back from vacation! After you dig yourself out of emails and other things, can you check out @kelshmo's suggestions and move this forward? Thank you!