GOLEM paper check-list

[gkirgizov]

This is the meta-issue tracking progress with GOLEM paper that introduces GOLEM framework with use-cases and adaptive features.

What's required from collaborators who add their use-cases

• Use-case description (a subsection in paper)
• Review of domain use case (a paragraph in Review)
• Experiment in their use-case on GOLEM+bandits (adaptive mutations) vs. baseline GOLEM (random mutations)

---

Pre-requisite PRs & Issues:

• [x] #101
• [x] #104
• [x] #124
• [x] #29
• [x] #159

Experiments:

• [ ] Adaptive mutations -- show that there's no need to expertly fine-tune mutations for tasks.
  • [ ] simple bandits proof-of-concept -- pruning unneeded mutations
  • [ ] simple bandits (vs baseline) proof-of-concept -- can have mutations with dynamic probabilities
  • [ ] simple bandits (vs baseline) real case -- molecule search
  • [ ] contextual bandits with simple graph features (vs simple bandits) -- molecule search
• [ ] (Optional) ReproductionController in Noisy domains -- show that this feature helps in real tasks

[gkirgizov]

Paper Focus:

• Modular approach (specification of anything), incl. Verification Rules;
• Bandits: adaptability per each domain (universal set of mutations; bandits that adapt them)
• (optionally) ReproductionController -- controls population & compensates for mistakes in real use cases

[gkirgizov]

Preliminary results on Simple bandits VS. Baseline GOLEM:

• no significant difference apart from that bandits start slower
• mutation probabilities to which converges bandit are very close to uniform. This means that either Bandit has not enough signal to learn, or that uniform probabilities are optimal (hardly), or that training procedure has some errors.

Need trying contextual bandits and offline pretraining.

Experiment setup: 10-15 minutes and 5-10 trials per case. Agent with UCB algorithm (alpha=1.25)