How to structure codebase

[jmhvandoorn]

Let's discuss here how we want to structure our code base. I will propose something, you shoot :)

As main layout:

Euro-NeurIPS-2022 |--- .github/workflows |--- data |--- dynamic |--- hgs_vrptw |--- instances |--- notebooks |--- static -- LICENSE.md -- README.md -- analysis.py -- benchmark.py -- benchmark_dynamic.py -- controller.py -- create_submission.sh -- environment.py -- plotting.py -- poetry.lock -- pyproject.toml -- setup.cfg -- solver.py -- tools.py

It would actually be my preference to move any file that cannot be directly called as a script out of the main folder (e.g. plotting.py, tools.py, environment.py)

However, not sure if/how we could do that, without messing with code we are not allowed to mess with.

---

We then add a directory that handles the static solves:

static -- init.py -- solver.py

Where solver.py is the only place in all of the code that imports hgspy and contains a method "solve_static" that builds a (customizable) solver. The init exposes this method and exposes all operator (names)s or other things that might be needed as input for this method.

---

And one directory for the dynamic solves

dynamic |--- strategies -- init.py -- solver.py -- utils.py

Solver.py contains three functions:

• ("private") solve_epoch that wraps around static.solve_static and handles input/output
• ("public") solve_dynamic that solves an entire dynamic instance
• ("public") run_oracle that uses dynamic.solve_static to solve the hindsight problem of a dynamic instance.

The init exposes the "solve_dynamic" method as well as all strategies to choose from

---

With a subdirectory for all strategies. (Which I think is a lot cleaner than just putting them in dynamic directly)

strategies |--- rollout -- init.py -- baselines.py

Where baselines.py contains methods for the greedy, random and lazy baselines (+ one generic private method that is used by all three of them).

init exposes all strategies to the "dynamic" level.

---

rollout -- init.py -- algorithms.py -- config.?? -- simulate.py

One file with a method for each different rollout algorithm. One file with a method for simulating an instance and one that wraps around the static solver.

---

I would then propose having a config file in static, dynamic and dynamic/strategies/rollout where we put all the solver configurations we use, that are overwritten in a hierarchical fasion. But let's discuss how we want to handle configuration afterwards / seperately as soon as we have the codebase structure clear.

[N-Wouda]

It would actually be my preference to move any file that cannot be directly called as a script out of the main folder (e.g. plotting.py, tools.py, environment.py)

However, not sure if/how we could do that, without messing with code we are not allowed to mess with.

I was thinking about this because I have yet another script to place somewhere, for tuning. In most of my projects, there's almost nothing in the top-level directory. Some config and documentation files, but not much more.

I typically put scripts away in a script/ directory (the actual name can vary, scripts/ is just a go-to). I call a script benchmark.py in such a folder as python -m script.benchmark <arguments> from the project root. We could do something similar?

I do not think we can move controller.py, environment.py and tools.py as many things depend on those being in the project root. But our own code certainly can be moved.

[N-Wouda]

@jaspervd96 I think we have 60-70% of the above in main now. Not all, but most. The only thing we might still want to do is move executable entrypoints out of the root - but that's not a huge priority for me, at least.

[N-Wouda]

I think we can close this issue now, since we're not going to change much about the current structure anymore in the next two days anyway.