alan-turing-institute / CROP

CROP is a Research Observation Platform
MIT License
25 stars 4 forks source link

Restructure repo #287

Closed nbarlowATI closed 1 year ago

nbarlowATI commented 2 years ago

Following on from #283, now that we understand the contents of the repo a bit better, it would be better to simplify/harmonise it. For example, the code for the GES and Arima models are quite hard to find (in versioning/something/something/) First suggestion (to be discussed/revised!) for top-level directories:

CROP
  |- core (or keep as __app__) - contains db schema, common functions to query db
  |- webapp - everything else for the flask app
  |- ingress_functions - everything else for getting external data into our db (including Azure function app code)
  |- model_functions - all code for running the GES and Arima models (including Azure function app code)
  |- merge the contents of `utils` and `misc_scripts` - maybe still call it `utils`, but we do already have several `utils.py` files (`__app__/crop/utils.py` and `webapp/utilities/utils.py` to name two).
mhauru commented 2 years ago

Yes please! I've been especially wanting to move the model code and separate core from model_functions. Might also move the app/Dockerfile somewhere else, it took me a long while to realise that it doesn't actually include the function apps, but rather only sets the environment in which they will be deployed.

Is the name __app__ some flask specific thing? I dislike the name, but I thought flask wanted it that way for some reason.

The ingress_functions deployment part might need something extra to be able to deal with the fact that some of the dependencies are not under the folder where host.json is. Shouldn't be undoable, I may even have already done this in the model functions.

mhauru commented 1 year ago

With the javascript dependencies cleaned up, I can think of the following restructuring items: 1) Move more code into the backend. Most notably database queries should all happen there. This mostly affects the front end code, but also probably GES a bit. 2) Maybe util_scripts might still use some cleaning up? I'm not quite sure what is there.

I'm tempted to close this and open a new issue for 1. 2 might be covered by the "refresh tests" issue? Could have a separate issue too. It's just that every time I look at the board thinking of what to do, when I see this I have to pause and go "what was it again that remained to be done here, or are we done with restructuring?" If you can think of other restructuring items, feel free to reopen this or create new issues.