Investigate need for BaseModel class

[mhauru]

How much does it share with BaseIngress?

[EdwinB12]

181 summarizes a lot of the required features from the front ends perspective. This issue will contain more specific discussion about how we enable this through the backend. Also related to #81.

My initial thoughts are to split up the major tasks into model initialisation and model run. This could be implemented in a similar way to the BaseIngress class. Potentially, we have a BaseService class that BaseIngress and BaseModel inherits from. It may not even be necessary to have seperate Ingress and Model classes, but I suspect it might be due to different API endpoints with the backend.

The BaseService Class would handle:

• passing parameters to the service
• running the service
• scheduling the service

User would then inherit from BaseService to write the custom methods.

class BaseService():

def __init__(self):

def init_instance(**params):
"""
This method handles the ingestion of the parameters and saves the instance of the model/ingress in the database. 
Therefore, in the database, there can be multiple instances of models/ingress but with different parameters. 
"""

def run():
"""
This method runs the instance of a model/ingress immediately
"""

def schedule():
"""
This method schedules running the run method. This may have to be the method that requires some Azure interaction. 
I really don't know what this looks like to be honest.
"""

This would link up with the frontend, something like:

Choose Model from drop down list : Weather_Ingress

service = WeatherIngress()

Enter parameters : {'start': 'present', 'end': '2 hours'}

service.init_instance( {'start': 'present', 'end': '2 hours'})

Run Now

service.run()

Schedule : EveryDay

service.schedule(EveryDay)

[EdwinB12]

To Do for first PR on this topic:

Compulsory things to do

• [ ] Implement BaseService Class and get BaseIngress to Inherit
• [ ] Create BaseModel class that inherits from BaseService
• [ ] Move Arima over to BaseModel
• [ ] Check WeatherIngress still works when inheriting from BaseService Class

Extras

• [ ] Ensure user can override parameters, preferably from GUI
• [ ] Add a run now functionality via frontend
• [ ] Design Scheduler system. This will likely be a PR in its own right as this needs integration with AzureFunctions.

[mhauru]

I'm trying to understand how this sits around the backend/frontend and backend/functions divides. Presumably the backend would have endpoints for things like insert-service, insert-service-parameters, schedule-service, and run-service? And the backend would also have the scheduler running. Do I understand correctly that the BaseService class would sit on the frontend or functions side of the divide, and just call these end points? The same way BaseIngress is a bit of code that will run e.g. in an Azure function.

[EdwinB12]

These are the questions I've been asking myself.

• Yes backend would have those endpoints
• Basically, yes this would work exactly like BaseIngress. Really its a template for a developer to pick up and follow.
• I don't really understand our options for scheduling/Azure Functions to be honest, probabaly something we should chat about.

[mhauru]

I haven't thought about it in detail, but for the scheduling I'm imagining something where the backend server, the same one that runs the backend API, every time it starts it reads from the database information about what are all the scheduled services, and then sets up cron jobs for all of them. Where by cron jobs I don't mean literal cron jobs, but some sort of background process that does a thing at given regular times.

So I would imagine the flow being something like

1. The frontend, due to user pressing a button, sends a request to the schedule-service endpoint, with a payload of something like

   {
   "service_name": "Weather ingress",
   "parameter_set": {
   "longitude": 0.01
   "latitude": 53.2
   },
   "schedule": "Once a day at 1pm",
   }

   The schedule would be given in some less silly format of course.

2. The backend writes those parameters into a new row in a table called service_parameters, and writes a row in a table called service_schedules. The latter table would have three columns, one referencing which service this is a schedule for, one referencing a row in service_parameters, and one for the value of the schedule.
3. The backend, once done with the database writes, calls a function called something like update_scheduled_jobs, which goes and discovers in the table service_schedule that there is new scheduled service to run, and sets up a new cron job/timer to make sure that the right service is called with the right parameters at the right time.

The table for services would probably only have two columns, one for the name of the service ("Weather ingress"), and one for the URL of the endpoint to send a request to when calling that service (for weather ingress, the URL for the relevant Azure function, which triggers it to run). Maybe in the future a third column for a JSON schema for the parameters.

[EdwinB12]

closed by #190