Complete the API to dynamically make prediction based on problems and get latest raw data for data scientist.

[DurianDan]

The API needs to have 2 routes that:

• Dynamically make prediction based on phase_id and prob_id:
  • Implement a function to create corresponding Predictor and SQLModel
  • The Predictor needs to dynamically load the right registered model from mlflow based on phase_id and prob_id
  • The SQLModel needs to save the raw data from the request to the PostgreSQL server
• Load latest raw data from postgres: The data scientist will send request to this route to get the latest data for researching and creating new model.

Notes for Data Scientist:

• When training a model: specify the python packages used to create the model in all requirements.txt files: deployment/pip_requirements/*requirements.py;

  • !!!Make sure the packages and their versions are matched both in the api server and the mlflow server.
  • So that the API can correctly run the loaded mlflow model

• when a model is ready to registered (for serving in the API): tutorial to register a model

  • Name the model with format: phase{phase_id}-prob{prob_id} E.g.: phase2-prob1
  • Must specify the stage as "production" or "development".
    • mlflow can load the latest model in a stage, when the stage is specified. details: get_latest_version
    • then the API will use the stage name and model name to load the latest mlflow model

CI/CD:

• check for similar python packages in deployment/pip_requirements/: fastapi.requirements.txt and mlflow.requirements.txt
  • every packages in mlflow server must be installed in the API server

[DurianDan]

• after the commit c266371ba0ead390acbdfabcf744656007a48243, todo next:
  • [ ] implement SQLModelsFactory in app/problems_manager/sql_models/factory.py
  • deployment:
    • [X] imlement mlflow service
    • [ ] test the ProblemsManager with phase1 problems and train code (provided by Mr.Hung).
    • [ ] API route to load latest raw data from sql server => For data scientist.
  • CI/CD:
    • [ ] check for similar packages in deployment/pip_requirements: fastapi.requirements.txt, mlflow.requirements.txt
    • [ ] every python packages in the mlflow server must be installed in the fastapi API