[FEATURE]: Trainspace Run CRUD Endpoints

[karkir0003]

Feature Name

CRUD Endpoints for Trainspace runs

Your Name

Karthik Subramanian

Description

Purpose

DLP has migrated the underlying data representation of training requests of users to a model called the Trainspace model. However, a problem is that Trainspace is designed to store metadata for a given user's training request. During the model training phase, there are several result files generated that get written to dlp-executions-bucket. In order to ensure a clear purpose behind each database table, we propose to create a separate DynamoDB table to store "trainspace runs".

Trainspace vs. Trainspace Run

Trainspace: The datatype that stores data on a user's training request Trainspace Run: The datatype that stores result files related to a user running a training request

Implementation Steps

1. We want trainspace run and trainspace to be separate concepts. In order to maintain good practices on database design, we will create a new DynamoDB table in SST called trainspace-run-db with the following schema:

• run_id (unique id denoting a trainspace run. This will be a primary key)
• trainspace_id (what trainspace request is used in the run. This will be a GSI)
• user_id (id of user initiating the trainspace run. This will be another GSI)
• run_timestamp (timestamp that denotes when a trainspace run starts)
• result_csv_uri (S3 URL for result csv file)
• model_pt_uri (S3 URL for model.pt file)
• onnx_uri (S3 URL for onnx file)
• confusion_matrix_uri (S3 URL for confusion matrix file)
• auc_roc_uri (S3 URL for auc_roc file)

2. Create the following endpoints in SST (that will live in API Gateway) a. GET trainspace run by trainspace id b. GET trainspace run by run id c. GET all trainspace runs by user id d. DELETE trainspace run by run id e. CREATE trainspace run f. UPDATE trainspace run by run id (we update individual run here, not overwrite all trainspace runs!)

3. Add calls to the new SST endpoints in the Django tabular.py file (since that's the current trainer we have right now)

   ### Tasks
   - [ ] https://github.com/DSGT-DLP/Deep-Learning-Playground/issues/995
   - [ ] https://github.com/DSGT-DLP/Deep-Learning-Playground/issues/996
   - [ ] https://github.com/DSGT-DLP/Deep-Learning-Playground/issues/997

[github-actions[bot]]

Hello @karkir0003! Thank you for submitting the Feature Request Form. We appreciate your contribution. :wave:

We will look into it and provide a response as soon as possible.

To work on this feature request, you can follow these branch setup instructions:

1. Checkout the main branch:

    git checkout nextjs

2. Pull the latest changes from the remote main branch:

    git pull origin nextjs

3. Create a new branch specific to this feature request using the issue number:

    git checkout -b feature-992

   Feel free to make the necessary changes in this branch and submit a pull request when you're ready.

   Best regards, Deep Learning Playground (DLP) Team