This PR is a continuation of #970 with changes related to input file handling.
Few cosmetic changes included in the PR: File and Tool CIDs are displayed as File ID and Tool ID because they are SHA 256 hashes and no longer CIDs.
Important changes to note:
Datafiles are passed in the payload with the S3 URIs. So they are no longer CID/filename, but S3://bucket-name/path/to/file. There is a corresponding new column in the DataFile table to track these S3 URIs.
There is a custom RayJobResponse unmarshalling, to get uuid, pdb, metrics, and the rest of the files. The x and y axis for these metrics are specified in the tool manifest.
(Note) To-do in the next PR:
Tool migration to S3 from IPFS
Create Job UUID before ray job submission and send it with the request body
Fix the convexity side service logic to use this above UUID throughout all the sub services for a single job
Checkpoints at the moment throw error with unmarshalling json due to missing job uuid, which is the same issue mentioned above, which can be ignored for now. This doesn't affect the checkpoint plotting after the job returns with the response.
What type of PR is this?
Description
This PR is a continuation of #970 with changes related to input file handling. Few cosmetic changes included in the PR: File and Tool CIDs are displayed as File ID and Tool ID because they are SHA 256 hashes and no longer CIDs.
Important changes to note:
(Note) To-do in the next PR:
Related Tickets & Documents
This PR can be tested along with PR-111
Steps to Test
Test steps mentioned in PR-111 in Convexity Repo
To test, use this tool manifest: tools/ray/design_with_colabdesign.json