search

MIT-AI-Accelerator / c3po-model-server

Other
1 stars 1 forks source link

P1 mistral upload #165

Open emiliecowen opened 5 months ago

emiliecowen commented 5 months ago

Function: prepare_upload_summarization_model in src/ppg_utils/ppg_topic_summarization.py

Mistral file for upload in: https://mitlincolnlaboratory-my.sharepoint.us/:f:/g/personal/emilie_cowen_ll_mit_edu/EqGFmCrPCZdKhhXINuCCkfwBDcjFxPS7Z1siJwG-_-1cXQ?email=Daniel.Gwon%40ll.mit.edu&e=Hp7IhI

Note: you may get an error locally on step 1, if you have run init. Just retrieve the latest llm pretrained id from the get endpoint and proceed to step 2. you will need to run step 1 on p1, though.

Once the p1 upload is complete, initial_data.py needs to be updated similar to create the staging DB object, similar to what was done for gpt4all.

dygwon commented 4 months ago 
{
  "sha256": "14789fe0f3a1e9c7b1b92ef4d82d90e7d312f6fdda43b524067569c0795203d8",
  "model_type": "mistrallite.Q4_K_M.gguf",
  "use_base_model": false,
  "id": "306247c6-69a6-4f4f-8ab4-178e5d2e5946",
  "uploaded": true,
  "version": 34,
  "originated_from": "app"
}
dygwon commented 1 month ago

Trying to upload again and the server appears to be timing out.

Ticket asking P1 to upload the binaries: https://jira.il2.dso.mil/servicedesk/customer/portal/73/P1MDOHD-23734

P1 id { "sha256": "14789fe0f3a1e9c7b1b92ef4d82d90e7d312f6fdda43b524067569c0795203d8", "model_type": "mistrallite.Q4_K_M.gguf", "use_base_model": false, "id": "5bf07c55-5a74-43b9-8625-b7644c9b45ec", "uploaded": false, "version": 1, "originated_from": "app" }