bug: Models Import API

[gabrielle-ong]

Cortex version

v1.0.1-204

Describe the Bug

Discord feedback from Lumi: https://discord.com/channels/1107178041848909847/1148534827390341150/1301169811933368341

1. Missing API Reference for POST /models/import
2. request: model is ignored (should be saved name? - should this be modelHandle, modelID or model?
3. response: modelHandle is empty - should be the model name that is used to save the model
4. ModelDTO has many additional fields
5. Missing API Reference for POST /models/import

export interface ModelDto {
    // Existing Fields
    id: string;
    cache_type?: string;
    caching_enabled?: boolean;
    cpu_threads?: number;
    ctx_len?: number;
    engine?: string;
    flash_attn?: boolean;
    frequency_penalty?: number;
    grammar_file?: string;
    grp_attn_n?: number;
    grp_attn_w?: number;
    max_tokens?: number;
    mlock?: boolean;
    n_batch?: number;
    ngl?: number;
    pre_prompt?: string;
    presence_penalty?: number;
    prompt_template?: string;
    stop?: string[];
    stream?: boolean;
    temperature?: number;
    top_p?: number;
    use_mmap?: boolean;

    // the fields that were not mentioned in the API doc.
    ai_template?: string;
    created?: number;
    dynatemp_exponent?: number;
    dynatemp_range?: number;
    files?: string[];
    gpu_arch?: string;
    ignore_eos?: boolean;
    min_keep?: number;
    min_p?: number;
    mirostat?: boolean;
    mirostat_eta?: number;
    mirostat_tau?: number;
    model?: string;
    n_parallel?: number;
    n_probs?: number;
    name?: string;
    object?: string;
    os?: string;
    owned_by?: string;
    penalize_nl?: boolean;
    precision?: string;
    quantization_method?: string;
    repeat_last_n?: number;
    repeat_penalty?: number;
    seed?: number;
    system_template?: string;
    text_model?: boolean;
    tfs_z?: number;
    top_k?: number;
    typ_p?: number;
    user_template?: string;
    version?: string;
}

Steps to Reproduce

No response

Screenshots / Logs

No response

What is your OS?

• [ ] MacOS
• [ ] Windows
• [X] Linux

What engine are you running?

• [X] cortex.llamacpp (default)
• [ ] cortex.tensorrt-llm (Nvidia GPUs)
• [ ] cortex.onnx (NPUs, DirectML)

[LumiWasTaken]

A bit of insight from my side after originally reporting it:

I have pulled the most recent version and actually decided to re-try since it seemed odd to me.

And as it turns out, it actually works now!

So the issues are boiled down to:

• Missing API Reference for POST /models/import
• should "model" in the importModel param be renamed to be modelHandle, modelID or id (the current convention shows it to be 'id'
• ModelDTO has many additional fields

[dan-homebrew]

@gabrielle-ong I'd like you to own this ticket first and add docs. Once the Docs are clear, we can have a discussion with @namchuai next week to figure out the DTO naming fields