Images failing to upload to JS Client

[branaway]

Describe the bug

https://huggingface.co/spaces/InstantX/InstantID

I was trying to invoke this endpoint with both the Python client and the JavaScript client. The Python client works fine and the JS client doesn't work. I received an error message without much detail:

{ type: 'status', endpoint: '/generate_image', fn_index: 2, time: 2024-05-28T17:59:56.842Z, queue: true, message: null, stage: 'error', code: undefined, success: false }

Have you searched existing issues? 🔎

• [X] I have searched and found no existing issues

Reproduction

import { Client, FileData } from "@gradio/client";
import path from 'path'
import fs from 'fs/promises'

try {

const response_0 = await fetch("https://instantx-instantid.hf.space/--replicas/tej4q/file=/tmp/gradio/c70974d8881709b5dc254d7f61f70c4a2c1cf27a/sam_resize.png");
const userImg = await response_0.blob();

const response_1 = await fetch("https://instantx-instantid.hf.space/--replicas/tej4q/file=/tmp/gradio/71e330ef57fdbb85c07361f1bdbc3aba366c5cea/pose4.jpg");
const refImg = await response_1.blob();

    // const userBuffer = await fs.readFile(path.join(process.cwd(), 'imgs/users/OIP.HtnbVUj5aYlTxYh8-eV2eQHaJ3.jpeg'));
    // //  // Read the image file from the local file system
    // const userImg = new Blob(
    //  [userBuffer],
    //  { type: 'application/octet-stream' }
    // )

    // const refbuffer = await fs.readFile(path.join(process.cwd(), 'imgs/styles/1ac930a169eab44e709b78f7009cc98b.jpeg'));
    // // Repeat for the second image if necessary
    // const refImg = new Blob(
    //  [refbuffer],
    //  { type: 'application/octet-stream' }
    // )

    const app = await Client.connect("https://instantx-instantid.hf.space/--replicas/5s9fg/");

    const api_info = await app.view_api();

    // console.log(JSON.stringify(api_info, null, 2))

    const sub = app.submit("/generate_image", [
        userImg,
        refImg,
        "a nice photo", // string  in 'Prompt' Textbox component        
        "", // string  in 'Negative Prompt' Textbox component       
        "(No style)", // string  in 'Style template' Dropdown component     
        30, // number (numeric value between 1 and 100) in 'Number of sample steps' Slider component        
        0.6, // number (numeric value between 0 and 1.5) in 'IdentityNet strength (for fidelity)' Slider component      
        0.3, // number (numeric value between 0 and 1.5) in 'Image adapter strength (for detail)' Slider component      
        0.4, // number (numeric value between 0 and 1.5) in 'Pose strength' Slider component        
        0, // number (numeric value between 0 and 1.5) in 'Canny strength' Slider component     
        0, // number (numeric value between 0 and 1.5) in 'Depth strength' Slider component     
        ["pose"], // undefined  in 'Controlnet' Checkboxgroup component     
        5, // number (numeric value between 0.1 and 20.0) in 'Guidance scale' Slider component      
        123123, // number (numeric value between 0 and 2147483647) in 'Seed' Slider component       
        "EulerDiscreteScheduler", // string  in 'Schedulers' Dropdown component     
        false, // boolean  in 'Enable Fast Inference with LCM' Checkbox component       
        true, // boolean  in 'Enhance non-face region' Checkbox component
    ])
        .on("data", (data) => console.log(data))
        .on("status", (status) => console.log(status));

    // console.log(result);
}
catch (e) {

    console.error('error;', e)
}

Screenshot

No response

Logs

No response

System Info

The freshly installed Gradio client module with yarn.

Severity

Blocking usage of gradio

[branaway]

By the look of the http sniffing captures, the images were never picked up by the client and uploaded. Null paths were sent in place the blob.

[abidlabs]

cc @hannahblair as I believe you are currently looking into the file handling related issues for the js client

[branaway]

I was able to work around the initial hurdle by directly reference a already uploaded image using {path: '/tmp/gradio/.../image.png'} as the image input parameter and the interference was able to progress to the finish and returned me a path of the supposedly result image. However, a 404 raised when I tried to access it. Again the Python client worked fine.

The http sniffer showed that there are lots of heartbeats packs been sent to the server for the Python client, while there was none in case of the JavaScript client.

Update: That was until I upgraded the client to 0.20 from 0.19.4. The latest client refused to run with:

error; Error: There is no endpoint matching that name of fn_index matching that number. at get_endpoint_info (file:///Users/bran/localProjects/hf-instantID/node_modules/@gradio/client/dist/index.js:1798:11) at Client.submit (file:///Users/bran/localProjects/hf-instantID/node_modules/@gradio/client/dist/index.js:1233:51) at file:///Users/bran/localProjects/hf-instantID/gen.js:79:18

[branaway]

I like the event model the js client offers process updating after invoking the submit method on the client, better then the Python model in which case I need to keep polling to find out the progress ( or is it?)

[jonigata]

Same issue. When I read get_endpoint_info, I found that fn_index was undefined. This was because api_map was set to { predict: undefined, js_fn: undefined, js_fn_1: undefined }. When I looked into where api_map was being created, I saw the following code in map_names_to_ids:

fns.forEach(({ api_name, id }) => {
  if (api_name)
    apis[api_name] = id;
});

At this point, the content of fns (from config.dependencies) was as follows, indicating there was no id:

"dependencies": [
  {
    "targets": [
      [
        10,
        "click"
      ],
      [
        0,
        "submit"
      ],
      [
        1,
        "submit"
      ]
    ],
    "inputs": [
      0,
      1
    ],
    "outputs": [
      2
    ],
    "backend_fn": true,
    "js": null,
    "queue": null,
    "api_name": "predict",
    "scroll_to_output": false,
    "show_progress": "full",
    "every": null,
    "batch": false,
    "max_batch_size": 4,
    "cancels": [],
    "types": {
      "continuous": false,
      "generator": false
    },
    "collects_event_data": false,
    "trigger_after": null,
    "trigger_only_on_success": false,
    "trigger_mode": "once",
    "show_api": true,
    "zerogpu": true
  },
  ...
]

[Amishmaqbool]

I am having same issue when trying to uload an image in node enviourment so later i can connect my node env to react native