[Web]

[catsandsoup32]

Describe the issue

When exporting my model from torch as below and testing ORT in python, everything is fine, but I get an uncaught error when trying to run the model in ORT web in a chrome extension environment.

full_model = Model_1(vocab_size=217, d_model=256, nhead=8, dim_FF=1024, dropout=0, num_layers=3)
full_model.load_state_dict(torch.load('runs/Exp8E8End_Acc=0.6702922582626343.pt', map_location = device, weights_only=True))
full_model.eval()
seq_len = Dim("seq_len", min=1, max=200)
torch.onnx.export(
    full_model.to(device), 
    (torch.randn(1,3,384,512).to(device),
     torch.ones([1, 1], dtype=torch.long).to(device),
     torch.triu(torch.ones(1, 1) * float("-inf"), diagonal=1).to(device)
    ),
    "model_2.onnx",
    export_params = True,
    input_names = ["src", "tgt", "tgt_mask"],
    output_names = ["output1"],
    dynamo = True,
    dynamic_shapes = {"src": None, "tgt": {1: seq_len}, "tgt_mask": {0: seq_len, 1:seq_len}},
    report = True,
    optimize = True,
    verify = True
)

The markdown torch export success report

Thank you in advance for the help, any insight into what's going on would be great.

To reproduce

{
  "manifest_version": 3,
  "name": "TeXan",
  "version": "0.1",
  "description": "Handwriting to LaTeX",

  "icons": {
      "16": "icons/icon-16.png",
      "32": "icons/icon-32.png",
      "48": "icons/icon-48.png",
      "128": "icons/icon-128.png"
  },
  "action": {
    "default_title": "TeXan",
    "default_popup": "popup.html",
    "default_icon": {
        "16": "icons/icon-16.png",
        "32": "icons/icon-32.png",
        "48": "icons/icon-48.png",
        "128": "icons/icon-128.png"
    }
  },
  "sandbox": {
    "pages": ["popup.html"]
  },
  "web_accessible_resources": [
    {
      "resources": ["popup.html"],
      "matches": ["<all_urls>"]
    }
  ],
  "content_security_policy": {
    "sandbox": "sandbox allow-scripts; script-src 'unsafe-eval';" 
  }

// file test.js, I installed ORT through npm

async function testInference() {
    const session = await ort.InferenceSession.create('model_2.onnx');

    const src_array = new Float32Array(3*384*512); 
    const src_tensor = new ort.Tensor( 'float32', src_array, [1, 3, 384, 512] );

    const tgt_array = new BigInt64Array(2);
    const tgt_tensor = new ort.Tensor('int64', tgt_array, [1, 2] );

    const tgt_mask_array = new Float32Array(4);
    const tgt_mask_tensor = new ort.Tensor( 'float32', tgt_mask_array, [2, 2] );

    const test_output = await session.run(
        {src: src_tensor, tgt: tgt_tensor, tgt_mask: tgt_mask_tensor}
    );

}

testInference();

model_2.onnx.data file model_2.onnx file

Urgency

No response

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.20.0

Execution Provider

'xnnpack' (WebAssembly XNNPACK), 'wasm'/'cpu' (WebAssembly CPU)

[catsandsoup32]

Nevermind, not an issue. https://onnxruntime.ai/docs/tutorials/web/large-models.html