Error: Module has already been disposed

[talperetz]

I'm getting plenty of these

Module has already been disposed
    at TVMArray.getHandle (@mlc-ai_web-llm.js?v=32662908:5934:17)
    at Instance.setPackedArguments (@mlc-ai_web-llm.js?v=32662908:7140:45)
    at GrammarFactory.packedFunc [as fGrammarSMFromTokenTable] (@mlc-ai_web-llm.js?v=32662908:7207:16)
    at GrammarFactory.getGrammarStateMatcherFromTokenTable (@mlc-ai_web-llm.js?v=32662908:12754:17)
    at LLMChatPipeline.<anonymous> (@mlc-ai_web-llm.js?v=32662908:13119:90)
    at Generator.next (<anonymous>)
    at @mlc-ai_web-llm.js?v=32662908:990:67
    at new Promise (<anonymous>)
    at __awaiter (@mlc-ai_web-llm.js?v=32662908:972:10)
    at LLMChatPipeline.prefillStep (@mlc-ai_web-llm.js?v=32662908:13072:12)

[time2bot]

Same here. Since deleting the application data from the browser and trying to reload a model - any model - Windows task manager shows the start of the inference process (we see the onboard intel GPU starting to work at 100% for a few seconds) and then instead of text, we get a console error:

"Error: Unable to find a compatible GPU. This issue might be because your computer doesn't have a GPU, or your system settings are not configured properly. Please check if your device has a GPU properly set up and if your your browser supports WebGPU. You can also consult your browser's compatibility chart to see if it supports WebGPU. For more information about WebGPU support in your browser, visit https://webgpureport.org/ at Object. (index.js:782:27) at Generator.next () at E (index.js:25:62)

https://webgpureport.org/ shows:

webgpu appears to be disabled

WGSL language features:

---

packed_4x8_integer_dot_product
pointer_composite_access
readonly_and_readwrite_storage_textures
unrestricted_pointer_parameters

misc:

---

fallback adapter not supported
getPreferredCanvasFormat bgra8unorm

dedicated workers:
==================
webgpu API exists
requestAdapter(compat) failed
requestAnimationFrame successful
transferControlToOffscreen successful
OffscreenCanvas successful
CanvasRenderingContext2D successful

shared workers:
===============
webgpu API exists
requestAdapter(compat) failed
transferControlToOffscreen successful
OffscreenCanvas successful
CanvasRenderingContext2D successful

service workers:
================
webgpu API exists
requestAdapter(compat) failed
transferControlToOffscreen successful
OffscreenCanvas successful
CanvasRenderingContext2D successful

[time2bot]

I've updated my intel onboard GPU driver to the latest version using: Intel's online tool and it seems to have fixed the issue.

The https://webgpureport.org/ now shows:

1 high-performance

===================

adapter info:

---

architecture gen-9
description
device
vendor intel

flags:

---

isFallbackAdapter false

limits:

---

maxBindGroups 4 maxBindGroupsPlusVertexBuffers 24
maxBindingsPerBindGroup 1000
maxBufferSize 2147483648 (2gb)
maxColorAttachmentBytesPerSample 128
maxColorAttachments 8 maxComputeInvocationsPerWorkgroup 1024
maxComputeWorkgroupSizeX 1024
maxComputeWorkgroupSizeY 1024
maxComputeWorkgroupSizeZ 64
maxComputeWorkgroupStorageSize 32768 (32k)
maxComputeWorkgroupsPerDimension 65535 (64k)
maxDynamicStorageBuffersPerPipelineLayout 8 maxDynamicUniformBuffersPerPipelineLayout 10
maxInterStageShaderComponents 112
maxInterStageShaderVariables 28
maxSampledTexturesPerShaderStage 16
maxSamplersPerShaderStage 16
maxStorageBufferBindingSize 2147483644 (2gb)
maxStorageBuffersPerShaderStage 10
maxStorageTexturesPerShaderStage 8 maxTextureArrayLayers 2048 (2k)
maxTextureDimension1D 16384 (16k)
maxTextureDimension2D 16384 (16k)
maxTextureDimension3D 2048 (2k)
maxUniformBufferBindingSize 65536 (64k)
maxUniformBuffersPerShaderStage 12
maxVertexAttributes 30
maxVertexBufferArrayStride 2048 (2k)
maxVertexBuffers 8 minStorageBufferOffsetAlignment 256
minUniformBufferOffsetAlignment 256

features:

---

bgra8unorm-storage
depth-clip-control
depth32float-stencil8
float32-filterable
indirect-first-instance
rg11b10ufloat-renderable
shader-f16
texture-compression-bc
timestamp-query

WGSL language features:

---

packed_4x8_integer_dot_product
pointer_composite_access
readonly_and_readwrite_storage_textures
unrestricted_pointer_parameters

misc:

---

fallback adapter not supported
getPreferredCanvasFormat bgra8unorm

dedicated workers:
==================
webgpu API exists
requestAdapter successful
requestDevice successful
getContext("webgpu") successful
requestAnimationFrame successful
transferControlToOffscreen successful
OffscreenCanvas successful
CanvasRenderingContext2D successful

shared workers:
===============
webgpu API exists
requestAdapter successful
requestDevice successful
getContext("webgpu") successful
transferControlToOffscreen successful
OffscreenCanvas successful
CanvasRenderingContext2D successful

service workers:
================
webgpu API exists
requestAdapter successful
requestDevice successful
getContext("webgpu") successful
transferControlToOffscreen successful
OffscreenCanvas successful
CanvasRenderingContext2D successful

[nicmeriano]

I'm facing the same error but I'm noticing it's happening when I try to call the engine.chat.completions.create() function using different schemas or report_format options. In my case I'm using the same engine instance across the app so a potential solution might be to initialize different models per schema. Not sure if there's a better way.

// OK
await model.chat.completions.create({
  messages: messages,
  response_format: {
    type: "json_object",
    schema: schemaA,
  },
});

// OK
await model.chat.completions.create({
  messages: messages,
  response_format: {
    type: "json_object",
    schema: schemaA,
  },
});

// Error: Module has already been disposed
await model.chat.completions.create({
  messages: messages,
  response_format: {
    type: "json_object",
    schema: schemaB
  },
});

[CharlieFRuan]

This should be fixed by https://github.com/mlc-ai/web-llm/pull/571 and will be available in the next npm!

[CharlieFRuan]

@talperetz @time2bot @nicmeriano Should be fixed with npm version 0.2.66. Closing this for now, feel free to open another one if issue persists!