[Bug] safari broken

[mikestaub]

Bug Description

On MacOS Safari Tech Preview and iOS Safari with the WebGPU feature flag enabled, the site doesn't load and there are no errors in the console. Navigating to webgpureport.org in both environments shows that WebGPU is in fact enabled.

Steps to Reproduce

Load chat.webllm.ai and see the stuck loading spinner

Expected Behavior

The site should load correctly

Screenshots

No response

Desktop OS

MacOS

Desktop Browser

Safari Tech Preview

Desktop Browser Version

Release 196

Smartphone Device

iPhone 13

Smartphone OS

Safari

Smartphone Browser

No response

Smartphone Browser Version

No response

Additional Logs

No response

[Neet-Nestor]

@mikestaub Is there any logs in your console?

[tqchen]

i think this is again issue of service worker not available but web worker being ok

[tqchen]

see a related issue in webkit https://bugs.webkit.org/show_bug.cgi?id=266793

[Neet-Nestor]

i think this is again issue of service worker not available but web worker being ok

I assume so so I would like to confirm the logs. Currently if service worker fails to load in 5s it should automatically fall back to web worker.

I will do testing on safari tech preview myself.

[mikestaub]

When I set the loglevel to 'TRACE' and reload the page the only thing I see in the console is Service Worker API is available and in use. before the app hangs. I can confirm the service worker is activated by setting a debug breakpoint, so it may be a different bug.

[Neet-Nestor]

Thanks for this, this is really helpful. If this is the only log printed, this means the webllm engine in service worker is not completely activated somehow (could be related to the issue of Safari mentioned above by tqchen).

I will need to have a deeper look into this case and make sure the webapp falls back to web worker.

[Neet-Nestor]

https://github.com/mlc-ai/web-llm-chat/commit/1cb52bf8351f420121f3238e685c8f27548a9323

@mikestaub I deployed a quick fix to force web worker fallback in 5 seconds timeout. Could you please try again whether this works for you now?

[mikestaub]

Yes, the app loads with that fix, thanks!

[mikestaub]

However, when I try to chat with stablelm-2-zephyr-1_6b-q4f32_1-MLC ( and all other tested models ) in Safari Tech Preview with the same settings that work well in Chrome, it gives incoherent responses. Will try to create an easy way to reproduce.

[Neet-Nestor]

In my case, it gives me the following error:

Error: Cannot initialize runtime because of requested maxBufferSize exceeds limit. requested=1024MB, limit=922MB. This error may be caused by an older version of the browser (e.g. Chrome 112). You can try to upgrade your browser to Chrome 113 or later.

This seems a webgpu issue of Safari for me. If you got similar errors, maybe better to move the issue to https://github.com/mlc-ai/web-llm/ instead.

[mikestaub]

Yes it is a known issue: https://github.com/mlc-ai/web-llm/issues/386