marek-mihok commented 4 months ago

Wave 1.1.0 pre-release QA

Test results

Unit tests

All pass

Visual regression tests

As expected, only minor (barely noticeable) changes were introduced: left-side cells in markdown tables has no left padding anymore and links are little bit bolder.

Manual testing

Components changed since 1.0.0 were tested.

ui.audio_annotator ❌ ui.chatbot ✅ ui.card_menu ✅ ui.inline ✅ ui.image_annotator ✅ ui.link ✅ ui.markdown ✅ ui.notification_bar ✅ ui.picker ✅ ui.spin_box ✅ ui.stats ✅ ui.table ✅ ui.tall_stats ✅ ui.textbox ✅ ui.toolbar ❌

Audio annotator

Timeline works inconsistently across browsers, different files and file formats on Safari and Firefox:

While trying aac sample, cursor stucks few seconds before end on Safari and jumps to the end on Firefox:

https://github.com/h2oai/wave/assets/23740173/6380f24f-4220-48aa-864e-cd6d0a3f1ca9

It gets even more interesting for flac files. The music is still playing, but the cursor is jumping on one place until the music ends. This was reproduced only for Safari:

https://github.com/h2oai/wave/assets/23740173/47af5498-73ba-427f-af47-6d72f216692a

However when trying different aac file, issue persists for Safari only.

Ogg file was working for Chrome and Firefox, however on Safari it couldn't be decoded:

The mp4 file cannot be played on Safari:

https://github.com/h2oai/wave/assets/23740173/eb9a4545-0c99-45c6-a881-faa64653ecb7

For m4a, wav and mp3 files I was not able to reproduce any of the issues.

Audio annotator not properly loaded when being on different OSX desktop which is not currently displayed on any of the monitors.

https://github.com/h2oai/wave/assets/23740173/fcbf7aaa-92c1-4548-9f4d-61c8829a902f

@mturoci we had this kind of issue in past with ui.plot

Both wave server and app crashed while playing around with audio annotator in multiple browsers:

Logs from waved server:


...
2024/02/26 13:39:20 # {"addr":"127.0.0.1:57534","route":"/900f344a-5b9c-4500-a7be-5b5fd1c484b6","t":"ui_add"}
2024/02/26 13:39:20 * /900f344a-5b9c-4500-a7be-5b5fd1c484b6 {"d":[{"k":"example","d":{"view":"form","box":"1 1 -1 -1","items":[{"audio_annotator":{"name":"annotator","title":"Drag to annotate","path":"/_f/8a239a20-8c90-42a5-82dd-a3c3b1441452/audio.mp3","tags":[{"name":"f","label":"Flute","color":"$blue"},{"name":"d","label":"Drum","color":"$brown"}]}},{"button":{"name":"submit","label":"Submit","primary":true}}]}}]}
2024/02/26 13:39:20 # {"path":"/_f/8a239a20-8c90-42a5-82dd-a3c3b1441452/audio.mp3","t":"file_download"}
2024/02/26 13:39:22 # {"addr":"127.0.0.1:57521","t":"ui_drop"}
panic: concurrent write to websocket connection

goroutine 280 [running]: github.com/gorilla/websocket.(messageWriter).flushFrame(0x14000696e78, 0x1, {0x10306f278?, 0x140002e2f00?, 0x376de3a3ef6?}) /Users/mmihok/go/pkg/mod/github.com/gorilla/websocket@v1.4.2/conn.go:610 +0x46c github.com/gorilla/websocket.(Conn).WriteMessage(0x140000f58c0, 0x484393777f?, {0x10306f278, 0x0, 0x0}) /Users/mmihok/go/pkg/mod/github.com/gorilla/websocket@v1.4.2/conn.go:763 +0xf4 github.com/h2oai/wave.(Client).flush(0x140001564d0) /Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/client.go:249 +0x26c created by github.com/h2oai/wave.(SocketServer).ServeHTTP /Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/socket.go:99 +0x604 exit status 2 make: *** [run] Error 1 mmihok@Mareks-MacBook-Pro wave %


- Logs from `audio_annotator.py` app:

INFO: 127.0.0.1:57535 - "POST /disconnect HTTP/1.1" 200 OK Unhandled exception Traceback (most recent call last): File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpx/_transports/default.py", line 69, in map_httpcore_exceptions yield File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpx/_transports/default.py", line 373, in handle_async_request resp = await self._pool.handle_async_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 216, in handle_async_request raise exc from None File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 196, in handle_async_request response = await connection.handle_async_request( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpcore/_async/connection.py", line 101, in handle_async_request return await self._connection.handle_async_request(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpcore/_async/http11.py", line 143, in handle_async_request raise exc File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpcore/_async/http11.py", line 113, in handle_async_request ) = await self._receive_response_headers(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpcore/_async/http11.py", line 186, in _receive_response_headers event = await self._receive_event(timeout=timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpcore/_async/http11.py", line 224, in _receive_event data = await self._network_stream.read( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpcore/_backends/anyio.py", line 31, in read with map_exceptions(exc_map): File "/opt/homebrew/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/contextlib.py", line 155, in exit self.gen.throw(typ, value, traceback) File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions raise to_exc(exc) from exc httpcore.ReadError

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/h2o_wave/h2o_wave/server.py", line 360, in _process await self._handle(q) File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/examples/audio_annotator.py", line 35, in serve await q.page.save() File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/h2o_wave/h2o_wave/core.py", line 634, in save await self.site._save(self.url, p) File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/h2o_wave/h2o_wave/core.py", line 887, in _save res = await self._http.patch(_rebase(_config.hub_address, url), content=patch) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpx/_client.py", line 1966, in patch return await self.request( ^^^^^^^^^^^^^^^^^^^ File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpx/_client.py", line 1574, in request return await self.send(request, auth=auth, follow_redirects=follow_redirects) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpx/_client.py", line 1661, in send response = await self._send_handling_auth( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpx/_client.py", line 1689, in _send_handling_auth response = await self._send_handling_redirects( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpx/_client.py", line 1726, in _send_handling_redirects response = await self._send_single_request(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpx/_client.py", line 1763, in _send_single_request response = await transport.handle_async_request(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpx/_transports/default.py", line 372, in handle_async_request with map_httpcore_exceptions(): File "/opt/homebrew/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/contextlib.py", line 155, in exit self.gen.throw(typ, value, traceback) File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpx/_transports/default.py", line 86, in map_httpcore_exceptions raise mapped_exc(message) from exc httpx.ReadError Failed transmitting unhandled exception Traceback (most recent call last): File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpx/_transports/default.py", line 69, in map_httpcore_exceptions yield File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpx/_transports/default.py", line 373, in handle_async_request resp = await self._pool.handle_async_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 216, in handle_async_request raise exc from None File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 196, in handle_async_request response = await connection.handle_async_request( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpcore/_async/connection.py", line 99, in handle_async_request raise exc File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpcore/_async/connection.py", line 76, in handle_async_request stream = await self._connect(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpcore/_async/connection.py", line 122, in _connect stream = await self._network_backend.connect_tcp(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpcore/_backends/auto.py", line 30, in connect_tcp return await self._backend.connect_tcp( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpcore/_backends/anyio.py", line 112, in connect_tcp with map_exceptions(exc_map): File "/opt/homebrew/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/contextlib.py", line 155, in exit self.gen.throw(typ, value, traceback) File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions raise to_exc(exc) from exc httpcore.ConnectError: All connection attempts failed

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/h2o_wave/h2o_wave/server.py", line 372, in _process await q.page.save() File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/h2o_wave/h2o_wave/core.py", line 634, in save await self.site._save(self.url, p) File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/h2o_wave/h2o_wave/core.py", line 887, in _save res = await self._http.patch(_rebase(_config.hub_address, url), content=patch) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpx/_client.py", line 1966, in patch return await self.request( ^^^^^^^^^^^^^^^^^^^ File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpx/_client.py", line 1574, in request return await self.send(request, auth=auth, follow_redirects=follow_redirects) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpx/_client.py", line 1661, in send response = await self._send_handling_auth( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpx/_client.py", line 1689, in _send_handling_auth response = await self._send_handling_redirects( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpx/_client.py", line 1726, in _send_handling_redirects response = await self._send_single_request(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpx/_client.py", line 1763, in _send_single_request response = await transport.handle_async_request(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpx/_transports/default.py", line 372, in handle_async_request with map_httpcore_exceptions(): File "/opt/homebrew/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/contextlib.py", line 155, in exit self.gen.throw(typ, value, traceback) File "/Users/mmihok/Documents/h2o-ai/projects/wave/repo/wave/py/venv/lib/python3.11/site-packages/httpx/_transports/default.py", line 86, in map_httpcore_exceptions raise mapped_exc(message) from exc httpx.ConnectError: All connection attempts failed



I was not able to reproduce the exact cause, it always happens in different scenario:

https://github.com/h2oai/wave/assets/23740173/6c58adef-fe7f-4896-b9b8-136e21e1549b

### Toolbar

Image is not downloaded directly if `download=True` if path is external link - https://wave.h2o.ai/img/logo.svg
I advise updating `toolbar.py` with the same path as in `link.py` example - http://localhost:10101/assets/brand/h2o-wave-b&w.png

https://github.com/h2oai/wave/assets/23740173/6768a909-af8f-470a-9a0b-0d59e61e3587

mturoci commented 4 months ago

Thanks for putting this together @marek-mihok!

1

No big deal. The important thing is that the most common formats work. If you don't manage to find a fix in <= 15min, leave it be until somebody reports it.

2

What was the fix for ui.plot? Can it be applied here as well?

3

This needs fixing. Please spend some more time to figure out a reliable repro.

4

Fixing is a better idea than changing the example.

marek-mihok commented 4 months ago

2- What was the fix for ui.plot? Can it be applied here as well?

It was achieved by setting a fixed default width for chosen components: https://github.com/h2oai/wave/pull/1988/files I don’t think this is what we want here. I tried also delayed init(), but it didn’t help - canvasRef.current.getBoundingClientRect() still returns default 300:150 canvas dimensions when the tab is not visible to the user. I’m thinking about visibilitychange event - https://developer.mozilla.org/en-US/docs/Web/API/Document/visibilitychange_event

3- This needs fixing. Please spend some more time to figure out a reliable repro.

I was able to figure out, it happens only on Firefox and it was not present in v1.0.0 The server crash happens randomly, mostly while playing around with component and after previous interaction. However it does not always crash directly on interaction, but sometimes few seconds later. The point is I was not able to reproduce it only when waiting and not interacting. Maybe this can help? https://github.com/gorilla/websocket/issues/119

4 - Fixing is a better idea than changing the example.

This won’t be possible since modern browsers support only the same origin downloads due to CORS policy.

Also I found one more issue with audio annotator:

https://github.com/h2oai/wave/assets/23740173/31d41096-a161-4953-9e57-c8baec217894

mouse down -> place cursor into the audio clip portion selection rectangle while annotating -> go outside of the rectangle and then go back inside -> mouse up

No tag is selected and therefore annotating is done with an empty tag and default red color. I'm working on the fix. Is there a reason for creating the empty tag annotation first or is it a bug?

https://github.com/h2oai/wave/assets/23740173/99507dc7-c9c1-4f99-ae01-c6105c8daf3a

mturoci commented 4 months ago

2 It was achieved by setting a fixed default width for chosen components

So we don't really understand the underlying cause (why are the dimensions different), correct?

3 it happens only on Firefox and it was not present in v1.0.0

Try reverting my reconnect feature and see if that was the culprit. Glancing over the code, websocket writes seem to be synchronized properly though.

4 modern browsers support only the same origin downloads due to CORS policy

TIL. You can proceed with your original suggestion + update the docs for download attribute. Something in the lines of "works only for files served by Wave".

marek-mihok commented 3 months ago

2 So we don't really understand the underlying cause (why are the dimensions different), correct?

✅ Correct. I've fixed it using visibilitychange event.

3 Try reverting my reconnect feature and see if that was the culprit.

It is, I was not able to reproduce the issue after reverting this commit.

4 You can proceed with your original suggestion + update the docs for download attribute.

Done ✅ We already had the correct info in docs: The download attribute will only start the download if the path points to a location on the same origin

mturoci commented 3 months ago

The download attribute will only start the download if the path points to a location on the same origin

Too technical IMO. Please change to: "The download attribute will only start the download if the path points to a location on the same origin (wave server)".

h2oai / wave

Pre-release QA before v1.1.0 #2276

Wave 1.1.0 pre-release QA

Test results

Manual testing

Audio annotator

Also I found one more issue with audio annotator: