TProtocolexception: Invalid Data after chrome update

andrejohansson commented 2 months ago

Steps To Reproduce

After a recent update to chrome we get the following error on a page with dropdown filters. The same page worked in chrome before the update and it works in Firefox still. Seems to be something with duckdb?

Uncaught (in promise) Error: Invalid Error: TProtocolException: Invalid data
    at Go.runQuery (/_app/immutable/workers/duckdb-browser-eh.worker-efa74917.js:11:16927)
    at lc.onMessage (/_app/immutable/workers/duckdb-browser-eh.worker-efa74917.js:10:55591)
    at Qu.globalThis.onmessage (/_app/immutable/workers/duckdb-browser-eh.worker-efa74917.js:24:10943)
onMessage @ index.e5eb6c77.js:4

Steps we can take to reproduce the issue you're seeing. Sample data is helpful.

Environment

OS: Windows
Browser: Chrome Version 122.0.6261.112 (Official Build) (64-bit)

Expected Behavior

Able to select different options in our dropdowns and get updated data views.

Actual Behaviour

On first page load, an error message is shown in console. When selecting options in our dropdowns our graphs and tables get blank values.

Workarounds

Use an older browser version or another browser.

csjh commented 2 months ago

There's a good chance upgrading to @evidence-dev/evidence@29.0.2 will fix this

hughess commented 2 months ago

A few people in slack have mentioned this and it appears to be related to the browser cache in some way. If you clear your cache and refresh the page it should work - or open it in an incognito/private browser

andrejohansson commented 1 month ago

It seems the upgrade to 29.0.2 helped for me!

ksstolwijk commented 1 month ago

I am still running into the issue when using version 30.0.1 for @evidence-dev/evidence, for chrome I am using version 123.0.6312.86

mcrascal commented 1 month ago

@ksstolwijk have you tried clearing the cache in your browser?

ksstolwijk commented 1 month ago

@mcrascal we indeed tried to clear the cache, which initially seemed to help. However, the error appeared again after a while.

It might also be noteworthy to say that some of my colleagues who are working on linux machine are not experiencing this issue so far. Whereas I and most of my other colleagues work on Windows machines.

hughess commented 1 month ago

Have seen this issue pop up a few times in user projects in the last week

chandrasena commented 4 weeks ago

Still getting this with v33.0.1. FYI, I have 3 dropdrown boxes in the report.

fboerman commented 2 weeks ago

aah so its not only me. I have the same issue! and @hughess is right if you clear cache it resolves. still not good UX though...

hughess commented 2 weeks ago

This seems to be only on windows, or at least haven’t been able to reproduce on Mac

timothyhoward commented 1 week ago

I've been running into the same issue here and have been in contact with @hughess regarding the issue. I can concur with the observations of others that it seems to be Windows-specific and that when the cache is disabled (either through dev tools or through Incognito/InPrivate mode) the application functions as expected. It makes me think that either the backend cached parquet files are corrupting or that there is a path issue (i.e. backslashes / forwardslashes) that affects Windows but doesn't affect Unix/Linux-based systems.

I've ran a quick debug on my affected instance and the error is raised in the @evidence-dev/universal-sql package at /src/client-duckdb/browser.js during the setParquetUrls function. An error is caught (Error: Invalid Error: TProtocolException: Invalid data at O.onMessage (http://localhost:3000/@fs/C:/Users/<username>/dev/<application-name>/node_modules/.vite/deps/@duckdb_duckdb-wasm.js?v=0f823d57:351:15)) by the rejectTables function.

After the page has errored out, when reload is attempted I get a text value from the runQuery function: CREATE OR REPLACE VIEW \"datathing\".\"fct_education_by_region_sector\" AS (SELECT * FROM read_parquet('datathing_fct_education_by_region_sector.parquet'));

datathing is the source name
fct_education_by_region_sector is the table name

With the cache off, the page that errored out will load correctly (no TProtocolException raised), but there is another exception that is raised during the debug:

Error: Timeout while initializing database from the withTimeout function in @evidence-dev/universal-sql at /src/client-duckdb/both.js. This isn't shown in the error console but is raised as an exception breakpoint.

timothyhoward commented 1 week ago

This may be related to this: https://github.com/duckdb/duckdb-wasm/issues/1658

As per the issue above, results in a TProtocolException when calling a Parquet file by URL, and is Windows specific, only affecting Chrome and Edge (Firefox is unaffected).

timothyhoward commented 1 week ago

~~The issue seems to be fixed with the latest development version of DuckDB.~~ I've placed the following override into the package.json file:

"overrides": {
  "@duckdb/duckdb-wasm": "1.28.1-dev190.0"
}

Early days, but in initial testing no exceptions are raised with the cache enabled and the page now loads correctly in Edge and Chrome.

fboerman commented 1 week ago

Oh wow nice investigation thx! Will do this override as well let's see if it fixes it for me as well

-------- Original Message -------- On 6 May 2024, 06:50, Tim Howard wrote:

The issue seems to be fixed with the latest development version of DuckDB. I've placed the following override into the package.json file:

"overrides": { @.***/duckdb-wasm": "1.28.1-dev190.0" }

Early days, but in initial testing no exceptions are raised with the cache enabled and the page now loads correctly in Edge and Chrome.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

timothyhoward commented 1 week ago

After some further testing, the issue has unfortunately reappeared. Does seem to be an ongoing issue with @duckdb/duckdb-wasm.

timothyhoward commented 1 week ago

I had a working version of Evidence aside from the one affected, that had an override for axios (due to a vulnerability). Upgraded this on the affected one, and the errors don't seem to be as present, but I am getting a new error in the console together with the TProtocolException:

Error in Query! Invalid Error: ZSTD Decompression failure

@hughess What settings does Evidence use to save it's Parquet files in static/data? Asking because I found a similar issue here that was closed a while ago: https://github.com/duckdb/duckdb/issues/2882

Edit: Turning off certain compression schemes in the dev tools, now confronted with this error Extension Autoloading Error: An error occurred while trying to automatically install the required extension 'parquet': Extension "https://extensions.duckdb.org/v0.10.2/wasm_eh/parquet.duckdb_extension.wasm" could not be loaded because its signature is either missing or invalid and unsigned extensions are disabled by configuration (allow_unsigned_extensions)

timothyhoward commented 1 week ago

Hi all,

I've found a mitigation to stop this from occurring until @duckdb/duckdb-wasm fix the issue. If you turn caching off for certain file types (parquet, arrow, js, json), you won't get the TProtocolException in Chrome/Edge. It will be a little slower as there will be more requests made to the server for the data files, but it will stop the error being raised.

Here's a configuration for IIS, which can be adapted to other web servers:

<rewrite xdt:Transform="InsertIfMissing">
  <outboundRules rewriteBeforeCache="true" xdt:Transform="InsertIfMissing">
    <rule name="Temporary fix for DuckDB parquet reload issue" preCondition="IsCacheControlled" xdt:Locator="Match(name)" xdt:Transform="InsertIfMissing">
      <match serverVariable="RESPONSE_Cache-Control" pattern=".*" />
      <action type="Rewrite" value="no-store, max-age=0" />
    </rule>
    <preConditions xdt:Transform="InsertIfMissing">
      <preCondition name="IsCacheControlled" xdt:Locator="Match(name)" xdt:Transform="InsertIfMissing">
        <add input="{REQUEST_FILENAME}" pattern="\.(parquet|arrow|js|json)$" />
      </preCondition>
    </preConditions>
  </outboundRules>
</rewrite>

I strongly assume the issue is associated with this: https://github.com/duckdb/duckdb-wasm/issues/1658

aborruso commented 5 days ago

Hi, I have the same problem for the demo default project. After I change the title of index page and save it, the browser reload and I have

hughess commented 5 days ago

Thanks for sharing - we're looking into ways to mitigate this issue in advance of the next release of duckdb-wasm (which we hope will contain a fix). Here are the notes we recently shared in our slack community:

Known Issue for self-hosted Windows users

Situation:

If you are on Windows and using Chrome or Edge, you may have seen a TProtocolException: invalid data error (https://github.com/evidence-dev/evidence/issues/1693)
It may also appear as an empty data error in components depending on the situation

Cause:

This is caused by browser caching behaviour and likely comes from this issue in DuckDB WASM: https://github.com/duckdb/duckdb-wasm/issues/1658

Workarounds:

Disable caching for parquet files in your host/deployment settings (see github issue for more info: https://github.com/evidence-dev/evidence/issues/1693)
Use a private/incognito browser window
Use Firefox instead of Chrome or Edge

Upcoming:

We are testing a patch to eliminate these errors during local development
We plan to include information in the docs to assist with managing these settings in your deployment

evidence-dev / evidence