duckdb / duckdb-wasm

WebAssembly version of DuckDB
https://shell.duckdb.org
MIT License
1.32k stars 132 forks source link

duckdb web version is single-threaded #1206

Open l1t1 opened 2 years ago

l1t1 commented 2 years ago

What happens?

a sql run on duckdb web version took 9 sec. the same sql run on duckdb CLI took 2 sec.

To Reproduce

open the edge and visit https://shell.duckdb.org/, after it shows

DuckDB Web Shell
Database: v0.5.1
Package:  @duckdb/duckdb-wasm@1.17.1-dev18.0

Connected to a local transient in-memory database.
Enter .help for usage hints.

run

.timer on

with t as (select x from generate_series(3,1e4::int,2) t(x)),
p as(select x from t
except
select t1.x*t2.x from t t1,t t2
where t1.x*t2.x<1e4
),
s as(
select x,x*x x2 from p)
select s.*,s2.* from s,s s2
where
s.x<s2.x and
reverse(s.x2::varchar)::int=s2.x2
;

and run same sql on duckdb CLI, compare the time costs.

OS:

windows 7

DuckDB Version:

v0.5.1

DuckDB Client:

Web version

Full Name:

lutao

Affiliation:

study

Have you tried this on the latest master branch?

Have you tried the steps to reproduce? Do they include all relevant data and configuration? Does the issue you report still appear there?

l1t1 commented 2 years ago

web version only use one core? 2022-10-30_155605 2022-10-30_173128

Mytherin commented 2 years ago

Not all browsers support multi-threading in the browser. It is expected that certain browsers are limited to a single thread. Have you tried this on chrome as well?

l1t1 commented 2 years ago

my browser says it use Chromium engine, as chrome does

Microsoft Edge 版本 107.0.1418.24 (正式版本) (32 位) 此浏览器基于 Chromium 开源项目及其他 开源软件

prusswan commented 3 months ago

@carlopi just to check if this is still a limitation on duck-wasm?

The docs mentioned:

By default, the WebAssembly client only uses a single thread.

Does this mean it is possible to allow multiple threads by rebuilding from source? Would this allow for queries to be performed faster on an interactive dashboard? I'm thinking about remote files (e.g. those that are served over httpfs/s3) as well as local files.

domoritz commented 3 months ago

IIRC the COI version supports multithreading but it needs special server header flags to run https://shell.duckdb.org/docs/interfaces/index.DuckDBBundles.html.