search

duckdb / duckdb-web

DuckDB website and documentation
https://duckdb.org
MIT License
144 stars 278 forks source link

Clarify disk spilling for memory connections #2859

Open soerenwolfers opened 2 months ago

soerenwolfers commented 2 months ago

https://duckdb.org/docs/guides/performance/how_to_tune_workloads

says 

If DuckDB is running in in-memory mode, it cannot use disk to offload data if it does not fit into main memory. To enable offloading in the absence of a persistent database file, use the [SET temp_directory statement](https://duckdb.org/docs/configuration/pragmas#temp-directory-for-spilling-data-to-disk):

But according to https://duckdb.org/docs/configuration/overview.html

the temp_directory setting is set to a non-null ".tmp" value by default.

Could you clarify whether that means disk spilling is activated by default after all, and if not whether relative paths like ".tmp" are ignored in general?

soerenwolfers commented 1 month ago

Related: https://github.com/duckdb/duckdb-web/issues/3058

soerenwolfers commented 1 month ago

@szarnyasg What does the enhancement label mean here?

szarnyasg commented 1 month ago

I was looking for a "clarification" label but didn't find it, so I settled on enhancement. This is more of a bug in the documentation though, so I can bump its priority and take a look next week.

soerenwolfers commented 1 month ago

Related issue that could be addressed in the documentation at the same time, but with opposite problem: https://github.com/duckdb/duckdb-web/issues/3058 -- docs say it defaults to 0 but probably zero is interpreted taken as "unlimited".