search

greenlion / warp

WarpSQL Server, an open source OLAP focused distribution of the world's most popular open source database bundled with OLAP performance related plugins such as the WARP storage engine..
http://warpsql.blog
Other
41 stars 2 forks source link

Server gets stuck in 100% CPU loop if warp_cache_size is too small #57

Closed greenlion closed 2 years ago

greenlion commented 3 years ago

For SSB SF30, I had to use warp_cache_size=12 (didn't try 11) to be able to get all the SSB queries to run. Either there is something holding open files because it isnt' being deleted or what, I am not sure. Investigate for beta 3.

greenlion commented 2 years ago

Fixed in 8f1c9888197b328f397bebae81f53be51db2387c

greenlion commented 2 years ago

Columns were not being evicted from the fastbit cache under cache pressure. There was a define that needed to be set in FastBit to make that work. However, now, with parallel query, the active working set could exceed the warp_cache_size. If this happens, the database gets stuck in a loop trying to free objects, but none can be free'd. If this happens, decrease warp_parallel_join_threads or decrease the number of columns in a WARP partition (or both). The dimension primary keys MUST fit into the warp cache or it will get stuck, so make sure the WARP cache is large enough to store the dimensions. This should pretty much always be the case though.

greenlion commented 2 years ago

Was caused by neglecting to delete an array of column values which needed to be deleted after they were accessed.