Description
The page https://vaex.readthedocs.io/en/latest/conf.html#thread-count specifies that by setting vaex.settings.main.thread_count, you set the thread count, while this in practice this happens only when setting the environment variable VAEX_NUM_THREADS.
This was observed during run on server that had large differences between actual number of processors (96) and number of threads allowed for the particular vaex program (via a batch system) which resulted in the process being killed, even though the program was setting vaex.settings.main.thread_count to a small number of threads.
Description The page https://vaex.readthedocs.io/en/latest/conf.html#thread-count specifies that by setting vaex.settings.main.thread_count, you set the thread count, while this in practice this happens only when setting the environment variable VAEX_NUM_THREADS.
This was observed during run on server that had large differences between actual number of processors (96) and number of threads allowed for the particular vaex program (via a batch system) which resulted in the process being killed, even though the program was setting vaex.settings.main.thread_count to a small number of threads.
This seems indeed to be the case by checking the code, thread_count_default is set at https://github.com/vaexio/vaex/blob/98b9f7924d57bf5cc75212182bf0d283d7ebc059/packages/vaex-core/vaex/multithreading.py#L21 but as far as I can tell that does not check at all the settings.
Software information
import vaex; vaex.__version__)
: {'vaex': '4.12.0', 'vaex-core': '4.12.0', 'vaex-viz': '0.5.3', 'vaex-hdf5': '0.12.3', 'vaex-server': '0.8.1', 'vaex-astro': '0.9.1', 'vaex-jupyter': '0.8.0', 'vaex-ml': '0.18.0'}Additional information Code to reproduce:
results in (incorrect)
While if you run
you get (the correct):