Whenever we set up an AsterixDB cluster for Twittermap, we need to be careful about the number of concurrently opened files. For example, in CentOS, /etc/security/limits.conf handles this setting. Since the default value is only 1,024, it's not enough to handle concurrent datasets. Especially, for Twittermap, we create a view for each keyword and update the view frequently, it can easily hit the maximum. Adding the following settings will prevent this issue.
USERNAME soft nofile 500000
USERNAME hard nofile 500000
You can verify the setting by typing ulimit -a.
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 127529
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 500000
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 4096
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
Added by Qiushi (4/16/2021)
One thing needs to be noted:
However, to make sure the system user for AsterixDB's communication among machines will use the settings in the system config /etc/security/limits.conf, we have to make sure UsePAM yes is enabled in the SSH config /etc/ssh/ssh_config. Otherwise, the system user of AsterixDB will still use the default value 1024, and the symptom is when you start the AsterixDB, there will be Can Not Open File exceptions in cc.log with the reason of too many opened files;
baiqiushi
commented
3 years ago
Whenever we set up an AsterixDB cluster for Twittermap, we need to be careful about the number of concurrently opened files. For example, in CentOS,
/etc/security/limits.confhandles this setting. Since the default value is only 1,024, it's not enough to handle concurrent datasets. Especially, for Twittermap, we create a view for each keyword and update the view frequently, it can easily hit the maximum. Adding the following settings will prevent this issue.USERNAME soft nofile 500000 USERNAME hard nofile 500000
You can verify the setting by typing
ulimit -a.Added by Qiushi (4/16/2021)
One thing needs to be noted: However, to make sure the system user for AsterixDB's communication among machines will use the settings in the system config
/etc/security/limits.conf, we have to make sureUsePAM yesis enabled in the SSH config/etc/ssh/ssh_config. Otherwise, the system user of AsterixDB will still use the default value 1024, and the symptom is when you start the AsterixDB, there will be Can Not Open File exceptions incc.logwith the reason oftoo many opened files;