Alluxio / alluxio

Alluxio, data orchestration for analytics and machine learning in the cloud
https://www.alluxio.io
Apache License 2.0
6.87k stars 2.94k forks source link

When start alluxio by using misconf, the function of the logging system is abnormal. #17405

Open MEILIDEKCL opened 1 year ago

MEILIDEKCL commented 1 year ago

Alluxio Version: 2.9.0

Describe the bug When set alluxio.master.ttl.checker.interval to zero or negative, the log system of master will quickly reach the top. The number of master.log quickly reach 100, and all the info is same. To Reproduce Firstly, make sure the conf(alluxio-site.properties) is like this.

alluxio.master.hostname=localhost
alluxio.master.mount.table.root.ufs=/tmp
alluxio.master.ttl.checker.interval=-1

Secondly, use shell below.

hadoop@hadoop-Standard-PC-i440FX-PIIX-1996:~/alluxio-2.9.0$ ./bin/alluxio-mount.sh SudoMount
Unmounting /mnt/ramdisk
Formatting RamFS: /mnt/ramdisk (2757178709)
hadoop@hadoop-Standard-PC-i440FX-PIIX-1996:~/alluxio-2.9.0$ ./bin/alluxio format
Executing the following command on all worker nodes and logging to /home/hadoop/alluxio-2.9.0/logs/task.log: /home/hadoop/alluxio-2.9.0/bin/alluxio formatWorker
Waiting for tasks to finish...
All tasks finished
Executing the following command on all master nodes and logging to /home/hadoop/alluxio-2.9.0/logs/task.log: /home/hadoop/alluxio-2.9.0/bin/alluxio formatJournal
Waiting for tasks to finish...
All tasks finished
hadoop@hadoop-Standard-PC-i440FX-PIIX-1996:~/alluxio-2.9.0$ du -sh logs/
12K     logs/
hadoop@hadoop-Standard-PC-i440FX-PIIX-1996:~/alluxio-2.9.0$ ./bin/alluxio-start.sh local
Assuming NoMount by default.
Starting master @ hadoop-Standard-PC-i440FX-PIIX-1996. Logging to /home/hadoop/alluxio-2.9.0/logs
Starting job master @ hadoop-Standard-PC-i440FX-PIIX-1996. Logging to /home/hadoop/alluxio-2.9.0/logs
Starting worker @ hadoop-Standard-PC-i440FX-PIIX-1996. Logging to /home/hadoop/alluxio-2.9.0/logs
Starting job worker @ hadoop-Standard-PC-i440FX-PIIX-1996. Logging to /home/hadoop/alluxio-2.9.0/logs
Starting proxy @ hadoop-Standard-PC-i440FX-PIIX-1996. Logging to /home/hadoop/alluxio-2.9.0/logs
-----------------------------------------
Starting to monitor all local services.
-----------------------------------------
--- [ OK ] The master service @ hadoop-Standard-PC-i440FX-PIIX-1996 is in a healthy state.
--- [ OK ] The job_master service @ hadoop-Standard-PC-i440FX-PIIX-1996 is in a healthy state.
--- [ OK ] The worker service @ hadoop-Standard-PC-i440FX-PIIX-1996 is in a healthy state.
--- [ OK ] The job_worker service @ hadoop-Standard-PC-i440FX-PIIX-1996 is in a healthy state.
--- [ OK ] The proxy service @ hadoop-Standard-PC-i440FX-PIIX-1996 is in a healthy state.
hadoop@hadoop-Standard-PC-i440FX-PIIX-1996:~/alluxio-2.9.0$ du -sh logs/
1004M   logs/
hadoop@hadoop-Standard-PC-i440FX-PIIX-1996:~/alluxio-2.9.0$ ls logs/ | wc -l
115

Expected behavior I find that the log system will continuously log the same info until the number reach 100. But it is still not good. In my view, I think the system should check this configuration before start-up for avoiding wasting resources.

Are you planning to fix it Yes, of course. I have fixed a version.

Additional context Add any other context about the problem here.

ssz1997 commented 1 year ago

Thanks for raising the issue

MEILIDEKCL commented 1 year ago

@jiacheliu3 hello! I have created a new pr, can you review it? Thank you so much.

github-actions[bot] commented 5 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in two weeks if no further activity occurs. Thank you for your contributions.