watermark.high.ratio, watermark.low.ratio not working

[takersk]

Alluxio Version: v2.8.0

Describe the bug eviction does not proceed

To Reproduce None

Expected behavior Run eviction when storage usage reaches 90%

Urgency None

Are you planning to fix it None

Additional context

1. configmap

   ...
   ALLUXIO_JAVA_OPTS: >-
   -Dalluxio.master.journal.type=EMBEDDED
   -Dalluxio.master.journal.folder=/journal
   -Dalluxio.master.embedded.journal.addresses=,alluxio-master-0:19200,alluxio-master-1:19200,alluxio-master-2:19200
   -Dalluxio.hub.manager.rpc.hostname=alluxio-hub
   -Dalluxio.master.keytab.file=/tmp/hadoop-cdp-write.keytab
   -Dalluxio.master.mount.table.root.option.alluxio.underfs.hdfs.configuration=/secrets/hdfsConfig/core-site.xml:/secrets/hdfsConfig/hdfs-site.xml
   -Dalluxio.master.mount.table.root.option.alluxio.underfs.version=2.6.0-cdh5.14
   -Dalluxio.master.mount.table.root.ufs=hdfs://cdp/cdpdev/prod/
   -Dalluxio.master.nameservices.alluxio-cluster=master1,master2,master3
   -Dalluxio.master.principal=hadoop-cdp-write@KAKAO.HADOOP
   -Dalluxio.master.rpc.address.alluxio-cluster.master1=alluxio-master-0:19998
   -Dalluxio.master.rpc.address.alluxio-cluster.master2=alluxio-master-1:19998
   -Dalluxio.master.rpc.address.alluxio-cluster.master3=alluxio-master-2:19998
   -Dalluxio.master.rpc.addresses=alluxio-master-0:19998,alluxio-master-1:19998,alluxio-master-2:19998
   -Dalluxio.master.security.impersonation.hadoop-cdp-write.groups=shophow
   -Dalluxio.master.security.impersonation.hadoop-cdp-write.users=hadoop-cdp-write
   -Dalluxio.master.security.impersonation.root.groups=shophow
   -Dalluxio.master.security.impersonation.root.users=hadoop-cdp-write
   -Dalluxio.security.stale.channel.purge.interval=365d
   -Dalluxio.underfs.version=2.6.0-cdh5.14
   -Dalluxio.user.file.metadata.sync.interval=0
   -Dalluxio.user.file.replication.durable=2
   -Dalluxio.user.short.circuit.enabled=false
   -Dalluxio.worker.keytab.file=/tmp/hadoop-cdp-write.keytab
   -Dalluxio.worker.principal=hadoop-cdp-write@KAKAO.HADOOP 
   ALLUXIO_JOB_MASTER_JAVA_OPTS: >-
   -Dalluxio.master.hostname=${ALLUXIO_MASTER_HOSTNAME}
   -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=jvm-jobmaster.hprof
   -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps
   -Xloggc:gc-jobmaster.log -XX:+UseGCLogFileRotation -XX:GCLogFileSize=5m
   -XX:NumberOfGCLogFiles=100 
   ALLUXIO_JOB_WORKER_JAVA_OPTS: >-
   -Dalluxio.worker.hostname=${ALLUXIO_WORKER_HOSTNAME}
   -Dalluxio.job.worker.rpc.port=30001 -Dalluxio.job.worker.data.port=30002
   -Dalluxio.job.worker.web.port=30003 -XX:+HeapDumpOnOutOfMemoryError
   -XX:HeapDumpPath=jvm-jobworker.hprof -verbose:gc -XX:+PrintGCDetails
   -XX:+PrintGCDateStamps -Xloggc:gc-jobworker.log -XX:+UseGCLogFileRotation
   -XX:GCLogFileSize=5m -XX:NumberOfGCLogFiles=100 
   ALLUXIO_MASTER_JAVA_OPTS: >-
   -Dalluxio.master.hostname=${ALLUXIO_MASTER_HOSTNAME}
   -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=jvm-master.hprof
   -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:gc-master.log
   -XX:+UseGCLogFileRotation -XX:GCLogFileSize=5m -XX:NumberOfGCLogFiles=100 
   ALLUXIO_WORKER_JAVA_OPTS: >-
   -Dalluxio.worker.hostname=${ALLUXIO_WORKER_HOSTNAME}
   -Dalluxio.worker.rpc.port=29999 -Dalluxio.worker.web.port=30000
   -Dalluxio.user.short.circuit.enabled=false -Dalluxio.worker.ramdisk.size=40G
   -Dalluxio.worker.tieredstore.levels=2
   -Dalluxio.worker.tieredstore.level0.alias=MEM
   -Dalluxio.worker.tieredstore.level0.dirs.mediumtype=MEM
   -Dalluxio.worker.tieredstore.level0.dirs.path=/dev/shm
   -Dalluxio.worker.tieredstore.level0.dirs.quota=30G
   -Dalluxio.worker.tieredstore.level0.watermark.high.ratio=0.9
   -Dalluxio.worker.tieredstore.level0.watermark.low.ratio=0.7
   -Dalluxio.worker.tieredstore.level1.alias=SSD
   -Dalluxio.worker.tieredstore.level1.dirs.mediumtype=SSD
   -Dalluxio.worker.tieredstore.level1.dirs.path=/tmp/ssd
   -Dalluxio.worker.tieredstore.level1.dirs.quota=200G
   -Dalluxio.worker.tieredstore.level1.watermark.high.ratio=0.9
   -Dalluxio.worker.tieredstore.level1.watermark.low.ratio=0.7
   -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=jvm-worker.hprof
   -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:gc-worker.log
   -XX:+UseGCLogFileRotation -XX:GCLogFileSize=5m -XX:NumberOfGCLogFiles=100 
   ALLUXIO_WORKER_TIEREDSTORE_LEVEL0_DIRS_PATH: /dev/shm
   ...

2. master UI

[HelloHorizon]

@yuzhu can you take a look?

[takersk]

@HelloHorizon @yuzhu are you in progress???

[takersk]

@HelloHorizon @yuzhu have you miss this issue? please check again

[takersk]

@HelloHorizon @yuzhu have you miss this issue? please check again

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in two weeks if no further activity occurs. Thank you for your contributions.