search

Intel-bigdata / SSM

Smart Storage Management for Big Data, a comprehensive hot/cold data optimized solution
Apache License 2.0
134 stars 67 forks source link

EC/Compact/compress actions lead to lose files temperature. #2181

Closed lipppppp closed 4 years ago

lipppppp commented 4 years ago

For example, we read the file "/test/3.txt" three times. The fid of thi file is 17013.After I read this file three times, the count number and fid are inserted in the table. When ec/compact/compress actions are used, the file fid is changed. But the accessCount table still have the old fid about this file. So the temperature information of this file is lost in some way. image image image

lipppppp commented 4 years ago

Same in the rule, the file "/test/2.txt" fid is 17010. The rule is "file:every 5s | path matches "/test/*" and accessCount(1hour) > 5 | ec -policy XOR-2-1-1024k". After I read this file six times, the count number and fid are inserted in the table. But the file fid is changed to 17015. The hot file page can‘t show the count and the temperature information of this file is lost. image image image

lipppppp commented 4 years ago

EC/compact/compress related actions will create new files, so I suggest that the fid in these table should be updated after these actions.

PHILO-HE commented 4 years ago

Let me try to fix this issue.

PHILO-HE commented 4 years ago

Fixed by #2182.

PHILO-HE commented 4 years ago

Closing this issue, thanks @lipppppp for reporting this critical issue!