Loki 2.6.1 Crashes

[jocelynthode]

Describe the bug After a short period of time when using Loki 2.6.1 I start to get the errors. After a time the container won't even start

This issue does not appear when running Loki image 2.5.0.

Or more easily I only have this error with the Helm Chart 2.8.0+. When using the Helm Chart 2.6.5 I have no issues.

To Reproduce Steps to reproduce the behavior:

1. Started Loki image 2.6.1

Expected behavior Loki should work

Environment:

• Rancher Kubernetes
• Helm

Screenshots, Promtail config, or terminal output

level=error ts=2022-09-01T11:05:31.817081459Z caller=table.go:152 table-name=index_19236 msg="failed to remove working directory /data/loki/boltdb-shipper-compactor/index_19236" err="unlinkat /data/loki/boltdb-shipper-compactor/index_19236/.nfs0000000000014dce00000072: device or resource busy"
level=error ts=2022-09-01T11:05:31.81717923Z caller=compactor.go:492 msg="failed to compact files" table=index_19236 err="recovered from panic opening boltdb file: runtime error: invalid memory address or nil pointer dereference"
level=error ts=2022-09-01T11:05:31.817279032Z caller=compactor.go:432 msg="failed to run compaction" err="recovered from panic opening boltdb file: runtime error: invalid memory address or nil pointer dereference"

[liguozhong]

level=error ts=2022-09-01T11:05:31.817279032Z caller=compactor.go:432 msg="failed to run compaction" err="recovered from panic opening boltdb file: runtime error: invalid memory address or nil pointer dereference"

hi, I'm not very familiar with 'compactor' , but I suggest that you post a complete log, only this line of log is believed to be difficult for other maintainers to solve this issue.