As the number of jobs run increases, the directory structure under jobs directory grows broader. As a result accessing the directory becomes slower to the point that ls no longer works.
A possible solution is to split the id into chunks e.g. 8-bytes each end store it in a nested directory structure. e.g.
As the number of jobs run increases, the directory structure under jobs directory grows broader. As a result accessing the directory becomes slower to the point that ls no longer works. A possible solution is to split the id into chunks e.g. 8-bytes each end store it in a nested directory structure. e.g.