The recent reindexing of PDP files has been running for over a week on leopard, much longer than I expected. Didn't it only take about three days last time?
The correct thing to do in the future, when indexing files this large, would be to queue them up for the compute nodes. A script to automate this process - take a list of files as input, break them into groups of four or five, and output a PBS job for each group, would possibly be useful.
(A much nicer solution: add capability for indexing files to the existing jobqueuing system)
corviday
commented
5 years ago
The recent reindexing of PDP files has been running for over a week on leopard, much longer than I expected. Didn't it only take about three days last time?
The correct thing to do in the future, when indexing files this large, would be to queue them up for the compute nodes. A script to automate this process - take a list of files as input, break them into groups of four or five, and output a PBS job for each group, would possibly be useful.
(A much nicer solution: add capability for indexing files to the existing jobqueuing system)