search

dmwm / DMWMMON

1 stars 8 forks source link

Adding SpaceMon scripts used at Wisconsin #39

Closed cvuosalo closed 7 years ago

cvuosalo commented 7 years ago

At Wisconsin we tried the existing SpaceMon scripts for Hadoop and found them to be very slow. We developed new scripts that reduce the time to two hours for dumping the Hadoop /store file list and uploading it to the database.

nataliaratnikova commented 7 years ago

Great, thanks. I guess the cmsprod environment takes care of all details needed for the authenticated upload.

cvuosalo commented 7 years ago

@nataliaratnikova: Yes, we maintain a x509 proxy for the cmsprod user on the machine where we run the storage dump script.