quota handling has changed in lustre 2.4

degremont commented 11 years ago

This ticket is opened to track changes required in shine regarding new quota handling in lustre 2.4:

--param on mkfs.lustre and tunefs.lustre are now deprecated.
/proc/fs/lustre/lquota is not used anymore to set quota parameters. So the bunit, iunit, btune and bunit are now obsolete in the LMF file.
quotas are now turned on or off by executing a 'lctl conf_param' on the MGS.

For fixing, we need to know if shine is deemed to handle several versions of lustre, or if version 1.3 only supports lustre version 2.4.

Could you please comment on this ? Thanks.

Reported by: theryf

degremont commented 11 years ago

So far, Shine always succeed to be compatible with all Lustre, from 1.8. We would like to keep this compatibility this as long as possible. Even if one day, some Lustre version should be dropped, we will never reduce the compatibility range to something like "only 2.4".

We need to know the whole process to setup Quota on Lustre 2.4. Do you have a doc pointer? There is already a couple of quota commands to run, even with Lustre 2.1, to fully setup this. We need to see if some format or tuning command are still required, starting with 2.4

Original comment by: degremont

degremont commented 11 years ago

New behaviour is described in http://build.whamcloud.com/job/lustre-manual/lastSuccessfulBuild/artifact/lustre_manual.pdf section 21.

Unless I missed something, the description above is an exhaustive list of the changes regarding shine. With shine's current behaviour, it consists in:

add lustre version conditionals on code dealing with format and tuning options for quotas
decide if shine should automatically (or not) run the lctl conf_param commands on MGS. If yes, where would be the best place in terms or execution stream, to put this new action.

I've set up a try-fix (try, because functionally it works well, but in terms of genericity and correctness regarding shine design, it may not be accurate), that deals with the conditionals on one part, and the automatic lctl command execution on the other part. Here, I chose to handle lctl execution in the Start command, after the tuning has been successfully applied, because this command must be executed on a started MGS (if the filesystem starts successfully, we can assume MGS is started), and the filesystem must be registered to the MGS (in the same way, if the filesystem has started successfully, it must be registered to the MGS). This approach has some drawbacks, though:

Each time a target is started, lctl conf_param is executed on the MGS (which , from the outside doesn't seem to be a problem, but can it be an issue regarding lustre internals ?)
Do we really want to handle this lctl automatically: it could for example be handled by a new shine command (shine quota ?), or left unhandled by shine, in which case, all quota parameters in the LMF file would become useless in lustre 2.4.

You can have a look at this try-fix at the following location: https://github.com/bullxpfs/lustre-shine/commit/47fc87a2e2eae1977467e3dcef6224e1f0e14c5a

Regards

Original comment by: theryf

degremont commented 11 years ago

Thanks for the proposal.

I think we should make land a patch that just disable the useless part regarding quota, starting from 2.4.

A more generic solution should be think about to manage lctl conf_param aspects, especially about command that should be run very few times, independently of format, start or mount by example...

Original comment by: degremont

degremont commented 11 years ago

OK, so let's abandon the 'lctl conf_param' part for now. Please find a new patch proposal deactivating quota handling for lustre 2.4: https://github.com/bullxpfs/lustre-shine/commit/fb8f23675d1b4a58b8b8d949e156f1e5f4847525

It would be a good thing, however, if we found a shine based solution for handling new lustre 2.4 activation process (with lctl conf_param) in a proper way. Perhaps we could open a new ticket to specifically handle this case ? Regards.

Original comment by: theryf