quota handling has changed in lustre 2.4

[degremont]

This ticket is opened to track changes required in shine regarding new quota handling in lustre 2.4:

• --param on mkfs.lustre and tunefs.lustre are now deprecated.
• /proc/fs/lustre/lquota is not used anymore to set quota parameters. So the bunit, iunit, btune and bunit are now obsolete in the LMF file.
• quotas are now turned on or off by executing a 'lctl conf_param' on the MGS.

For fixing, we need to know if shine is deemed to handle several versions of lustre, or if version 1.3 only supports lustre version 2.4.

Could you please comment on this ? Thanks.

Reported by: theryf

[degremont]

So far, Shine always succeed to be compatible with all Lustre, from 1.8. We would like to keep this compatibility this as long as possible. Even if one day, some Lustre version should be dropped, we will never reduce the compatibility range to something like "only 2.4".

We need to know the whole process to setup Quota on Lustre 2.4. Do you have a doc pointer? There is already a couple of quota commands to run, even with Lustre 2.1, to fully setup this. We need to see if some format or tuning command are still required, starting with 2.4

Original comment by: degremont

[degremont]

New behaviour is described in http://build.whamcloud.com/job/lustre-manual/lastSuccessfulBuild/artifact/lustre_manual.pdf section 21.

Unless I missed something, the description above is an exhaustive list of the changes regarding shine. With shine's current behaviour, it consists in:

• add lustre version conditionals on code dealing with format and tuning options for quotas
• decide if shine should automatically (or not) run the lctl conf_param commands on MGS. If yes, where would be the best place in terms or execution stream, to put this new action.

I've set up a try-fix (try, because functionally it works well, but in terms of genericity and correctness regarding shine design, it may not be accurate), that deals with the conditionals on one part, and the automatic lctl command execution on the other part. Here, I chose to handle lctl execution in the Start command, after the tuning has been successfully applied, because this command must be executed on a started MGS (if the filesystem starts successfully, we can assume MGS is started), and the filesystem must be registered to the MGS (in the same way, if the filesystem has started successfully, it must be registered to the MGS). This approach has some drawbacks, though:

• Each time a target is started, lctl conf_param is executed on the MGS (which , from the outside doesn't seem to be a problem, but can it be an issue regarding lustre internals ?)
• Do we really want to handle this lctl automatically: it could for example be handled by a new shine command (shine quota ?), or left unhandled by shine, in which case, all quota parameters in the LMF file would become useless in lustre 2.4.

You can have a look at this try-fix at the following location: https://github.com/bullxpfs/lustre-shine/commit/47fc87a2e2eae1977467e3dcef6224e1f0e14c5a

Regards

Original comment by: theryf

[degremont]

Thanks for the proposal.

I think we should make land a patch that just disable the useless part regarding quota, starting from 2.4.

A more generic solution should be think about to manage lctl conf_param aspects, especially about command that should be run very few times, independently of format, start or mount by example...

Original comment by: degremont

[degremont]

OK, so let's abandon the 'lctl conf_param' part for now. Please find a new patch proposal deactivating quota handling for lustre 2.4: https://github.com/bullxpfs/lustre-shine/commit/fb8f23675d1b4a58b8b8d949e156f1e5f4847525

It would be a good thing, however, if we found a shine based solution for handling new lustre 2.4 activation process (with lctl conf_param) in a proper way. Perhaps we could open a new ticket to specifically handle this case ? Regards.

Original comment by: theryf

[degremont]

OK. I've made few comment on your patch.

Let's create a new ticket to track what could be done for conf_param needs.

Original comment by: degremont

[degremont]

Thanks for your comments. I've updated the patch based on these comments at: https://github.com/bullxpfs/lustre-shine/commit/f2c2ee875a2b3fd922c64c431fb844f9cfdd52e3 Regards

Original comment by: theryf

[degremont]

I've realized your patch is modifying Format action but you are not changing the tests. There was a bug in CmdTests which was preventing to detect this. I've pushed a commit [7cc94a] to fix this.

Please rebase your patch on top of this commit and fix CmdTests quota tests. Look at MDT and OST quota test to have an example.

Original comment by: degremont

[degremont]

I've pushed a new patch containing both the lustre 2.4 specific code and unit tests checking this. You can find it at the following location: https://github.com/bullxpfs/lustre-shine/commit/d6a34b448cc24db37420dca8174e90df4080d746

Regards.

Original comment by: theryf

[degremont]

You used Lustre 2.4.2 in your tests. Is this the first Lustre version supporting this behaviour? Lustre 2.4.0 does not support this?

Original comment by: degremont

[degremont]

This behaviour begins with Lustre 2.4. Actually, I've put an arbitrary version that is greater than 2.4 What version do you want me to set up in the tests ? 2.4.1 ? 2.4.0.1 ? 2.4 ? Please tell me and I'll push another patch. Thanks.

Original comment by: theryf

[degremont]

In these cases, please used the first Lustre version which is concerned by this behaviour. I think that '2.4' should do the trick.

Original comment by: degremont

[degremont]

OK thanks. I've pushed a new patch to address this: https://github.com/bullxpfs/lustre-shine/commit/30c907d41d5454dbd8c32a8a77d10efbc7d0e0d5

Regards.

Original comment by: theryf

[degremont]

• status: new --> closed
• Resolution: --> fixed

Original comment by: degremont

[degremont]

Fixed in commit [fcb683]

Thanks for this patch

Original comment by: degremont