Way to simply run all vfs/fs related tests?

[brauner]

Hey,

Thanks for ltp! It's an excellent project and I use it regularly for testing. Is there any way to run all vaguely vfs/fs related tests? Currently, I just always run the whole testsuite but that often takes too long and I'm mostly interested in the vfs/fs stuff to make sure that there are no regressions for patches we accept. If there isn't a way right now would it be possible for you to add a switch for that?

Thanks!

[Avinesh]

Hi @brauner , I think you are looking for ./runltp -f fs to execute only filesystem test suite as described in https://github.com/linux-test-project/ltp#running-tests

[brauner]

Oh neat, thanks @Avinesh. And the list of available suites is in scenario_groups?

[pevik]

@brauner Kernel maintainer interested in running tests, great :).

I think you are looking for ./runltp -f fs ...

You don't want to bother with legacy runltp, you want to use https://github.com/linux-test-project/runltp-ng (LTP specific, it might be later replaced with https://github.com/acerv/kirk/, which should support more kernel testing frameworks).

Also any test listed in syscalls which has .all_filesystems = 1 is likely VFS related. Also, there are fanotify tests in syscalls, if you're interested in. Others might point other runtest files.

[brauner]

@brauner Kernel maintainer interested in running tests, great :).

Well, I always run LTP and xfstsests on my test machines. :D Just that often I only want to run a subset like, fs_perms, fs, containers, fs_bind and so on but none of the rest.

I think you are looking for ./runltp -f fs ...

You don't want to bother with legacy runltp, you want to use https://github.com/linux-test-project/runltp-ng (LTP specific, it

Aha, I'll take a look.

might be later replaced with https://github.com/acerv/kirk/, which should support more kernel testing frameworks).

Also any test listed in syscalls which has .all_filesystems = 1 is likely VFS related. Also, there are fanotify tests in syscalls, if you're interested in. Others might point other runtest files.

Hm, yeah, all of that. Is there a neat switch for that?

[metan-ucw]

@brauner not yet unfortunately, what we are working on though, is passing more metadata to the test runner so that you can filter out tests parametrically, exactly for the purpose of running subsets of test that are related to fs/timers/etc.

[metan-ucw]

@acerv I suppose that you haven't had time to work on more advanced test filtering yet, or did you?

[pevik]

It'd be interesting to have in metadata subject of the testing, which could be later used for filtering and nowadays we could have script which would create special runtest by parsing this info in ltp.json.

[acerv]

@metan-ucw in kirk --skip-file and --skip-tests options permits to skip tests when needed. They both support regexp. But there's no more than that for now

[metan-ucw]

@acerv that's not enough, what we need is to be able to do queries on the test metadata and a way how to save pre-derfined queries so that we have simple way how to enable users to do things as 'runltp-ng --test=timers' or 'runltp-ng --test=fs'

[acerv]

@metan-ucw is metadata providing tests grouping already?

[brauner]

Hm, fwiw, I'm getting a success report for gf01 and gf06 from runltp but a failure report from runltp-ng.

[brauner]

Not it succeeds, weird. Ignore it. This is the wrong place for this anyway.

[metan-ucw]

@acerv once you have metadata, you have the mapping from test binary name to a list of attributes that can be used for the filtering. I suppose that we should sit down and discuss how to implement this.