Lustre extractor

[blastmaster]

As already said we need a specialized extractor for the lustre parallel file system. Please start with the points from our last meeting. Don't mention the lctl command at the moment, just focus on the lustre API. The lctl stuff must be executed on the lustre server nodes, so that this might be another special case for the future but not in this task.

[blastmaster]

Hint to use the lustre API, include #include <lustre/lustreapi.h> and link against liblustreapi -llustreapi afterward. Lustre is a system installation on taurus, under /usr/lib64..., therefore loading additional modules is not necessary.

[k4lipso]

Overview of lfs commands. If they have a tick they are working and will be implemented.

file required

• [x] lfs getstripe
• [x] lfs data_version
• [x] lfs hsm_state

  no file required

• [ ] lfs quota - works partially "Some errors happened when getting quota info."
• [ ] lfs check <osts|mds|server> - help i dont get it to work, what can i provide as "server"?
• [x] lfs getdirstripe - looks like it is allways returning the same
• [x] lfs df
• [x] lfs getname
• [x] lfs path2fid
• [x] lfs pool_list - no errors, but also no pools listed
• [x] lfs osts
• [x] lfs mdts

[k4lipso]

@blastmaster For gathering striping information we have to create a dummy file on a lustre mountpoint. Functions for dummy file creation and getting file striping information allready exists, but we still need to find a place to create that file.

Right now, to find a place, we gather every lustre mountpoint using the lustreapi and then pass the mountpoint path to the following function: https://github.com/blastmaster/sysmap/blob/3bf46acfdd5720fd9195337a511bca8be0ef89c2/lib/src/linux/lustre_extractor.cc#L298-L322

Basicly this function gets the username of the current user, appends it to the mountpoint path, checks if that path exists and if that is the case it creates the dummy file. The problem is that the dummy file creation only works if an folder named like the current user exists on that mountpoint. This approach could fail on other HPCs. Another approach would be to check every folder on the mountpoint for access rights which could be a very time intensive task. Do you have any recommendation on handling this problem?