utils: apply access rules to load/load2 in one shot

[jarkkojs]

Open load/load2 before calling nftw(). Inside the callback only collect rules using smack_accessed_add_from_file(). Finally, after returning from nftw(), call smack_accesses_save() or smack_accesses_clear().

[jarkkojs]

This is one first small iterative step towards better rules loading (#37, #51). This of course cannot be considered v1.0.x branch at least before #14 is completed but can be put to the development branch.

[jarkkojs]

Won't be merging this before v1.0.2 release.

[jarkkojs]

@rafal-krypa: ping. Can you give a test workload for this?

[rafal-krypa]

Here I have put an archive with sample policy directory: test-rules.tar.bz2. It's close to policy that I used for benchmarks.

[jarkkojs]

Thanks! I'll see what's the content. If it looks good I will consider launching a test repository not so much for the functional or unit tests but for performance tests.

[jarkkojs]

com.samsung.swiftkey-lpm.rule fails without this patch (and with this patch with error message). If you apply the patch that I've put to #59. And execute sudo smackload ~/devel/accesses.d/com.samsung.swiftkey-lpm.rule you will get the error message 'Applying rules failed'. There must be a bug in smack_accesses_apply().

[jarkkojs]

OK, now all bugs are fixed (mainly #64 and #65) that prevented efficient testing :) The main problem actually was 3.8 kernel. It seems that change-rule works well on 3.12 but now with 3.8 kernel. I would be interested to hear, which kernel commit fixes this issue.

[jarkkojs]

I generated 50 M (megabyte) rule set in 200 files to test performance (I can generate larger data scripts, created a bash script for that). I'll run that with perf to get some metrics.. Also will apply that for #14. It contains 200000 rules total applying 200 labels so it should be also good test case for merging. I'll publish the generator script soon.

It has only normal at the moment but I will extend it to have also modify rules before I publish it.

[rafal-krypa]

For this patch alone I wouldn't expect any significant performance changes. It will change after combining it with rule merging and sending multiple rules per write to load2 and change-rule.

[jarkkojs]

With the patch I get on my virtual machine:

<issue53> jsakkine@jsakkine-smackbuntu:~/devel/smack-jarkkos.git
$ sudo LD_LIBRARY_PATH=$PWD/libsmack/.libs perf stat -e cpu-clock,task-clock utils/.libs/smackload   ~/testdata/

 Performance counter stats for 'utils/.libs/smackload /home/jsakkine/testdata/':

      10416.555362 cpu-clock                                                   
      10416.551423 task-clock                #    0.998 CPUs utilized          

      10.433981520 seconds time elapsed

Without the patch I get:

<v1.0.x> jsakkine@jsakkine-smackbuntu:~/devel/smack-jarkkos.git
$ sudo LD_LIBRARY_PATH=$PWD/libsmack/.libs perf stat -e cpu-clock,task-clock utils/.libs/smackload   ~/testdata/

 Performance counter stats for 'utils/.libs/smackload /home/jsakkine/testdata/':

      11187.869364 cpu-clock                                                   
      11187.791690 task-clock                #    0.998 CPUs utilized          

      11.211966378 seconds time elapsed

The difference stayed about the same when I tried this multiple times.

I generated the access rules with the following script that I wrote:

#!/bin/bash

LABELS=(`aspell dump master | tr "'" "_" | shuf | head -200`)
ACCESS=('r' 'w' 'x' 'a' 't')
LABELS_LEN=${#LABELS[@]}

function get_access_code()
{
    local result
    for i in `seq 0 1 4`; do
    if test `expr $RANDOM % 2` -eq 0; then
        result="$result${ACCESS[$i]}"
    else
        result="$result-"
    fi
    done

    eval "$1=$result"
}

function print_access_rules()
{
    for i in `seq 0 1 10000`; do
    local subject_i=`expr $RANDOM % $LABELS_LEN`
    local object_i=`expr $RANDOM % $LABELS_LEN`
    local acc=''
    get_access_code acc
    echo ${LABELS[$subject_i]} ${LABELS[$object_i]} $acc >> $1
    done
}

for i in `seq 0 1 200`; do
    print_access_rules $i.txt
done

It's not that much but it is a significant difference.

[jarkkojs]

Now I can merge this patch.