Recovery during recovery test

[karczex]

FEAT: pmreorder based test for power failure during recovery.

Rationale

Process of recovery after power failure involves writing to pmem, so we need to test if it's possible continue it after another power failure.

Implementation details

This may be done in generic way:

1. Run under pmemcheck test code (i.e multi_region_pmreorder test)
2. Run pmreorder check with pmemcheck
3. run pmreorder check for each generated storelog

Tricky part

With little refactor in pmreorder.py, it may easily be changed into python module to be used directly from python. This would tremendously simplify tracking of storelogs.

Roughly tested POC - seems to work

./singly_linked_list_pmreorder create /dev/shm/sllp1

valgrind --tool=pmemcheck -q --log-stores=yes --print-summary=no --log-file=/dev/shm/logfile.storelog --log-stores-stacktraces=no --expect-fence-after-clflush=yes  ./singly_linked_list_pmreorder fill /dev/shm/sllp

pmreorder -l /dev/shm/logfile.storelog -o /dev/shm/x.pmreorder -r ReorderAccumulative -p "/usr/bin/valgrind \-\-tool=pmemcheck \-q \-\-log-stores=yes \-\-print-summary=no \-\-log-file=/dev/shm/logfile-$(cat /proc/sys/kernel/random/uuid).storelog \-\-log-stores-stacktraces=no \-\-expect-fence-after-clflush=yes  ./singly_linked_list_pmreorder check"

pmreorder -l /dev/shm/logfile-be119487-d4c9-43f1-aea5-e28733078988.storelog -o /dev/shm/x.pmreorder -r ReorderAccumulative -p "./singly_linked_list_pmreorder check" 

[igchor]

1. How about running pmreorder inside pmreorder (put last command inside the 3rd one)?
2. In your proposed solution you would also need to get the intermediate states from the first pmreorder call

[lukaszstolarczuk]

Done