FPT_AEX_EXT.5 test devices?

[woodbe]

https://github.com/commoncriteria/mobile-device/blob/954ded4330200e30c1e4b35b7b7398b6d6a3598c/input/mobile-device.xml#L6997-L7000

This has the tester take one device and reboot 5 times, then says that no mappings should be in the same on both devices. Does the test need to be done on multiple devices, or is this that in the 5 reboots it should not be in the same place?

Also, this is only 4 random bits, which only gives 16 "options" in terms of the position. Out of 5 tests it would seem likely to run into one duplicate. In the ASLR section for apps there is some additional text about what do do if the apps load into the same location (and that has 8 bits, not the 4 bits here). It seems like some of that text should be added. Obviously if the vendor uses more bits this becomes less of a problem, but it does seem like it should be further clarified.

[lewyble]

@kgal - thoughts?

[kgal]

I agree with the argument given. According to my calculations there's about a 50% chance of this test succeeding (or failing) with this requirement being minimally implemented (i.e. 16/16 15/16 14/16 13/16*12/16 ~= 50%).

Can we do better? Not without some additional burden on the testers, but I think so:

Assuming this birthday problem calculator is accurate, we could have 16 trials (reboots). Allowing at most one duplicate would give us 20% false pass rate for 3 bits and 23% failure rate for 4 bits. Allowing testers to repeat the entire process once for a failed test would pull the false failure rate down to 5% and increase the false pass rate to 24%. I think this is doable.

Allowing for no repeats is very constricting and doesn't give a lot of separation between 3 bits and 4 bits. Having 3 reboots with no repeats would false fail for 4 bits only 18% of the time while false passing 63% of the time. Allowing a single retested would reduce false fail to 3% while false passing 84% of the time. This might be okay as I'm only considering 3 bits, for less, the false pass rate is considerably lower and these are the really bad implementations.

[woodbe]

Just as an additional thought,5.2 could be adjusted to something like this:

The base address of any kernel-space memory mapping will consist of [selection: 4, [assignment: a number greater than 4]] unpredictable bits.

Then the testing could be readily adjusted to note this. As an example, Samsung actually provides 6 bits for KASLR, so having that would make the testing easier for meeting 4, which would be fine, but also allow for an explicit claim of a higher number. The testing could even provide a number of tests based on the number of bits, though we would need to be careful with the number of tests. For example when I run an evaluation, we may be testing 9 devices at once (similar for Apple from what I have seen), so you are talking about 16 reboots on something like 10 devices (yes, I do get that they can be done in parallel, but it is still a LARGE amount of work). Increasing that for higher bits to get a similar number would be even more of an issue, so a maximum number of tests may be useful (i.e. test 16 times for 4 bits, 20 for 5 bits and 25 for 6 bits), just to limit the amount of time. I would love to assign a percentage above 16, but that could get high quickly (something like 16 or 40% of the total number of possible outcomes, whichever is higher).

Either way, I would like to see a selection that would let a vendor claim a higher number of bits for the KASLR, even if it didn't lead to any different testing (since the 4 bits and the testing is about ensuring that as the base).

[kgal]

I agree that vendors should be able to specify the number of unpredictable bits...we probably don't need the the selection, just the assignment. And would making this requirement threshold (instead of objective) and increasing the minimal number of bits to 6 be unreasonable? Also I think the evaluation activity need only check that the minimum number of bits (be it 4 or 6 or whatever) be implemented.

[woodbe]

So at the moment, the two main OSes undergoing evaluation are Android and iOS (nothing else has been evaluated to the MDF v3 that I know of or can find), and Android devices are claiming this requirement, but iOS is not. So I wouldn't want to make this mandatory unless it is confirmed that this is possible for iOS (or being done but they just haven't bothered to claim it, which is always a possibility).

I would be Ok with it just being an assignment with a number of 4 or higher (I did the selection so it was clear that was the "default"), but I wouldn't want to enforce something that I do not know is ready to be met across all platforms.

@DewSecGitHub or @The-Fiona should chime in on whether to make this mandatory since they could answer that question better then me.

[kgal]

According to this irrefutable source, iOS has 8 bits for KASLR. But, yeah, something more authorative than the Wikipedia page should be consulted.