Different detection logic used by Android and iOS apps

[hellais]

It has been reported by users that the middlebox test results report inconsistent results between iOS and Android. The recent reports, though, are false positives and the root cause is that one of the test helpers was down following the migration I was doing of them (see: https://github.com/ooni/sysadmin/issues/377)

I suspect this may be due to subtle differences in how the is_anomaly value is being calculated and evaluated.

I looked up the code for calculating the value in Android & iOS but it does seem OK: https://github.com/ooni/probe-android/blob/73afe98f9307c0c764eee92767f10b5b4645c444/app/src/main/java/org/openobservatory/ooniprobe/test/test/HttpHeaderFieldManipulation.java#L31 https://github.com/ooni/probe-ios/blob/6395dc7a48d33c219afee2c965f1e40f927476fc/ooniprobe/Test/Test/HttpHeaderFieldManipulation.mm#L33

It may be useful to try to reproduce the bug to feed a measurement of this test from the affected period from iOS and Android and compare what the app does.

Here are some measurements to use as test cases: https://explorer.ooni.org/measurement/20191027T235150Z_AS8048_M0hdIeSk38N0dP4gF2Bk2cS0YYUbqpk9g1UYuQ3ho3NmUAMY1Q https://explorer.ooni.org/measurement/20191027T235113Z_AS262827_k3jWYSMnHkH7AOBqid3A9aRkGrHEz69S10Pw4y8wPyBxfzk9u2

[hellais]

I was able to reproduce this problem by comparing the view of the same result on Android and iOS:

{
  "annotations": {
    "engine_name": "libmeasurement_kit",
    "engine_version": "0.10.6",
    "engine_version_full": "v0.10.6",
    "flavor": "full",
    "network_type": "wifi",
    "platform": "android"
  },
  "data_format_version": "0.2.0",
  "id": "0b2a2bbe-a778-4a64-8cd0-c4da8cd89a79",
  "input_hashes": [],
  "measurement_start_time": "2019-10-28 13:27:38",
  "options": [],
  "probe_asn": "AS30722",
  "probe_cc": "IT",
  "probe_ip": "127.0.0.1",
  "report_id": "",
  "software_name": "ooniprobe-android",
  "software_version": "2.2.0-beta.3",
  "test_helpers": {
    "backend": "http://37.218.247.95:80"
  },
  "test_keys": {
    "agent": "agent",
    "client_resolver": "172.253.197.4",
    "requests": [
      {
        "request": {
          "body": "",
          "headers": {
            "AcCEPt-LAnGuagE": "en-US,en;q=0.8",
            "AcCept-cHArSet": "ISO-8859-1,utf-8;q=0.7,*;q=0.3",
            "HoST": "Y9efH2LfN6VBFzZ.com",
            "aCCEpT-enCoDInG": "gzip,deflate,sdch",
            "accePT": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
            "uSER-AGEnt": "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6"
          },
          "method": "GET",
          "tor": {
            "is_tor": false
          },
          "url": "http://37.218.247.95/"
        },
        "response": {
          "body": {
            "data": "H4sIAAAAAAAAA7PJKMnNsePlsslITUyxsynJLMlJtTMxMFHwyy9RcMsvzUux0YcI2uiDlQCVJuWnVIK0JKfmlaQW2dlkGKLrAIrY6EOlQWYDFUF5eemZeRXIcvow0/ShLgEAC9lhM5IAAAA=",
            "format": "base64"
          },
          "code": 404,
          "headers": {
            "Connection": "keep-alive",
            "Content-Encoding": "gzip",
            "Content-Type": "text/html",
            "Date": "Mon, 28 Oct 2019 13:27:42 GMT",
            "Keep-Alive": "timeout=120",
            "Server": "nginx",
            "Transfer-Encoding": "chunked"
          },
          "response_line": "HTTP/1.1 404 Not Found"
        }
      }
    ],
    "tampering": {
      "request_line_capitalization": true,
      "total": true
    }
  },
  "test_name": "http_header_field_manipulation",
  "test_runtime": 0.4194920063018799,
  "test_start_time": "2019-10-28 13:27:35",
  "test_version": "0.0.1"
}

{
  "data_format_version" : "0.2.0",
  "software_version" : "2.2.0-beta.3",
  "annotations" : {
    "engine_name" : "libmeasurement_kit",
    "engine_version_full" : "v0.10.6",
    "platform" : "ios",
    "engine_version" : "0.10.6",
    "network_type" : "wifi"
  },
  "test_helpers" : {
    "backend" : "http:\/\/37.218.247.95:80"
  },
  "options" : [

  ],
  "test_name" : "http_header_field_manipulation",
  "test_version" : "0.0.1",
  "probe_cc" : "IT",
  "input" : null,
  "input_hashes" : [

  ],
  "probe_ip" : "127.0.0.1",
  "id" : "fb8379e2-635d-4cdb-9781-f9478f02e74e",
  "report_id" : "20191028T095138Z_AS30722_i34Sbq8ryra6TCuBIvMZbq6IwqJDxvedqplxr8EKgoecBtyb6r",
  "probe_city" : null,
  "test_keys" : {
    "tampering" : {
      "request_line_capitalization" : true,
      "total" : true,
      "header_name_diff" : null,
      "header_field_name" : null
    },
    "failure" : "connection_refused",
    "client_resolver" : "172.253.197.5",
    "agent" : "agent",
    "requests" : [
      {
        "failure" : "connection_refused",
        "request" : {
          "body" : "",
          "method" : "GET",
          "headers" : {
            "AccepT-lANGuAgE" : "en-US,en;q=0.8",
            "ACCepT-encoDIng" : "gzip,deflate,sdch",
            "accept" : "text\/html,application\/xhtml+xml,application\/xml;q=0.9,*\/*;q=0.8",
            "accept-ChArSet" : "ISO-8859-1,utf-8;q=0.7,*;q=0.3",
            "user-AgeNT" : "Mozilla\/5.0 (iPhone; U; CPU iPhone OS 3 1 2 like Mac OS X; en-us) AppleWebKit\/528.18 (KHTML, like Gecko) Mobile\/7D11",
            "hOsT" : "ofHqTPD3OHp94mE.com"
          },
          "url" : "http:\/\/37.218.247.95\/",
          "tor" : {
            "exit_ip" : null,
            "exit_name" : null,
            "is_tor" : false
          }
        },
        "response" : {
          "headers" : {

          },
          "body" : null
        }
      }
    ],
    "socksproxy" : null
  },
  "measurement_start_time" : "2019-10-28 09:51:38",
  "test_runtime" : 0.041224002838134766,
  "probe_asn" : "AS30722",
  "test_start_time" : "2019-10-28 09:51:36",
  "software_name" : "ooniprobe-ios"
}

[lorenzoPrimi]

The problem here in is the evaluation of the tampering object. https://github.com/ooni/probe-android/blob/73afe98f9307c0c764eee92767f10b5b4645c444/app/src/main/java/org/openobservatory/ooniprobe/model/jsonresult/TestKeys.java#L299 https://github.com/ooni/probe-ios/blob/6395dc7a48d33c219afee2c965f1e40f927476fc/ooniprobe/Model/JsonResult/Tampering.m

Which interpretation is correct, iOS or Android?

[hellais]

Which interpretation is correct, iOS or Android?

Can you describe the logic currently being used to determine blocking in both platforms?

It's not super clear to me what is going on there.

[lorenzoPrimi]

tampering is an OR of these values in Android return header_field_name || header_field_number || header_field_value || header_name_capitalization || request_line_capitalization || total;

While in iOS for every of these keys we check if the key exists, it's not null and finally the bool value.

 if ([tampering objectForKey:key] &&
                    [tampering objectForKey:key] != [NSNull null] &&
                    [[tampering objectForKey:key] boolValue]) {
                    self.value = YES;
                }

[lorenzoPrimi]

So based on yesterday discussion we will move this detection on measurement kit, @bassosimone confirmed?

[bassosimone]

That is a long time from now, so maybe it's better to still make sure the logic is uniform.

[lorenzoPrimi]

In this case I would need one of you to describe me the correct algoritm based on the iOS and Android implementations (they are up there) @bassosimone @hellais

[hellais]

I don't think doing this in the app is a good use of our time.

I would rather we work on refactoring these views in react-native and have a uniform implementation of them in react-native or do it in the golang code.

[lorenzoPrimi]

This is not a UI thing that can be solved in react, it's a class implementation can should either be solved in the current app as quick fix or in golang. Golang as @bassosimone stated is long time from now, so I'd suggest 1) Give me the algorithm in pseudo code so I can compare it to the current app code. @bassosimone 2) look quickly at this code and see if something is not specular:

Android

    public boolean isAnomaly() {
        return header_field_name || 
        header_field_number || 
        header_field_value || 
        header_name_capitalization || 
        request_line_capitalization || 
        total;
    }

NOTE: these are boolean so a null value will initialize as false.

iOS:

 if ([tampering objectForKey:key] &&
                    [tampering objectForKey:key] != [NSNull null] &&
                    [[tampering objectForKey:key] boolValue]) {
                    self.value = YES;
                }

[hellais]

This is probably something we want to do inside of the golang engine and expose some function to compute the isAnomaly for the apps.

cc @bassosimone

[bassosimone]

This is probably something we want to do inside of the golang engine and expose some function to compute the isAnomaly for the apps.

The current approach with the apps is to expose to them the same API of MK, where all the communication is done using JSON documents. Hence, it may be that the best approach here is to compute the required output variables in any case and put them in a place where apps could easily fetch it. Ideally this place could be the measurement itself.

[hellais]

It might make sense to do this together with the migration to the new golang based API: https://github.com/ooni/probe/issues/1410