xhdix
commented
2 years ago Probe android failed but:
{
"annotations": {
"engine_name": "ooniprobe-engine",
"engine_version": "3.10.0-beta.3",
"flavor": "stableFull",
"network_type": "mobile",
"platform": "android"
},
"data_format_version": "0.2.0",
"input": "http://яндекс.рф/",
"measurement_start_time": "2021-12-17 15:24:14",
"probe_asn": "AS24940",
"probe_cc": "DE",
"probe_ip": "127.0.0.1",
"probe_network_name": "Hetzner Online GmbH",
"report_id": "20211217T152413Z_webconnectivity_DE_24940_n1_8uD59FPZgqD6nL3x",
"resolver_asn": "AS13335",
"resolver_ip": "162.158.83.147",
"resolver_network_name": "Cloudflare, Inc.",
"software_name": "ooniprobe-android",
"software_version": "3.4.1",
"test_helpers": {
"backend": {
"address": "https://wcth.ooni.io",
"type": "https"
}
},
"test_keys": {
"agent": "redirect",
"client_resolver": "162.158.83.147",
"network_events": [
{
"address": "[2a02:6b8:a::a]:80",
"failure": "unknown_failure: dial tcp [scrubbed]: connect: no route to host",
"operation": "connect",
"proto": "tcp",
"t": 1.501672187,
"tags": [
"tcptls_experiment"
]
},
{
"address": "77.88.55.55:80",
"operation": "connect",
"proto": "tcp",
"t": 1.66578177,
"tags": [
"tcptls_experiment"
]
},
{
"address": "5.255.255.5:80",
"operation": "connect",
"proto": "tcp",
"t": 1.667621249,
"tags": [
"tcptls_experiment"
]
},
{
"address": "5.255.255.55:80",
"operation": "connect",
"proto": "tcp",
"t": 1.667557656,
"tags": [
"tcptls_experiment"
]
},
{
"address": "77.88.55.66:80",
"operation": "connect",
"proto": "tcp",
"t": 1.831798281,
"tags": [
"tcptls_experiment"
]
}
],
"queries": [
{
"answers": [
{
"asn": 13238,
"as_org_name": "YANDEX LLC",
"answer_type": "A",
"ipv4": "77.88.55.55"
},
{
"asn": 13238,
"as_org_name": "YANDEX LLC",
"answer_type": "A",
"ipv4": "5.255.255.55"
},
{
"asn": 13238,
"as_org_name": "YANDEX LLC",
"answer_type": "A",
"ipv4": "5.255.255.5"
},
{
"asn": 13238,
"as_org_name": "YANDEX LLC",
"answer_type": "A",
"ipv4": "77.88.55.66"
}
],
"engine": "system",
"hostname": "xn--d1acpjx3f.xn--p1ai",
"query_type": "A",
"resolver_address": "",
"t": 0.539199739
},
{
"answers": [
{
"asn": 13238,
"as_org_name": "YANDEX LLC",
"answer_type": "AAAA",
"ipv6": "2a02:6b8:a::a"
}
],
"engine": "system",
"hostname": "xn--d1acpjx3f.xn--p1ai",
"query_type": "AAAA",
"resolver_address": "",
"t": 0.539199739
}
],
"control_failure": "unknown_failure: httpx: request failed: 400 Bad Request",
"control": {
"http_request": {
"body_length": 0,
"title": "",
"status_code": 0
},
"dns": {}
},
"tcp_connect": [
{
"ip": "2a02:6b8:a::a",
"port": 80,
"status": {
"failure": "unknown_failure: dial tcp [scrubbed]: connect: no route to host",
"success": false
},
"t": 1.501672187
},
{
"ip": "77.88.55.55",
"port": 80,
"status": {
"success": true
},
"t": 1.66578177
},
{
"ip": "5.255.255.5",
"port": 80,
"status": {
"success": true
},
"t": 1.667621249
},
{
"ip": "5.255.255.55",
"port": 80,
"status": {
"success": true
},
"t": 1.667557656
},
{
"ip": "77.88.55.66",
"port": 80,
"status": {
"success": true
},
"t": 1.831798281
}
],
"http_experiment_failure": "unknown_failure: invalid domain in DNSCache",
"body_proportion": 0,
"x_status": 8
},
"test_name": "web_connectivity",
"test_runtime": 1.835072291,
"test_start_time": "2021-12-17 15:24:13",
"test_version": "0.4.0"
}
gurshabad
bassosimone
Let's first document what happens with the current codebase if the input is
http://ουτοπία.δπθ.gras well as its punycode version (http://xn--kxae4bafwg.xn--pxaix.gr/).The rationale of this investigation is to answer https://github.com/ooni/run/pull/81#discussion_r771424651.
The reason why it matters is that we cannot prevent a user from pasting a punycode URL and running the test from command line. So, we need to know what the CLI does in this case (which is a proxy of what mobile would do, given that they use the same implementation). A broader issue is how the rest of the ecosystem would handle it.
Executive Summary
With i18n URL
Web Connectivity
This does not feel good at all. Regardless of where we apply the URL transformation, we cannot prevent a user from trying to test from the command line an i18n URL. Therefore, it's quite bad that the measurement fails.
Websteps
With punycode URL
Web Connectivity
Websteps
I inspected the measurement and it seems to be okay.