All OnionPerf downloads randomly started failing

[robgjansen]

For some reason all attempted downloads on my OnionPerf instance started failing due to READ errors starting on April 1st.

cd onionperf-deata/twistd/docroot
for i in *xz; do echo ${i}; xzcat ${i} | grep "is_error" | sort | uniq -c; done

That shows that there are no successful downloads after the file corresponding to April 1st.

2016-03-31.onionperf.analysis.json.xz
     30             "is_error": false,
2016-04-01.onionperf.analysis.json.xz
    164             "is_error": false,
    124             "is_error": true,
2016-04-02.onionperf.analysis.json.xz
    287             "is_error": true,
2016-04-03.onionperf.analysis.json.xz
    288             "is_error": true,
...

[robgjansen]

@kloesing have you noticed this problem on your instance?

[robgjansen]

Hmmm, looking through my tgen server log files, I am noticing a lot of port scanners or other random clients connecting to my server. My new AUTH mechanism is working somewhat correctly, in that most of the random connections are closed.

However, it looks like somehow one of the port scanners wedged my TGen server; the tgen server process is still alive but not producing any output since April 1st.

Looking through my TGen code that does the authentication check, I realize that my server is reading all the way up to a newline upon receiving an incoming connection, even though the authentication token itself is only 20 bytes long. So I think some smarter authentication check could be useful here, maybe checking one byte at a time and if any of the bytes don't match the token, reject the connection.

[kloesing]

Please find the output of that command in my OnionPerf data directory below:

2016-04-06.onionperf.analysis.json.xz
     65             "is_error": false,
      1             "is_error": true,
2016-04-07.onionperf.analysis.json.xz
    267             "is_error": false,
     21             "is_error": true,
2016-04-08.onionperf.analysis.json.xz
    269             "is_error": false,
     19             "is_error": true,
2016-04-09.onionperf.analysis.json.xz
    276             "is_error": false,
     12             "is_error": true,
2016-04-10.onionperf.analysis.json.xz
    280             "is_error": false,
      8             "is_error": true,
2016-04-11.onionperf.analysis.json.xz
    275             "is_error": false,
     13             "is_error": true,
2016-04-12.onionperf.analysis.json.xz
    268             "is_error": false,
     20             "is_error": true,
2016-04-13.onionperf.analysis.json.xz
    268             "is_error": false,
     20             "is_error": true,
2016-04-14.onionperf.analysis.json.xz
    272             "is_error": false,
     16             "is_error": true,

[robgjansen]

OK, great. So yours seems to be working correctly, but there are some failures - maybe due to Tor, maybe due to port scanners, or maybe both. I'm going to push an update to TGen soon with smarter authentication checking to hopefully minimize future problems (until we eventually have real crypto in place).