Spurious OSSEC alerts due to THS port scanning

[garrettr]

We received a report on our private support portal about frequent OSSEC alerts triggered by the following message in /var/log/tor/log:

[timestamp redacted] [warn] connection_edge_process_relay_cell (at origin) failed.

After enabling info-level logging for Tor, we were able to correlate this [warn]-level message with preceding [info]-level messages:

[timestamp redacted] [info] rend_service_set_connection_addr_port(): No virtual port mapping exists for port 443 on service qn4qfeeslglmwxgb
[timestamp redacted] [info] connection_exit_begin_conn(): Didn't find rendezvous service (port 443)
[timestamp redacted] [warn] connection_edge_process_relay_cell (at origin) failed.

These messages are triggered by an attempt to open a connection to a hidden service on a port that it does not listen on (e.g. the Source Interface only listens on port 80, but these messages were triggered by attempts to connect on 443). You can easily reproduce it by trying to open a connection to a given hidden service on a port that it does not listen on.

At this point, we decided to analyze some of the logs from the Tor Hidden Services we operate (e.g. for our websites) using our new ELK stack, and found that we were seeing the same patterns.

These log messages appear to be caused by various automated crawlers that try various common "ports of interest" on publicly advertised hidden services (e.g. OnionScan). This type of crawling happens constantly and is usually benign (e.g. for research, testing new tools, etc.).

Since this event is typically benign, it should not trigger dozens of OSSEC alerts a day. These alerts are spurious, waste admin's time and attention, and make it difficult to separate signal from noise in the alerts sent by OSSEC. I propose that we develop a mitigation to avoid sending spurious alerts and include it in the next point release (0.3.8).

[garrettr]

This should be addressed by changing the OSSEC rules to ignore this particular warning in the Tor log.

[garrettr]

There is a related torrc setting, HiddenServiceAllowUnknownPorts, but I am pretty sure we want to leave it on its default setting of 0. Setting it to 1 makes the warning stop appearing, but if I am reading man torrc correctly it also makes things slightly easier for port scanners.

[psivesely]

I can take care of rewriting the OSSEC rules. Besides the slowing down of potential port scanners by forcing them to build a new circuit in order to scan each port (note: not a security feature, just to frustrate these people), the only benefit I can see to this is that since 0 is the default setting we should preserve it as an "anti-fingerprinting" measure.

To elaborate on this latter statement, what I'm worried about is the case where through some means the URL of the ATHS for SSH is obtained and the service is scanned. Either way, the scanner will see that port 22 is open. However, if we switch this to 1 that may be a very unusual setting, and will set us apart from other random onion services who may only offer access via SSH. To be clear, this is so insignificant it's almost not worth mention--I just wanted to share the few thoughts I had on this manner for the sake of thoroughness.

[redshiftzero]

Adding the following rule to /var/ossec/rules/local_rules.xml on mon suppressed these spurious alerts for us:

  <rule id="200001" level="1">
    <if_sid>1002</if_sid>
    <match>connection_edge_process_relay_cell</match>
    <options>no_email_alert</options>
  </rule>

Unless there is another reason one should care about getting email alerts for a subset of these log entries...

[garrettr]

since 0 is the default setting we should preserve it as an "anti-fingerprinting" measure.

Good point @fowlslegs, that is another good reason to keep the default setting.

[garrettr]

@redshiftzero Thanks! That looks like exactly what we need.

Unless there is another reason one should care about getting email alerts for a subset of these log entries...

Not that I am aware of.

[ageis]

Just want to note that I opened a PR on our public ELK stack (ElasticSearch, Logstash and Kibana) repository that will allow us to parse those loglines and better analyze these trends by monitoring the activity on freepress3xxs3hk.onion and secrdrop5wyphb5x.onion. https://github.com/freedomofpress/ansible-role-elk/pull/44

[psivesely]

Thanks @redshiftzero and like @garrettr I also cannot think of any reason

one should care about getting email alerts for a subset of these log entries...

[psivesely]

I thought of a potential reason one might want emails for a subset of these alerts, but dismissed it: https://github.com/freedomofpress/securedrop/pull/1374#issuecomment-239874293.

[psivesely]

Resolved by #1374.