False positives

[robredpath]

Thanks for a very useful tool :D

However, we've had some issues using it whereby the website would produce false positives (ie, consider that a server was vulnerable when it wasn't) on servers the first time it was run, but not subsequently. This included servers running OpenSSL 0.9.8. Annoyingly, it's stopped happening now, but I couldn't see a commit where this issue was directly addressed, so I'm not sure if it's fixed itself (urgh) or if it's something fixed by the developer.

We've not seen this when running it on the command line.

Anyone else seen this? Anyone still seeing this?

[plentz]

@robredpath I saw this a few minutes ago as well. 1 false positive e another false negative.

[plentz]

update: now it seems to be giving green light for every site(tested 3 sites that I know that aren't ok yet and all 3 received "all good")

[robredpath]

Are you getting the same behaviour between the site and the CLI tool?

[morsik]

I have a lot of false-positives using command line too. If you run:

for i in $(cat ssl_hosts)
do
    (
        curl -s "http://localhost/bleed/$i" &
    )
done

you'll get also a lot of false positives. Local server can't handle this correcty.

When i do linear checking - results looks correct.

[FiloSottile]

http://filippo.io/Heartbleed/faq.html#loadissue

[robredpath]

Awesome. Thanks for your work on this, @FiloSottile !

That's slightly different from what I was seeing this morning - false reds followed by lots of correct greens.

If it's possible, could some mechanism to verify that the result displayed is definitely from my server be provided? I'm not entirely sure what that would look like but it would help immensely with confidence in results.

[FiloSottile]

Be careful, I can't think of anything that could cause false reds.

More probably you have false greens, check better...

2014-04-08 15:56 GMT+02:00 Rob Redpath notifications@github.com:

Awesome. Thanks for your work on this, @FiloSottilehttps://github.com/FiloSottile!

That's slightly different from what I was seeing this morning - false reds followed by lots of correct greens.

If it's possible, could some mechanism to verify that the result displayed is definitely from my server be provided? I'm not entirely sure what that would look like but it would help immensely with confidence in results.

Reply to this email directly or view it on GitHubhttps://github.com/FiloSottile/Heartbleed/issues/6#issuecomment-39850310 .

[malgorithms]

I believe I saw a couple false positives on the site, too. @FiloSottile you said the command line tool is unaffected, and it's reporting keybase.io:443 as safe:

# passes repeated tests, no FAILS
>./Heartbleed keybase.io:443
2014/04/08 11:06:03 keybase.io:443 - SAFE

And so is the site now. But 20 minutes ago I got a FAIL on the site, reloaded a few more successes, and got a FAIL again. And someone tweeted at me they did too. Now I cannot reproduce this again.

Max updated to 1.0.1g last night.

[pielgrzym]

Nice tool. I'm getting a false positive (red) via web ui / command line for a site running Openssl 0.9.8.

[FiloSottile]

@malgorithms @pielgrzym

I can't honestly think of how a false positive (let's define it as a RED w/ dumped memory including YELLOW SUBMARINE) could happen.

Can you please provide hosts - logs?

[malgorithms]

I can't reproduce it again, but what's your take on Keybase's server right now? We haven't changed anything in the last hour since someone else and I both saw the positive. Can you get a positive?

I never saw a false positive in the command line app. Just on the website. Is there a chance there was a bug (during heavy load? or a race condition?) where site A was getting site B's results back? That would in effect be a presentation bug...

[mboylan]

I also saw a false positive a single time to a web server we have running OpenSSL 9.8. I cannot reproduce it on the website and the CLI tool is continuously reporting back that everything is safe. Unfortunately, for obvious reasons, I cannot share the URL of the server I was testing.

[egp]

I did see one red fail on http://filippo.io/Heartbleed/#keybase.io:443 recently(after Chris announced it was fixed), but now all I get is green pass.

[egp]

Oh, the Irony. http://filippo.io/Heartbleed/#openssl.org:443 gets a red fail

[FiloSottile]

http://filippo.io/Heartbleed/faq.html#falsepositive

[malgorithms]

Thanks @FiloSottile - I'll have the net tab open on future tests, but so far I can't get it to happen again.

[yourcelf]

I have a debian 0.9.8o-4squeeze14 server that is reporting as vulnerable, both with the command line and the hosted service. That OpenSSL version /should/ be safe according to http://heartbleed.com. Either this tool or heartbleed.com's vulnerability table seems to be wrong.

[FiloSottile]

@yourcelf Host please?

[yourcelf]

@FiloSottile You have a private channel I can send to you?

[FiloSottile]

@yourcelf My GH profile email, grab the PGP keys if you want

[r15ch13]

I have the same problem as @yourcelf with version 0.9.8k-7ubuntu8.15

[yourcelf]

OK, figured it out: a backported mod-spdy was pulling in a newer version of OpenSSL that was vulnerable.

[r15ch13]

@yourcelf ha! disabling mod-spdy did the trick. thanks

[mmckinst]

For people affected by mod-spdy, you'll need to disable mod_spdy.so and the mod_ssl_with_npn.so module it comes with. Then load mod_ssl.so instead of mod_ssl_with_npn.so .

mod-spdy bug report: https://code.google.com/p/mod-spdy/issues/detail?id=85

[bradspry]

Tool reports red fail against Debian openssl 1.0.1e-2+deb7u6

But Debian security tracker reports openssl 1.0.1e-2+deb7u6 status as FIXED: https://security-tracker.debian.org/tracker/CVE-2014-0160

Please advise.

[schlamar]

@bradspry Did you forget to reboot after update? =)

[bradspry]

Where I went wrong was using the 'apt-get install --only-upgrade openssl' method, which did not upgrade package 'libssl1.0.0'. Updating package 'openssl' was not enough... You must also update package 'libssl1.0.0'.

[takinbo]

One other thing to note are binaries that have been statically compiled with vulnerable versions of the library. You want to make sure that those are checked as well.

I had an nginx binary that I had to replace to fix the problem.

[pielgrzym]

Easiest way to check what @takinbo mentioned is to run nginx -V and look for custom library paths.

[cjolowicz]

False positive for dealloc.org.

OpenSSL 0.9.8o (0.9.8o-4squeeze14) Apache 2.2.16 (2.2.16-6+squeeze12)

([]uint8) {
 00000000  02 00 79 68 65 61 72 74  62 6c 65 65 64 2e 66 69  |..yheartbleed.fi|
 00000010  6c 69 70 70 6f 2e 69 6f  59 45 4c 4c 4f 57 20 53  |lippo.ioYELLOW S|
 00000020  55 42 4d 41 52 49 4e 45  a9 74 e8 72 ad 0d 71 7e  |UBMARINE.t.r..q~|
 00000030  28 0e 67 ef fe ab 05 c2  44 58 37 86 45 3b 61 af  |(.g.....DX7.E;a.|
 00000040  14 87 3c 78 70 71 14 d8  f2 05 78 f9 49 13 d5 54  |..<xpq....x.I..T|
 00000050  da d5 d3 72 cd 9e 1f 2f  3c eb fb 2f e6 80 90 37  |...r.../<../...7|
 00000060  76 97 40 7f fd 5c 6d ee  96 61 60 92 5b f3 86 f8  |v.@..\m..a`.[...|
 00000070  c5 7d 92 bd 2d 2c 9d ab  e1 8d 7d 7b 6c 84 07 80  |.}..-,....}{l...|
 00000080  50 29 91 7c b1 6c 08 c5  3e 35 3f 98              |P).|.l..>5?.|
}

[horsepunchkid]

@cjolowicz And what version of libssl? Do you use mod-spdy as described above or something else that may be causing your server to use some other version? Did you restart apache?

[JackZielke]

Mine was a positive positive with openssl 0.9.8 (Ubuntu lucid). I had installed mod_spdy which includes a patched, vulnerable, ssl. I did not have spdy enabled, just installed. Updating mod-spdy-beta provided me with a non-vulnerable ssl. I just thought I would mention this.

[paprika60]

I checked a site and it is coming back consistantly red. But when I brought it to the attention of the technical team they said they aren't using OpenSSL. I'm not sure how to respond to them, and don't want to be a total dick if it's a false positive, but does that sound posible? A false positive if they aren't using OpenSSL?

[timayhelen]

Upon my initial test, my website cam back as Red. All subsequent tests, however, return the timeout error. This is the expected result, due to the fact that I am using Litespeed and it blocks the request. I cannot figure out why the original test returned a red result, however.

I flushed my firewall blocks before re-testing, and each test comes back with the timeout error. All other tools that I use to test also confirm that the website is not vulnerable, such as:

https://www.ssllabs.com/ssltest/analyze.html

Do you have any idea why an initial test would come back positive? Could this be a false positive?

[iamthegrackle]

I used the filippo tool and my site came back marked as Vulnerable. However when I got on a chat with my hosting company to ask them about patching it, after the tech checked my server her reported that:

I can see that your OpenSSL version is root@vps-1123238-14690 [~]# rpm -q openssl openssl-1.0.1e-16.el6_5.7.x86_64 The vulnerability is for openssl-1.0.1e-16.el6_5.4.0.1 and below. It seems you are not affected.

Is this correct?

[horsepunchkid]

@iamthegrackle That's probably a question you should ask on a RedHat forum instead, but RedHat's security advisory suggests that your openssl version is good:

https://rhn.redhat.com/errata/RHSA-2014-0376.html (under "Updated packages", openssl-1.0.1e-16.el6_5.7.x86_64.rpm is mentioned as good)

You may have some other package that includes its own bad version of libssl. See the above comments for some examples.

[cjolowicz]

@horsepunchkid I do not use mod-spdy. The version of libssl is 0.9.8o (0.9.8o-4squeeze14). I have not been able to reproduce the false positive, i.e. the host is now always reported as unaffected.

[davidkitfriedman]

@timayhelen

Just wanted to add that the test from Qualys is seeking to test for Heartbleed. Wasn't sure about that at first.

[0100011101001000]

I tested my OpenSSL using websites, all ok after patching. I had installed the Chrome extension though, and strangely enough, I happened to look down and it had found my IIS site support.ccnva.com as being vulnerable! Since I know this is not possible as IIS isn't effected, I just thought I would bring it up. Testing the main web tool does not show it as vulnerable, so the error is in the chrome extension.

[eryno]

@gbhoover, IIS isn't affected, but OpenSSL could be somewhere else in your server stack. For example, an nginx load balancer. Read the section titled "Is it only sites on Apache and nginx that are affected?" in http://www.troyhunt.com/2014/04/everything-you-need-to-know-about.html for more info.

[tsilen]

It would seem that there's some concurrency issue and the response is from time to time somehow leaked from one check to another, or something like that.

Queries directly to http://bleed-1161785939.us-east-1.elb.amazonaws.com/bleed/:

{"code":1,"data":"","error":"","host":"www.google.com"} {"code":1,"data":"","error":"","host":"www.google.com"} {"code":0,"data":"([]uint8) {\n 00000000 02 00 79 68 65 61 72 74 62 6c 65 65 64 2e 66 69 |..yheartbleed.fi|\n 00000010 6c 69 70 70 6f 2e 69 6f 59 45 4c 4c 4f 57 20 53 |lippo.ioYELLOW S|\n 00000020 55 42 4d 41 52 49 4e 45 4e b0 94 1e 5 b 12 db 2a |UBMARINEN...[.._|\n 00000030 f1 f0 2d 00 a3 af eb 83 83 e6 3d 3a 03 03 03 03 |..-.......=:....|\n 00000040 bc b3 72 46 5c 4f 91 80 2e 34 3a b5 52 3f 81 f3 |..rF\O...4:.R?..|\n 00000050 a6 58 0d 8d 69 07 e5 5e 30 4a d9 27 2e 76 7b 01 |.X..i..^0J.'.v{.|\n 00000060 61 07 9b 39 f5 e4 0d 10 5e 2b e7 53 4b b4 e2 fc |a..9....^+.SK...|\n 00000070 6b a6 98 9c df ac 98 2a 57 77 c5 12 26 b8 15 f1 |k......_Ww..\u0026...|\n 00000080 35 a6 f1 09 46 88 83 6b 7a a7 b4 70 |5...F..kz..p|\n}\n","error":"","host":"www.google.com"} {"code":1,"data":"","error":"","host":"www.google.com"} {"code":1,"data":"","error":"","host":"www.google.com"}

{"code":1,"data":"","error":"","host":"example.com"} {"code":0,"data":"([]uint8) {\n 00000000 02 00 79 68 65 61 72 74 62 6c 65 65 64 2e 66 69 |..yheartbleed.fi|\n 00000010 6c 69 70 70 6f 2e 69 6f 59 45 4c 4c 4f 57 20 53 |lippo.ioYELLOW S|\n 00000020 55 42 4d 41 52 49 4e 45 d1 3c fa 8c 7 6 c0 c6 be |UBMARINE.\u003c..v...|\n 00000030 50 b0 50 fc e6 13 46 1b 32 46 30 34 25 32 46 32 |P.P...F.2F04%2F2|\n 00000040 30 31 34 26 64 61 74 65 54 6f 3d 31 32 25 32 46 |014\u0026dateTo=12%2F|\n 00000050 30 35 25 32 46 32 30 31 34 26 72 65 74 75 72 6e |05%2F2014\u0026return|\n 00000060 46 72 6f 6d 3d 26 72 65 74 75 72 6e 54 6f 3d 26 |From=\u0026returnTo=\u0026|\n 00000070 66 6c 79 44 61 79 73 25 35 42 25 35 cc 97 5f 49 |flyDays%5B%5.._I|\n 00000080 d5 06 99 ea 1c d6 74 3b 7e f1 e6 b6 |......t;~...|\n}\n","error":"","host":"example.com"} {"code":1,"data":"","error":"","host":"example.com"} {"code":1,"data":"","error":"","host":"example.com"}

I've seen it happen with multiple domains from time to time (not very often). Probably requires that somebody does a check against a vulnerable one at the same time or just before.

[iamthegrackle]

@horsepunchkid

you said, "You may have some other package that includes its own bad version of libssl. See the above comments for some examples."

I have brought this thread and the examples mentioned to the attention of support at my hosting company, and they insist that everything is fine as long as I ran the server updates they outlined, which I did, yet my site still shows vulnerable using the filippo.io tool. The rep closed by saying,

"Also, I have tested from the following link and it says site is not vulnerable.

https://lastpass.com/heartbleed/?h=sales.mx

You can just ignore any warning from 3rd party sites as openSSL version in your server is already updated."

So either the filippo tool is wrong, or else they are totally disregarding the issues raised here and I would wonder just how many of the VPS customers at this very large host are still vulnerable ...

[FiloSottile]

@iamthegrackle

That site is vulnerable. Full stop.

Feel free to link support to this comment and if they have different evidence they can get in contact with me over email. The address is in my profile.

[iamthegrackle]

thanks. I let them know. they responded with a link to filippo.io with my IP address (ip::2087) instead of my domain name like I was doing. And when the 'ignore certificate' is checked, then the result is that it is fixed or unaffected.

So it my ip:2087 is used then it says fixed, if my domain is used then it says vulnerable.

[FiloSottile]

Have a look at the usual: load balancers, SSL terminators, different host-based setup, CDN, mod_SPDY...

2014-04-12 23:29 GMT+02:00 Michael notifications@github.com:

thanks. I let them know. they responded with a link to filippo.io with my IP address (ip::2087) instead of my domain name like I was doing. And when the 'ignore certificate' is checked, then the result is that it is fixed or unaffected.

So it my ip:2087 is used then it says fixed, if my domain is used then it says vulnerable.

Reply to this email directly or view it on GitHubhttps://github.com/FiloSottile/Heartbleed/issues/6#issuecomment-40292266 .