search

OpenDRR / opendrr

Parent project for the OpenDRR Platform / Projet parent pour la plate-forme OpenDRR
7 stars 2 forks source link

opendrr.github.io misclassified as malicious by some security scanners (and blocked by CIRA Canadian Shield DNS servers) #122

Closed anthonyfok closed 1 year ago

anthonyfok commented 3 years ago

To check

VirusTotal

Microsoft Defender SmartScreen

Check from Microsoft Edge or Internet Explorer 11

Progress

TODO (as of 2023-03-08):

No more blockers! Hurray!

Resolved

Original message

At today's meeting (April 14), I failed to open Joost's latest super-fast Elasticsearch based dynamic map web page on https://opendrr.github.io/, and then I discovered I couldn't open any pages under that domain.

It was due to the CIRA Canadian Shield "Projected" DNS servers that I was using (149.112.121.20 and 149.112.122.20), see https://www.cira.ca/cybersecurity-services/canadian-shield

Normally, opendrr.github.io points to GitHub IP addresses:

opendrr.github.io.  3464    IN  A   185.199.108.153
opendrr.github.io.  3464    IN  A   185.199.110.153
opendrr.github.io.  3464    IN  A   185.199.111.153
opendrr.github.io.  3464    IN  A   185.199.109.153

But 149.112.121.20 and 149.112.122.20 ("Protected") and 149.112.121.30 and 149.112.122.30 ("Family") point opendrr.github.io to:

$ dig @149.112.121.20 opendrr.github.io
...
;; ANSWER SECTION:
opendrr.github.io.  0   IN  A   75.2.78.236
opendrr.github.io.  0   IN  A   99.83.179.4

Both 75.2.78.236 and 99.83.179.4 302 redirect to https://www.cira.ca/CanadianShield/Active/MalwareBlock

Switching to the less protective "Private" DNS, i.e. 149.112.121.10 and 149.112.122.10, or switching to Google's Public DNS, for example, unblocked opendrr.github.io for me personally.

But yeah, the main point is how to take our website off their blacklist, and hopefully find out how we got on their blacklist in the first place.

According to https://www.cira.ca/cybersecurity-services/canadian-shield/faq-public:

How do you prevent false positives (i.e. accidentally blocking a legitimate domain)?

CIRA Canadian Shield leverages a threat feed that is global and used by ISPs around the world and is designed to have a very low false positive rate. The threat feed is used for the CIRA DNS Firewall that currently protects 1.8 million Canadian users and the rate of false positives to legitimate queries is something very close to zero. Please use our support form if you believe we are blocking a domain in error.

How do I report a false positive or a previously infected domain that has been cleaned up?

Based on our experience running a commercial version of the service, CIRA Canadian Shield has a very low false positive rate having only lodged a handful of requests on over 1.8 million users. Most times, domains that are reported to us as a false positive are found to be hosting malicious content without the knowledge of the domain owner. If you believe that your domain is being blocked incorrectly by CIRA Canadian Shield then please visit our support page to lodge the request for review.

If your site has been hijacked or misused by hackers and as a result has been placed on block lists (including ours) then you are in a very difficult situation. Once the problem has been rectified on your end you can request a review using our support page. This can involve multiple global vendors and so we cannot provide a time-frame for when the review will be complete.

I will be filing a support request here:

P.S. Related to to #119.

anthonyfok commented 3 years ago

Here is what I submitted at https://www.cira.ca/cybersecurity-services/canadian-shield/support-public (based on what I submitted to Google Safe Browsing about a month ago, see #119):

Hello,

I would like to report a false positive regarding opendrr.github.io, which I believe was placed on your malware blacklist in error.

OpenDRR (Open Disaster Risk Reduction) Platform is a project under Natural Resources Canada, Government of Canada. It is middleware between hazard or risk modeling environments like OpenQuake and end users who need to understand and evaluate risk to make economic and policy decisions. Please see https://github.com/OpenDRR/opendrr-platform for more information.

The https://opendrr.github.io/ is being served on GitHub Pages from three repos, namely:

The web pages, though incomplete, are up for demo and internal development. Besides rendered HTML files (from Markdown and AsciiDoc), OGC GeoPackage format data files (*.gpkg) for Canadian provinces and territories are offered for download. There are also some design documentation in PDF format, though not in GitHub Pages. Dynamic maps will be made available in the near future.

The above resources are all that is hosted as Jekyll-based statically generated website on https://opendrr.github.io/, so there is no phishing going on at https://opendrr.github.io/ to the best of our knowledge. Thank you.

We are currently tracking this as a GitHub issue too at https://github.com/OpenDRR/opendrr-platform/issues/122

We ran into a similar false positive, but with Google Safe Browsing, about a month ago. Thankfully, Google took us off the blacklist within 24 hours after reported the false positive, see our record at https://github.com/OpenDRR/opendrr-platform/issues/119

But yes, we wonder why our website would trigger such false positives in the first place. We would really appreciate it if you could shed some lights on this matter. Thank you very much!

Yours sincerely,

Anthony Fok anthony.fok@canada.ca on behalf of OpenDRR, Natural Resources Canada

anthonyfok commented 3 years ago

Based on advice found in https://community.webroot.com/webroot-secureanywhere-antivirus-12/reporting-webroot-false-positive-292670, I visited https://detail.webrootanywhere.com/servicewelcome.asp and submitted a support ticket:

Subject: False positive: opendrr.github.io listed as "Malicious" in error?

Please help by categorizing your issue Threat Found - False Positive

Message


Hello,

I would like to report a false positive regarding opendrr.github.io, which I believe was listed "Malicious" in error in the Webroot entry of the VirusTotal scan at https://www.virustotal.com/gui/url/419e325c8aac45e2422075e15c3561c00ea43e4683ae6c955afefec5a1ef4f81/detection

OpenDRR (Open Disaster Risk Reduction) Platform is a project under Natural Resources Canada, Government of Canada. It is middleware between hazard or risk modeling environments like OpenQuake and end users who need to understand and evaluate risk to make economic and policy decisions. Please see https://github.com/OpenDRR/opendrr-platform for more information.

The https://opendrr.github.io/ is being served on GitHub Pages, three of which are public facing, namely:

The web pages, though incomplete, are up for demo and internal development. Besides Jekyll rendered HTML files (from Markdown and AsciiDoc), OGC GeoPackage format data files (*.gpkg) for Canadian provinces and territories are offered for download. There are also some design documentation in PDF format, though not in GitHub Pages. Dynamic maps will be made available in the near future.

The above resources are all that is hosted as Jekyll-based statically generated website on https://opendrr.github.io/, so there is no phishing going on at https://opendrr.github.io/ to the best of our knowledge. Thank you.

We are currently tracking this as a GitHub issue too at https://github.com/OpenDRR/opendrr-platform/issues/122

We ran into a similar false positive, but with Google Safe Browsing, about a month ago. Thankfully, Google took us off the blacklist within 24 hours after reported the false positive, see our record at https://github.com/OpenDRR/opendrr-platform/issues/119

But yes, we wonder why our website would trigger such false positives in the first place. We would really appreciate it if you could shed some lights on this matter. Thank you very much!

Yours sincerely,

Anthony Fok anthony.fok@canada.ca on behalf of OpenDRR, Natural Resources Canada

anthonyfok commented 3 years ago

Webroot Support replied very quickly on the same day (Thu 2021-04-15):

Hello,

Thank you for contacting Webroot Support.

If you have a website that you feel is incorrectly identified, please follow the steps below to submit a change request.

  1. First, click here to open Webroot's URL Reputation Change Request page.

  2. Enter the web address or IP of the page you are submitting in the URL or IP field.

  3. After rating the URL, please enter your email address and product type (e.g. Webroot SecureAnywhere for Mac).

  4. Click the box next to “I’m not a robot” and wait for the checkmark to appear.

  5. Click "Submit" to send your request.

A response is generally provided within 24-48 hours. You can also view further details about our website reputation change process by clicking here.

Please let us know if you have any additional questions or concerns.

Thank you,

The Webroot Support Team

So, I submitted the following request at http://www.brightcloud.com/tools/change-request-url-reputation.php to change the classification to "Government", "Society", and "Reference and Research", with the comment "It would be great if you could shed some light as to what caused this false positive. Many thanks!" (terse because of the 150-characters limitation), as seen in the screenshot below:

Screenshot from 2021-04-15 11-09-02

And, again, kudos to Webroot BrightCloud, I received a very speedy response

Hello,

The Webroot BrightCloud Threat Intelligence scanners identified malicious activity on this website.

Unfortunately we cannot disclose the proprietary intelligence data collected regarding this site. If further details are needed you may elect to contact the webmaster or website owner in order to request more information about any recent changes to the site source code that may have triggered external scanners to flag the site as malicious.

For more details, you can always refer to free public sources such as https://www.virustotal.com/gui/url/5df3642a4c13b0da3a8e84b4f133a4885a2690cb2d89681fe75827c4f74f8463/detection.

Thank you,
Webroot BrightCloud Threat Intelligence Support

So, the next day (Fri 2021-04-16), I submitted a new request, this time specifying that I am one of the webmasters, and asserted that the "malicious" status was a misclassification:

Screenshot from 2021-04-16 12-40-45

And received the following speedy and positive response on the same day:

Hello again -

We have reviewed hxxps://opendrr[.]github[.]io/, and although we did not take your suggestion exactly, we have updated the site to the Reference and Research categories. This change is now published in the BrightCloud Service and is available in Database version 7.721.

Thanks again for your suggestion!

  • Webroot BrightCloud Threat Intelligence Support

Today (Wed 2021-04-21), I finally replied:

Thank you for your speedy response, and thank you updating our website to the Reference and Research categories, and thus taking our website off the “malicious” list, and Webroot now lists our site as “clean” at VirusTotal: https://www.virustotal.com/gui/url-analysis/u-5df3642a4c13b0da3a8e84b4f133a4885a2690cb2d89681fe75827c4f74f8463-1619026721/detection

While I understand that you “cannot disclose proprietary intelligence data collected regarding this site”, could you be so kind as to offer a tiny hint as to what might have triggered the false positive? Was it one of the ZIP archive of .gpkg files that the site links to? Or was it the website metadata saying it is a Canada.ca website while it is hosted on github.io? (I failed to change the Government of Canada Web Experience Toolkit, esp. GCWeb theme template metadata from the boilerplate) for our demo website, and finally got it fixed last week.)

Anyhow, thank you so much for your kind help!

Best regards,

Anthony Fok anthony.fok@canada.ca OpenDRR Project, Geological Survey of Canada (GSC Pacific), Natural Resources Canada

So, many kudos and thanks to Webroot BrightCloud Threat Intelligence Support, whose positive and pleasant interactions with me gives me much hope for continuing the "restoring our site's reputation" journey with other security scanners which currently misclassify our site according to VirusTotal, namely:

anthonyfok commented 3 years ago

CRDF

Request submitted on Mon 2021-05-03 at around 9:03 a.m.

https://threatcenter.crdf.fr/false_positive.php Screenshot from 2021-05-03 10-04-26

[TO BE CONTINUED]

November 2022

Still positive (or became positive again?) when checked on 2022-11-28. Submitted again:

image

and CRDF now says we are clean (according to VirusTotal) within minutes! Hurray! (Hope it stays that way.)

anthonyfok commented 3 years ago

Fortinet

Where to report? This forum discussion How to report / remove a false positive | Fortinet Technical Discussion Forums at https://forum.fortinet.com/tm.aspx?m=97165 from 2013 offers helpful info.

which shows:

Screenshot from 2021-05-03 10-38-37

So, FortiGuard supposedly removed the "Phishing" rating from our site on March 12, and re-classified it as Information Technology on April 14, so we must be clean right? But not on VirusTotal where Fortinet still lists us as "Phishing". When the mouse hovers over the word "Fortinet", it says:

May differ from commercial off-the-shelf product. The company decides the particular settings with which the engine should run in VirusTotal.

So, outdated database? Or other unexplored Fortinet listings other than "Web Filter"? See the (full?) list at https://www.fortiguard.com/learnmore#ips . The ones I have tried thus far are:

[TO BE CONTINUED]

November 2022

Note: The link we got from within VPN has a special id in it. (supposedly takes only 5 minutes?)

For query and submission outside of VPN, use https://globalurl.fortinet.net/rate/submit.php

for http://opendrr.github.io (2022-11-28)

Hello,

https://opendrr.github.io/ (including http://opendrr.github.io/ which should redirect to the HTTPS site) is a legitimate website used by the Open Disaster Risk Reduction (OpenDRR) team at the Geological Survey of Canada, Natural Resources Canada, to provide data for download and additional information regarding earthquake scenarios and probabilistic earthquake risk, etc.

Our recently (soft-)launched RiskProfiler website refers to https://opendrr.github.io/ for data download:

https://www.riskprofiler.ca/download-data/index.html

The fact that we are not currently using a .ca domain does not mean we are phishing. Rather, it is a demonstration of Natural Resources Canada, as part of Government of Canada, doing Open Science and Open Data with full transparency, public platforms such as GitHub and GitHub Pages

I have actually filed a similar request to FortiGuard back on March 12, 2021, and wondered why it didn't reflect in Fortinet. Here is what I recorded in our GitHub issue tracking this:

So, FortiGuard supposedly removed the "Phishing" rating from our site on March 12, [2021,] and re-classified it as Information Technology on April 14, [2021,] so we must be clean right? But not on VirusTotal where Fortinet still lists us as "Phishing"...

Somewhat ironically, our site is now blocked by our own internal network / VPN which uses Fortinet for malicious website detection, though it may be a blessing in disguise as I am finally directed to the right place to report the misclassification to you.

Please see https://github.com/OpenDRR/opendrr/issues/122 for more information on our OpenDRR team's quest to clear our name and GitHub Pages domain opendrr.github.io.

Thanks in advance for your kind consideration!

Best regards,

Anthony Fok Software Integration and Deployment Specialist Natural Resources Canada

Result: https://globalurl.fortinet.net/rate/support.php?ref=11ca85

for https://opendrr.github.io (2022-11-28)

Hello,

https://opendrr.github.io/ (including http://opendrr.github.io/ which should redirect to the HTTPS site) is a legitimate website used by the Open Disaster Risk Reduction (OpenDRR) team at the Geological Survey of Canada, Natural Resources Canada, to provide data for download and additional information regarding earthquake scenarios and probabilistic earthquake risk, etc.

Our recently (soft-)launched RiskProfiler website https://www.riskprofiler.ca/download-data/index.html refers to https://opendrr.github.io/ for data download.

The fact that we are not currently using a .ca domain does not mean we are phishing. Rather, it is a demonstration of Natural Resources Canada, as part of Government of Canada, doing Open Science and Open Data with full transparency, public platforms such as GitHub and GitHub Pages

I have actually filed a similar request to FortiGuard back on March 12, 2021, and wondered why it didn't reflect in Fortinet. Here is what I recorded in our GitHub issue tracking this:

So, FortiGuard supposedly removed the "Phishing" rating from our site on March 12, [2021,] and re-classified it as Information Technology on April 14, [2021,] so we must be clean right? But not on VirusTotal where Fortinet still lists us as "Phishing"...

Somewhat ironically, our site is now blocked by our own internal network / VPN which uses Fortinet for malicious website detection, though it may be a blessing in disguise as I am finally directed to the right place to report the misclassification to you.

Please see https://github.com/OpenDRR/opendrr/issues/122 for more information on our OpenDRR team's quest to clear our name and GitHub Pages domain opendrr.github.io.

Thanks in advance for your kind consideration!

Best regards,

Anthony Fok Software Integration and Deployment Specialist Natural Resources Canada

P.S. Sorry if this is a duplicate request. I received https://globalurl.fortinet.net/rate/support.php?ref=11ca85# which tells me our website has been reclassified as "Information Technology", but in actual fact, Fortinet still classifies our website in the "Phishing" category. Or perhaps it just take a day or two to actually take effect?

Result: https://globalurl.fortinet.net/rate/support.php?ref=Y1ba7d (which says "The category is now Information Technology", but the master database isn't actually changed? To check again tomorrow.)

anthonyfok commented 2 years ago

Good news! CIRA Canadian Shield DNS has taken our opendrr.github.io domain off the blacklist!

I don't know when it happened; probably in late June or in July? I did not document it back then, but glad to see that it is still working today.

Here is a DNS query from today, with the A records correctly pointing to GitHub servers:

$ dig @149.112.121.20 opendrr.github.io

; <<>> DiG 9.16.15-Debian <<>> @149.112.121.20 opendrr.github.io
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 11079
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;opendrr.github.io.     IN  A

;; ANSWER SECTION:
opendrr.github.io.  3600    IN  A   185.199.110.153
opendrr.github.io.  3600    IN  A   185.199.111.153
opendrr.github.io.  3600    IN  A   185.199.108.153
opendrr.github.io.  3600    IN  A   185.199.109.153

;; Query time: 88 msec
;; SERVER: 149.112.121.20#53(149.112.121.20)
;; WHEN: Thu Aug 05 17:50:35 MDT 2021
;; MSG SIZE  rcvd: 110

There are still one or two handful of security scanners (as listed on VirusTotal) that still flag us. I'll keep this issue open for the time being.

jvanulde commented 2 years ago

Good news. Thanks @anthonyfok

jvanulde commented 2 years ago

@anthonyfok can we close yet?

anthonyfok commented 1 year ago

@anthonyfok can we close yet?

@jvanulde Sorry, not yet. I failed to follow up with all the remaining misclassifying security scanners, and we just discovered (thanks to @NickyHastings, @wkhchow, etc.) that opendrr.github.io was inaccessible through VPN, and then we found out we were still being blocked by Fortinet. (Just fixed supposedly.)

I'd better stop being lazy and follow up with the remaining 8 security scanners which still misclassify us. 😅

jvanulde commented 1 year ago

@anthonyfok I ran into this on VPN. today.

anthonyfok commented 1 year ago

VIPRE

About VIPRE Security Group: https://en.wikipedia.org/wiki/VIPRE

False positive as see on VirusTotal

image

Submitting a request on 2023-02-03

After clicking "Submit", there is a small temporary message on the upper-right corner saying "Your request was successfully submitted." on https://helpdesk.vipre.com/hc/en-us?return_to=%2Fhc%2Frequests.

I forgot to mention VirusTotal... Oops!

Hope to hear from them soon!

Update

Hurray! VIPRE quickly resolved our support ticket #363835 and unblocked opendrr.github.io of 2023-02-06 UTC (2023-02-05 in Canada.)

anthonyfok commented 1 year ago

Trellix

About Trellix

According to Wikipedia, "Trellix (formerly FireEye and McAfee Enterprise) is a privately held cybersecurity company founded in 2022"; see https://en.wikipedia.org/wiki/Trellix

According to ZDNET, "During 2021, Symphony Technology Group (STG) picked up McAfee Enterprise for $4 billion in March, and followed it up in June with a $1.2 billion purchase of FireEye. With the merger of the two cybersecurity firms completed in October, the companies have been given a new name. The name is Trellix..." See https://www.zdnet.com/article/mcafee-enterprise-and-fireeye-are-now-called-trellix/

How is Trellix blocking opendrr.github.io

When trying to open https://opendrr.github.io/ from an NRCan laptop using Google Chrome or Microsoft Edge:

Note that the Trellix Endpoint Security Web Control extension does not kick when using as Guest, or in Incognito / InPrivate mode, unless, of course, the end user (e.g. me) explicitly allows the extension to load in Incognito / InPrivate mode.

On an NRCan laptop, the following apps are installed:

Where to report?

I don't know! I haven't been too successful in finding the online submission form or an email address, but I wasn't looking too hard either.

Note that since Trellix (former McAfee) and McAfee (still McAfee, for personal security products?) are two separate companies now, perhaps the false positive need to be reported to both companies?

To be continued.

anthonyfok commented 1 year ago

McAfee / Trellix (continued)

Today, after getting a friendly reminder at the Iteration Review meeting, and convinced that Trellix and McAfee still shares the same database somehow, I searched for "mcafee endpoint security report false positive", and was fortunately pointed to McAfee KB - How to submit false positives to McAfee (TS103032), which says:

​If McAfee WebAdvisor in your browser has classified your website as harmful, see: TS100806 - McAfee WebAdvisor website rating dispute resolution process

McAfee KB - McAfee WebAdvisor website rating dispute resolution process (TS100806) says:

The McAfee WebAdvisor and McAfee SiteAdvisor Site Rating represents the classification of a website's reputation by McAfee.

The site rating is based on several attributes. Based on our automated website scanning, these attributes provide the best indication of the reputation of a site over time.

These attributes include:

  • Suspicious downloads
  • Browser exploits
  • Number of emails sent by the website
  • Affiliations with other websites
  • Pop-ups generated

IMPORTANT: McAfee WebAdvisor and McAfee SiteAdvisor site ratings aren’t real-time scores; the ratings reflect the information we collected at the time of each site visit.

If you think that a website has been classified incorrectly, submit a dispute request asking for a re-evaluation of the website.

For website owners, before you submit a dispute request, make sure that you have addressed anything that could potentially produce a negative classification. For example, you must remove any downloads that might appear suspicious, reduce or eliminate the number of pop-ups generated.

How to submit a dispute request

  1. Go to sitelookup.mcafee.com.

  2. Click Register and complete the form.

  3. Click Register at the bottom of the form, and wait to receive an Account Validation email from McAfee TrustedSource.

  4. Open the email, and click the link to activate your account.

  5. Sign into sitelookup.mcafee.com.

  6. Select McAfee SiteAdvisor/WebControl (Enterprise) from the drop-down list.

  7. Type in the address of a website, for example www.mcafee.com, then click Check URL.

  8. Optionally, select up to three categories for the site.

  9. Type any additional information that you want to share in the Optional comment field.

    NOTE: Provide as much information as possible about why you feel that the rating is in error. If you’re the website owner, let us know us about changes that you made to your site, which might have an impact on the rating.

  10. Click Submit URL for Review. You receive a ticket ID for your records.

Wait for 5–7 days and verify your website again. If the website is still rated as risky, email: sites@mcafee.com to follow up on your request.

What about Trellix?

Meanwhile, on the Trellix side, I came across How to address a website, URL, or IP address that's miscategorized or uncategorized which was last modified on 14/11/2022 and points to https://www.trustedsource.org/ instead, though that website is inaccessible (ERR_CONNECTION_TIMED_OUT) as of today (March 2023).

Searching for "trustedsource.org" on Google (https://www.google.com/search?q=trustedsource.org) actually returns SiteLookup McAfee as the first result.

Trellix Support Community - What is the new sitelookup URL? - Support Community

actually trustedsource.org has been renamed to sitelookup.mcafee.com only recently (25th May 2022), so I don't expect another transition from mcafee to skyhigh domain. Trustedsource database was hosted on a separate domain for years and it should be a reason for it. Currently the trustedsource DB is used by many different products, not only McAfee (home & SMB), Trellix (endpoint & E/XDR) and SkyHigh (network/cloud security), but also some 3-party vendors. I can be wrong, but I expect sitelookup to stay on mcafee domain.

So I guess it is safe to assume that we need to submit the dispute request on one single site (sitelookup.mcafee.com), and ithe issue would be resolved for both McAfee- and Trellix-branded products.

anthonyfok commented 1 year ago

McAfee / Trellix (continued)

Finally submitted a request! Hope everything goes smoothly!

Screenshot from 2023-03-03 opendrr github io McAfee Trellix

anthonyfok commented 1 year ago

McAfee / Trellix (continued)

Track URL Ticket Status

image

Resolved as of 2023-03-07 (confirmed on 2023-03-08)

image

And it looks like https://opendrr.github.io/ is now fully accessible inside the NRCan network! (Probably also thanks to recent internal network changes that resolved another blocker.) :tada: