Logs Leaking

[MentalGear]

Describe the bug

When logs are set to "1 day", I can see logs leaking from other users. Example (I have not installed wordpress anywhere)

Steps to reproduce

Netlify dashboard, set to 1 day.

Configuration

No response

Deploy logs

Netlify dashboard, set to 1 day.

[fool]

Thanks for the report! Our team is discussing now (I work on the tech support team at Netlify).

Could you confirm something for us? Are you sure this isn't someone trying to access files like this on your own site? I do understand you don't use wordpress, but for at least the past 5 years, we have always gotten tens of millions of scans of our sites for wordpress vulnerabilities each month. These attackers are extremely not-savvy (quantity over quality for scans like these); of course, very few folks have wordpress hosted here since we can only effectively proxy to it.

But, on to how you could be affected:

• do you have a redirect for /* that routes to a function
• that could be passed parameters somehow , (e.g. via a cookie, request header, or via the URL path itself)
• that would cause your code to try to lookup such files?

Of course they should error with a not found if requested, but I could imagines logs like this, coming from a situation like that, rather than leaking from another customer.

To help us investigate, if you could share your site ID with us, we can look in our access logs to confirm/deny this is what's happening (we'd see requests including a path like that, to your site, and also a redirect and function configured that could lead to this happening) That site ID is not sensitive and is ok to share in a github issue like this one. You can find it on the site settings page, and it will look like 6e65bc7e-1bfb-48fe-b4fb-c1192a22b262. If you can't find that, perhaps you can find me the x-nf-request-id from a request you make in your browser, for that site, which is also safe to share in public?

Here's how: https://answers.netlify.com/t/support-guide-netlify-support-asked-for-the-x-nf-request-id-header-what-is-it-and-how-do-i-find-it/4385

Thanks in advance for your help!

[MentalGear]

Hi there,

Netlify's forum seems to think the logs are a result of bot attempts looking for those files to probe for security issues. Do you concur? Do requests like these also count towards the plan's quota or are they filtered out?

https://answers.netlify.com/t/security-log-leaking-cross-users/91086/6

On 2 May 2023, at 17:20, chris (fool) mccraw @.***> wrote:

Thanks for the report! Our team is discussing now (I work on the tech support team at Netlify).

Could you confirm something for us? Are you sure this isn't someone trying to access files like this on your own site? I do understand you don't use wordpress, but for at least the past 5 years, we have always gotten tens of millions of scans of our sites for wordpress vulnerabilities each month. Of course, very few folks have wordpress hosted here since we can only effectively proxy to it. But, do you have a redirect for /* that routes to a function, that could be passed parameters, or via the URL path, that would cause your code to try to lookup such files? Of course they should error with a not found if requested, but I could imagines logs like this, coming from a situation like that, rather than leaking from another customer.

To help us investigate, if you could share your site ID with us, we can look in our access logs to confirm/deny this is what's happening (we'd see requests including a path like that, to your site, and also a redirect and function configured that could lead to this happening) That site ID is not sensitive and is ok to share in a github issue like this one. You can find it on the site settings page, and it will look like 6e65bc7e-1bfb-48fe-b4fb-c1192a22b262. If you can't find that, perhaps you can find me the x-nf-request-id from a request you make in your browser, for that site, which is also safe to share in public?

Here's how: https://answers.netlify.com/t/support-guide-netlify-support-asked-for-the-x-nf-request-id-header-what-is-it-and-how-do-i-find-it/4385 https://answers.netlify.com/t/support-guide-netlify-support-asked-for-the-x-nf-request-id-header-what-is-it-and-how-do-i-find-it/4385 Thanks in advance for your help!

— Reply to this email directly, view it on GitHub https://github.com/netlify/build/issues/4991#issuecomment-1531666097, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAVUVG3ELUNITZD3MAPGNTLXEEQ5BANCNFSM6AAAAAAXN4O4AI. You are receiving this because you authored the thread.

[fool]

I can tell you that we have seen those bots for my entire tenure at Netlify (almost 7 years now). Can I confirm this is exactly what happened to your site? Not unless/until you answer my questions. But I can extrapolate:

1. Our founder @biilmann who was also the chief architect for a lot of years, spoke at a conference in mid-2017 and reported the tens of millions number back then. Looking today, I can see that in the past hour alone , such bots have requested paths including wlwmanifest.xml on over 16k sites, coming from around 3700 IP's. Pretty clearly scanners and scammers.

2. Yes, such requests count towards your quotas for all the relevant things:

   • bandwidth
   • functions invocations

If you'd like to prevent this, you could write an edge function to block such requests, using less bandwidth and not invoking a function. Here's some sample code that blocks based on country; you could adapt this to block based on path or substring in path pretty easily, I suspect: https://edge-functions-examples.netlify.app/example/country-block

[MentalGear]

Thank you for the detailed answer, @fool!

Since, as you indicated, these probes will only continue to accumulate over time and are already in the millions, I believe it would be in line with Netlify's motto to provide a simple toggle that could block all this unwanted traffic and function pollution.

This could even be the default setting, as you correctly noted that there is no real case for WP over Netlify (unless actual WP files are auto-detected). By default, blocking wlmanifest.xml or similar probes would be quite ideal.

Also huge bon to customer satisfaction and saving your own resources, e.g. on the free plan.

[fool]

Thanks for the suggestion! We likely won't build that, despite it making sense for you :) Why? It would break the customers who do proxy from wordpress, which are present in substantial numbers on our platform. We can't somehow magically "tell" what should be proxy'd to and not based on it being available or not - that wouldn't be what our customers who need this want, nor even what those who don't, expect.

I suppose the way to think about it is that: The internet will always scan your website once you publish it, so you'll want to build any protections you want that are specific to your use case, for yourself. We'll keep our platform open to uses of all sorts.

[MentalGear]

This why I called it an optional toggle for users who are sure they don't proxy WP. :P

On 2 May 2023, at 23:39, chris (fool) mccraw @.***> wrote:

Closed #4991 https://github.com/netlify/build/issues/4991 as completed.

— Reply to this email directly, view it on GitHub https://github.com/netlify/build/issues/4991#event-9150076775, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAVUVG7GQBHVKOAVLL4GJ23XEF5KNANCNFSM6AAAAAAXN4O4AI. You are receiving this because you authored the thread.

[fool]

you'll note that we don't have any other features that look like that today :) But, it is still a good suggestion, thank you!