404 errors are all grouped under the same route making it hard to look through

[xt0rted]

I've just moved my site over to asp.net core 2.0 using the UseStatusCodePagesWithReExecute middleware. Prior to this migration I was only using the Site Extension (the site was owin & nancy so I couldn't really use the sdk in the application) and when a 404 error came through the portal would show the actual url path. Now I'm seeing all 404 errors grouped under the same basic name of GET Home/Error [errorType].

Is there a way to report/show the requested path instead of the error page that gets shown? Right now this is what I'm seeing for the last 24 hours. There's about 8 different paths that produced these 18 entries but you don't know that without going through each entry.

Another thing I'm noticing is there's no info such as referrer or browser info. Should that information be included in the error details? It seems really useful in helping to track down where the issue may have come from. A good example of this is older versions of iOS Safari don't seem to handle spaces in file downloads and as a result a 404 is returned with a partial path such as /downloads/monthly which could be for any file that starts with monthly and span various pages on the site.

Version Info

SDK Version: 2.2.0 Beta 3 .NET Version: .net core 2.0 How Application was onboarded with SDK(VisualStudio/StatusMonitor/Azure Extension) : NuGet & Site Extension OS: Server 2016 Hosting Info (IIS/Azure WebApps/ etc): Azure WebApps

[cijothomas]

https://github.com/Microsoft/ApplicationInsights-aspnetcore/blob/develop/src/Microsoft.ApplicationInsights.AspNetCore/DiagnosticListeners/Implementation/HostingDiagnosticListener.cs#L276 Telemetry.Url is set to the full url. Do you observe a different behaviour?

I think the UserInterface is displaying Telemetry.Name which wont have full url, and look same.

Let us know if you can use Url to distinguish between 404 calls. Or correct me if i mis-read the issue.

[KLuuKer]

I have the same "problem" it's more of a UI thing.... The thing is all the 404's that are not hitting a route end up on the error handler route so they all end up under 1 entry in the UI, and because they are not split up in different entries you cannot make easy decisions about what url to fix first (or just skip because it's irrelevant)

maybe make this configurable somehow?

[SilentWolf8979]

I'm in the same boat as @KLuuKer. UseStatusCodePagesWithReExecute is how we're "supposed" to show friendly error pages, but it's also the only version of UseStatusCodePages* that causes all of the failures to be obfuscated in the UI. I'm not sure if this is an issue in the AI code, or if it's more of an Azure UI issue.

@cijothomas - Is there a repo for this functionality within Azure, or some better way to bring this issue up with the UI folks?

[Anduin2017]

Same issue here.

[EdiWang]

Same issue here. I use

app.UseStatusCodePagesWithReExecute("/error", "?statusCode={0}");

My error handler code is: https://github.com/EdiWang/Moonglade/blob/master/src/Moonglade.Web/Controllers/ErrorController.cs#L22

I have some 404 errors showing the desired URL like this:

But I still have some other 404 errors showing Error/Error.

[jerriep]

I get a lot of bots "passing by" and poking at my website to try and find security holes. They typically try and access some WordPress related URLs trying to poke for vulnerabilities.

This issue makes it very frustrating for me to differentiate between requests from these bots and "genuine" 404 errors (i.e. my application requests a wrong url) as I have to inspect each one individually.

It would help a lot to get this one resolved

[EdiWang]

@jerriep I have the same issue. There's a workaround, for example, my website is written in ASP.NET, there would not be any PHP files as WordPress, so I can set URL Rewrite/Redirect for all ".php" requests. Instead of making them 404, I can respond or redirect the request somewhere else so that Application Insights won't capture those bot scans as error results.

[jerriep]

Thanks for the tip @EdiWang, I'll look into it. Though my first feeling is that I'd rather keep them as 404 and know about them

[EdiWang]

Any update on this? It's April 2020 now, I am still being frustrated by this issue every single day.

[cijothomas]

@EdiWang Can you confirm if your url is populated correctly for 404 occurences? If yes, this issue is not required to be fixed at SDK level, but at UI - thoughts?

[EdiWang]

@cijothomas the problem is after using app.UseStatusCodePagesWithReExecute("/error", "?statusCode={0}"); the application insights data are grouped under "Error/Error" or "Home/Error" in @xt0rted 's case. Although we can click into each single one of them to see the actual request URL, it makes us extremely hard to check the 404 URLs on the application insights Failures page.

I don't know if this issue is in ASP.NET Core runtime or in Application Insight.

The reason many people use app.UseStatusCodePagesWithReExecute("/error", "?statusCode={0}"); is to display a friendly custom error page but preserve the URL. Without it, the app will just return error code response without body.

[github-actions[bot]]

This issue is stale because it has been open 300 days with no activity. Remove stale label or comment or this will be closed in 7 days.