search

MicrosoftDocs / architecture-center

Open Source documentation for the Azure Architecture Center on Microsoft Docs
http://aka.ms/architecture
Creative Commons Attribution 4.0 International
1.84k stars 1.68k forks source link

Benefit to this approach vs Azure Monitor #3713

Closed ejhenry closed 1 year ago

ejhenry commented 1 year ago

Is there a particular scenario this guidance seeks to address? If so, that would be good to call out.

It's not clear what the benefit to this approach is compared to using existing Azure Monitor capabilities. For example, with the introduction of workspace-based App Insights resources, the assertion that 'consolidating multiple applications into a single workspace is a challenge' seems a bit out of date.

Document Details

⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

Naveenommi-MSFT commented 1 year ago

@ejhenry Thanks for your feedback! We will investigate and update as appropriate.

yemrea commented 1 year ago

Thank you, Eric (@ejhenry) for raising this point. It looks like we may need to update the content as it is confusing people. The content is given in the article is completely refresh. These are solutions to two different problems.

In this article, we are proposing a solution for custom logging. If you use custom logging in application insights, logs are placed into different tables. See https://learn.microsoft.com/en-us/azure/azure-monitor/app/api-custom-events-metrics#custom-events-in-log-analytics. Like pointed out in this article, custom logs will be placed under a new table called customEvents whereas others will land into a different table (Events).

Second point here to emphasize here is: the proposed architecture is designated for enterprises, not for small/mid-size customers. If you're an enterprise with multiple teams and multiple applications, everyone likes to use certain fields in their tables. When each team adds a column for themselves, many columns will start appearing on custom tables (customEvents if we use Events as an example). In this article, we recommend them to a have a common data format, so tables do not have repeated columns and unnecessarily many columns. For a mid-size organization who uses a few different technologies, using workspace-based Application Insights could be easier, as they don't need to maintain and manage a Kusto cluster. With Log Analytics, we use Kusto cluster at the same way and this cluster is managed by Microsoft as a service.

Third point is if you are using different technologies in your enterprise (e.g., python, Java, .NET), introducing and maintaining a common data format for logs is not easy. You can create a nuget package and publish it internally. With this package, you can also enable application insights (without custom events) and collect all required data and send them to a shared /common log analytics workspace. I don't see them conflicting; I see them as complementary. This is actually what I meant with "Logs from different systems cause similar problems for big data solutions". If you start collecting data from different systems, you will end up having different data formats and verbiage. In a customer, I witnessed 100+ definition for a customer across different systems. Even different applications maintained by the same team, they had different definitions.

With all said, I will update Alternatives section to include these points to make it clearer. Also in the introduction part, I will also explicitly call out this is not a replacement for Application Insights or Azure Monitor. This is a complementary service. Azure Monitor has the exactly the same Architecture and utilises Azure Data Explorer behind the scenes. You can perform cross cluster queries if you need to correlate the data collected from Azure services with your application logs.