search

open-telemetry / opentelemetry-collector-contrib

Contrib repository for the OpenTelemetry Collector
https://opentelemetry.io
Apache License 2.0
2.99k stars 2.31k forks source link

[receiver/filelog] Regex is not matching with utf-16le #14615

Closed saikalyan-bhagavathula closed 1 year ago

saikalyan-bhagavathula commented 2 years ago

Describe the bug When I give the utf16-le encoded files to ocb all the log statements from the file are appending to a single line even I gave the multiline option.

Steps to reproduce By just giving the utf16-le encoded files to ocb .

What did you expect to see? Every Log Statement is splitting as the new one.

What did you see instead? All the log statements from the file are appending to a single line even I gave the multiline option.

What version did you use? Version: 0.61.0

What config did you use? Config: 

receivers:
  filelog:
    include:
      - "E:\\mssql\\MSSQL15.SQLEXPRESS\\MSSQL\\Log\\ERRORLOG"
    start_at: beginning
    encoding: "utf-16le"
    multiline:
      line_start_pattern: '^(?P<timestamp>\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}.\d{2})'
    operators:
      - type: regex_parser
        regex: '^(?P<timestamp>\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}.\d{2})\s*(?P<server>\w*)\s*(?P<message>.*)$'
        timestamp:
          parse_from: attributes.timestamp
          layout_type: gotime
          layout: "2006-01-02 15:04:05.00"
processors:
exporters:
  otlp:
    endpoint: "localhost:50052"
    tls:
      insecure: true
      insecure_skip_verify: true
    compression: none
    headers:
      - collector: "otlp-collector"
      - authorization: ""
service:
  pipelines:
    logs:
      receivers: [filelog]
      processors: []
      exporters: [otlp]

Environment OS: Windows 10 Compiler(if manually compiled): (e.g., "go 14.2")

Additional context Add any other context about the problem here.

djaglowski commented 2 years ago

@saikalyan-bhagavathula, there is an encoding setting within the multiline block that can be used to specify the type of encoding. Your sample config does not appear to specify this, so I suspect the receiver is treating your files as the default encoding, utf-8.

Please try:

receivers:
  filelog:
    include:
      - "E:\\mssql\\MSSQL15.SQLEXPRESS\\MSSQL\\Log\\ERRORLOG"
    start_at: beginning
    multiline:
      encoding: utf-16
      line_start_pattern: '^(?P<timestamp>\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}.\d{2})'
saikalyan-bhagavathula commented 2 years ago

@djaglowski Thanks for your reply I already gave the encoding option outside the multiline block.

If I gave it inside the block getting this error.

Error: failed to get config: cannot unmarshal the configuration: error reading receivers configuration for "filelog": 1 error(s) decoding:

My configuration was:

filelog:
  include:
    - "E:\\mssql\\MSSQL15.SQLEXPRESS\\MSSQL\\Log\\ERRORLOG"
  start_at: beginning
  encoding: "utf-16le"
  multiline:
    encoding: "utf-16le"
    line_start_pattern: '^(?P<timestamp>\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}.\d{2})'
  operators:
    - type: regex_parser
      regex: '^(?P<timestamp>\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}.\d{2})\s*(?P<server>\w*)\s*(?P<message>.*)$'
      timestamp:
        parse_from: attributes.timestamp
        layout_type: gotime
        layout: "2006-01-02 15:04:05.00"
djaglowski commented 2 years ago

@saikalyan-bhagavathula, my mistake. I misread how the settings were arranged in code. You had encoding in the correct position to begin with.

I agree this may be a bug. Just to check one last thing though, can you please provide a couple sample log statements?

saikalyan-bhagavathula commented 2 years ago

https://drive.google.com/file/d/1w6DqRPV3dXO7XLvz1HjHkfASwDMoeJIN/view?usp=sharing

This log file is pumped to the open telemetry with above config.

djaglowski commented 2 years ago

I looked into this some more and determined that encoding is not taken into account when creating the "splitFunc" for line_start_pattern or line_end_pattern.

For comparison, encoding is taken into account when using the default "newline splitFunc".

djaglowski commented 2 years ago

@saikalyan-bhagavathula, if possible, please post a few sample logs directly into a comment on this issue.

saikalyan-bhagavathula commented 2 years ago

2022-09-22 15:58:11.50 Server Microsoft SQL Server 2019 (RTM) - 15.0.2000.5 (X64) Sep 24 2019 13:48:23 Copyright (C) 2019 Microsoft Corporation Express Edition (64-bit) on Windows Server 2019 Standard Evaluation 10.0 (Build 17763: ) (Hypervisor)

2022-09-22 15:58:11.50 Server UTC adjustment: 5:30 2022-09-22 15:58:11.50 Server (c) Microsoft Corporation. 2022-09-22 15:58:11.50 Server All rights reserved. 2022-09-22 15:58:11.50 Server Server process ID is 6028. 2022-09-22 15:58:11.50 Server System Manufacturer: 'innotek GmbH', System Model: 'VirtualBox'. 2022-09-22 15:58:11.50 Server Authentication mode is WINDOWS-ONLY. 2022-09-22 15:58:11.50 Server Logging SQL Server messages in file 'E:\mssql\MSSQL15.SQLEXPRESS\MSSQL\Log\ERRORLOG'. 2022-09-22 15:58:11.50 Server The service account is 'NT Service\MSSQL$SQLEXPRESS'. This is an informational message; no user action is required. 2022-09-22 15:58:11.51 Server Registry startup parameters: -d E:\mssql\MSSQL15.SQLEXPRESS\MSSQL\DATA\master.mdf -e E:\mssql\MSSQL15.SQLEXPRESS\MSSQL\Log\ERRORLOG -l E:\mssql\MSSQL15.SQLEXPRESS\MSSQL\DATA\mastlog.ldf 2022-09-22 15:58:11.51 Server Command Line Startup Parameters: -s "SQLEXPRESS" -m "SqlSetup" -T 4022 -T 4010 -T 1905 -T 3701 -T 8015 2022-09-22 15:58:11.51 Server SQL Server detected 1 sockets with 4 cores per socket and 4 logical processors per socket, 4 total logical processors; using 4 logical processors based on SQL Server licensing. This is an informational message; no user action is required. 2022-09-22 15:58:11.51 Server SQL Server is starting at normal priority base (=7). This is an informational message only. No user action is required. 2022-09-22 15:58:11.51 Server Detected 2047 MB of RAM. This is an informational message; no user action is required. 2022-09-22 15:58:11.51 Server Using conventional memory in the memory manager. 2022-09-22 15:58:11.51 Server Page exclusion bitmap is enabled. 2022-09-22 15:58:11.53 Server Buffer Pool: Allocating 262144 bytes for 222073 hashPages. 2022-09-22 15:58:11.55 Server Default collation: SQL_Latin1_General_CP1_CI_AS (us_english 1033) 2022-09-22 15:58:11.63 Server Buffer pool extension is already disabled. No action is necessary. 2022-09-22 15:58:11.71 Server Perfmon counters for resource governor pools and groups failed to initialize and are disabled. 2022-09-22 15:58:11.74 Server Query Store settings initialized with enabled = 1, 2022-09-22 15:58:11.75 Server The maximum number of dedicated administrator connections for this instance is '1' 2022-09-22 15:58:11.75 Server This instance of SQL Server last reported using a process ID of 5632 at 9/22/2022 3:58:10 PM (local) 9/22/2022 10:28:10 AM (UTC). This is an informational message only; no user action is required. 2022-09-22 15:58:11.75 Server Node configuration: node 0: CPU mask: 0x000000000000000f:0 Active CPU mask: 0x000000000000000f:0. This message provides a description of the NUMA configuration for this computer. This is an informational message only. No user action is required. 2022-09-22 15:58:11.76 Server Using dynamic lock allocation. Initial allocation of 2500 Lock blocks and 5000 Lock Owner blocks per node. This is an informational message only. No user action is required. 2022-09-22 15:58:11.76 Server In-Memory OLTP initialized on lowend machine. 2022-09-22 15:58:11.78 Server [INFO] Created Extended Events session 'hkenginexesession'

2022-09-22 15:58:11.78 Server Database Instant File Initialization: enabled. For security and performance considerations see the topic 'Database Instant File Initialization' in SQL Server Books Online. This is an informational message only. No user action is required. 2022-09-22 15:58:11.78 Server Total Log Writer threads: 2. This is an informational message; no user action is required. 2022-09-22 15:58:11.78 Server Database Mirroring Transport is disabled in the endpoint configuration. 2022-09-22 15:58:11.78 Server clflushopt is selected for pmem flush operation. 2022-09-22 15:58:11.78 Server Software Usage Metrics is disabled. 2022-09-22 15:58:11.79 spid10s Warning ** 2022-09-22 15:58:11.79 spid10s SQL Server started in single-user mode. This an informational message only. No user action is required. 2022-09-22 15:58:11.79 spid10s Starting up database 'master'. 2022-09-22 15:58:11.89 Server CLR version v4.0.30319 loaded. 2022-09-22 15:58:12.01 Server Common language runtime (CLR) functionality initialized using CLR version v4.0.30319 from C:\Windows\Microsoft.NET\Framework64\v4.0.30319. 2022-09-22 15:58:12.04 spid10s SQL Server Audit is starting the audits. This is an informational message. No user action is required. 2022-09-22 15:58:12.05 spid10s SQL Server Audit has started the audits. This is an informational message. No user action is required. 2022-09-22 15:58:12.11 spid10s SQL Trace ID 1 was started by login "sa". 2022-09-22 15:58:12.12 spid10s Server name is 'WIN-ILQ466G2NHO\SQLEXPRESS'. This is an informational message only. No user action is required. 2022-09-22 15:58:12.14 spid10s Starting up database 'msdb'. 2022-09-22 15:58:12.15 spid14s Starting up database 'mssqlsystemresource'. 2022-09-22 15:58:12.18 spid14s The resource database build version is 15.00.2000. This is an informational message only. No user action is required. 2022-09-22 15:58:12.30 spid14s Starting up database 'model'. 2022-09-22 15:58:12.41 spid14s Clearing tempdb database. 2022-09-22 15:58:12.49 spid18s A self-generated certificate was successfully loaded for encryption. 2022-09-22 15:58:12.50 spid18s Server local connection provider is ready to accept connection on [ \.\pipe\SQLLocal\SQLEXPRESS ]. 2022-09-22 15:58:12.51 spid18s Dedicated administrator connection support was not started because it is disabled on this edition of SQL Server. If you want to use a dedicated administrator connection, restart SQL Server using the trace flag 7806. This is an informational message only. No user action is required. 2022-09-22 15:58:12.53 spid18s SQL Server is now ready for client connections. This is an informational message; no user action is required. 2022-09-22 15:58:12.54 Server SQL Server is attempting to register a Service Principal Name (SPN) for the SQL Server service. Kerberos authentication will not be possible until a SPN is registered for the SQL Server service. This is an informational message. No user action is required. 2022-09-22 15:58:12.54 Server The SQL Server Network Interface library could not register the Service Principal Name (SPN) [ MSSQLSvc/WIN-ILQ466G2NHO:SQLEXPRESS ] for the SQL Server service. Windows return code: 0xffffffff, state: 63. Failure to register a SPN might cause integrated authentication to use NTLM instead of Kerberos. This is an informational message. Further action is only required if Kerberos authentication is required by authentication policies and if the SPN has not been manually registered. 2022-09-22 15:58:12.65 spid14s Starting up database 'tempdb'. 2022-09-22 15:58:13.11 spid10s Recovery is complete. This is an informational message only. No user action is required. 2022-09-22 15:58:14.92 spid10s SQL Server is terminating in response to a 'stop' request from Service Control Manager. This is an informational message only. No user action is required. 2022-09-22 15:58:14.92 spid10s .NET Framework runtime has been stopped. 2022-09-22 15:58:14.98 spid10s SQL Trace was stopped due to server shutdown. Trace ID = '1'. This is an informational message only; no user action is required.

saikalyan-bhagavathula commented 1 year ago

You need to save these log lines in a file with utf-16le extension to reproduce my case.

saikalyan-bhagavathula commented 1 year ago

These are the sample log lines:

2022-09-22 15:58:11.50 Server Microsoft SQL Server 2019 (RTM) - 15.0.2000.5 (X64) Sep 24 2019 13:48:23 Copyright (C) 2019 Microsoft Corporation Express Edition (64-bit) on Windows Server 2019 Standard Evaluation 10.0 (Build 17763: ) (Hypervisor)

2022-09-22 15:58:11.50 Server UTC adjustment: 5:30 2022-09-22 15:58:11.50 Server (c) Microsoft Corporation. 2022-09-22 15:58:11.50 Server All rights reserved. 2022-09-22 15:58:11.50 Server Server process ID is 6028. 2022-09-22 15:58:11.50 Server System Manufacturer: 'innotek GmbH', System Model: 'VirtualBox'. 2022-09-22 15:58:11.50 Server Authentication mode is WINDOWS-ONLY. 2022-09-22 15:58:11.50 Server Logging SQL Server messages in file 'E:\mssql\MSSQL15.SQLEXPRESS\MSSQL\Log\ERRORLOG'. 2022-09-22 15:58:11.50 Server The service account is 'NT Service\MSSQL$SQLEXPRESS'. This is an informational message; no user action is required. 2022-09-22 15:58:11.51 Server Registry startup parameters: -d E:\mssql\MSSQL15.SQLEXPRESS\MSSQL\DATA\master.mdf -e E:\mssql\MSSQL15.SQLEXPRESS\MSSQL\Log\ERRORLOG -l E:\mssql\MSSQL15.SQLEXPRESS\MSSQL\DATA\mastlog.ldf 2022-09-22 15:58:11.51 Server Command Line Startup Parameters: -s "SQLEXPRESS" -m "SqlSetup" -T 4022 -T 4010 -T 1905 -T 3701 -T 8015 2022-09-22 15:58:11.51 Server SQL Server detected 1 sockets with 4 cores per socket and 4 logical processors per socket, 4 total logical processors; using 4 logical processors based on SQL Server licensing. This is an informational message; no user action is required. 2022-09-22 15:58:11.51 Server SQL Server is starting at normal priority base (=7). This is an informational message only. No user action is required. 2022-09-22 15:58:11.51 Server Detected 2047 MB of RAM. This is an informational message; no user action is required. 2022-09-22 15:58:11.51 Server Using conventional memory in the memory manager. 2022-09-22 15:58:11.51 Server Page exclusion bitmap is enabled.

If I filled the multiline in config then all log records are appending as single log record.

The config looks as below:

receivers:
  filelog:
    include:
      - "E:\\mssql\\MSSQL15.SQLEXPRESS\\MSSQL\\Log\\ERRORLOG"
    start_at: beginning
    encoding: utf-16le
    multiline:
      line_start_pattern: '^(?P<timestamp>\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}.\d{2})'
    operators:
      - type: regex_parser
        regex: '^(?P<timestamp>\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}.\d{2})\s*(?P<server>\w*)\s*(?P<message>.*)$'
        timestamp:
          parse_from: attributes.timestamp
          layout_type: gotime
          layout: "2006-01-02 15:04:05.00"

processors:

exporters:
  otlp:
    endpoint: "localhost:50052"
    tls:
      insecure: true
      insecure_skip_verify: true
    compression: none
    headers:
      - collector: "otlp-collector"
      - authorization: "32a925cb-8fb9-438e-817a-c25ff15d51f4"

service:
  pipelines:
    logs:
      receivers: [filelog]
      processors: []
      exporters: [otlp]

The output was:

Received: [{"resource":{},"instrumentation_library_logs":[{"instrumentation_library":{},"logs":[{"body":{"Value":{"StringValue":"2022-09-22 15:58:11.50 Server Microsoft SQL Server 2019 (RTM) - 15.0.2000.5 (X64) \r\n\tSep 24 2019 13:48:23 \r\n\tCopyright (C) 2019 Microsoft Corporation\r\n\tExpress Edition (64-bit) on Windows Server 2019 Standard Evaluation 10.0 \u003cX64\u003e (Build 17763: ) (Hypervisor)\r\n\r\n2022-09-22 15:58:11.50 Server UTC adjustment: 5:30\r\n2022-09-22 15:58:11.50 Server (c) Microsoft Corporation.\r\n2022-09-22 15:58:11.50 Server All rights reserved.\r\n2022-09-22 15:58:11.50 Server Server process ID is 6028.\r\n2022-09-22 15:58:11.50 Server System Manufacturer: 'innotek GmbH', System Model: 'VirtualBox'.\r\n2022-09-22 15:58:11.50 Server Authentication mode is WINDOWS-ONLY.\r\n2022-09-22 15:58:11.50 Server Logging SQL Server messages in file 'E:\mssql\MSSQL15.SQLEXPRESS\MSSQL\Log\ERRORLOG'.\r\n2022-09-22 15:58:11.50 Server The service account is 'NT Service\MSSQL$SQLEXPRESS'. This is an informational message; no user action is required.\r\n2022-09-22 15:58:11.51 Server Registry startup parameters: \r\n\t -d E:\mssql\MSSQL15.SQLEXPRESS\MSSQL\DATA\master.mdf\r\n\t -e E:\mssql\MSSQL15.SQLEXPRESS\MSSQL\Log\ERRORLOG\r\n\t -l E:\mssql\MSSQL15.SQLEXPRESS\MSSQL\DATA\mastlog.ldf\r\n2022-09-22 15:58:11.51 Server Command Line Startup Parameters:\r\n\t -s \"SQLEXPRESS\"\r\n\t -m \"SqlSetup\"\r\n\t -T 4022\r\n\t -T 4010\r\n\t -T 1905\r\n\t -T 3701\r\n\t -T 8015\r\n2022-09-22 15:58:11.51 Server SQL Server detected 1 sockets with 4 cores per socket and 4 logical processors per socket, 4 total logical processors; using 4 logical processors based on SQL Server licensing. This is an informational message; no user action is required.\r\n2022-09-22 15:58:11.51 Server SQL Server is starting at normal priority base (=7). This is an informational message only. No user action is required.\r\n2022-09-22 15:58:11.51 Server Detected 2047 MB of RAM. This is an informational message; no user action is required.\r\n2022-09-22 15:58:11.51 Server Using conventional memory in the memory manager.\r\n2022-09-22 15:58:11.51 Server Page exclusion bitmap is enabled.\r\n"}},"attributes":[{"key":"log.file.name","value":{"Value":{"StringValue":"ERRORLOG"}}}]}]}]}]

If I skip the filling multiline in config then multilined log records are not appending to actual log record They are comming as new log record.

The config looks as below:

receivers:
  filelog:
    include:
      - "E:\\mssql\\MSSQL15.SQLEXPRESS\\MSSQL\\Log\\ERRORLOG"
    start_at: beginning
    encoding: utf-16le
    operators:
      - type: regex_parser
        regex: '^(?P<timestamp>\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}.\d{2})\s*(?P<server>\w*)\s*(?P<message>.*)$'
        timestamp:
          parse_from: attributes.timestamp
          layout_type: gotime
          layout: "2006-01-02 15:04:05.00"

processors:

exporters:
  otlp:
    endpoint: "localhost:50052"
    tls:
      insecure: true
      insecure_skip_verify: true
    compression: none
    headers:
      - collector: "otlp-collector"
      - authorization: "32a925cb-8fb9-438e-817a-c25ff15d51f4"

service:
  pipelines:
    logs:
      receivers: [filelog]
      processors: []
      exporters: [otlp]

The output was:

Received: [{"resource":{},"instrumentation_library_logs":[{"instrumentation_library":{},"logs":[{"body":{"Value":{"StringValue":"2022-09-22 15:58:11.50 Server Microsoft SQL Server 2019 (RTM) - 15.0.2000.5 (X64) "}},"attributes":[{"key":"log.file.name","value":{"Value":{"StringValue":"ERRORLOG"}}}]},{"body":{"Value":{"StringValue":"\tSep 24 2019 13:48:23 "}},"attributes":[{"key":"log.file.name","value":{"Value":{"StringValue":"ERRORLOG"}}}]},{"body":{"Value":{"StringValue":"\tCopyright (C) 2019 Microsoft Corporation"}},"attributes":[{"key":"log.file.name","value":{"Value":{"StringValue":"ERRORLOG"}}}]},{"body":{"Value":{"StringValue":"\tExpress Edition (64-bit) on Windows Server 2019 Standard Evaluation 10.0 \u003cX64\u003e (Build 17763: ) (Hypervisor)"}},"attributes":[{"key":"log.file.name","value":{"Value":{"StringValue":"ERRORLOG"}}}]},{"time_unix_nano":1663842491500000000,"body":{"Value":{"StringValue":"2022-09-22 15:58:11.50 Server UTC adjustment: 5:30"}},"attributes":[{"key":"timestamp","value":{"Value":{"StringValue":"2022-09-22 15:58:11.50"}}},{"key":"server","value":{"Value":{"StringValue":"Server"}}},{"key":"message","value":{"Value":{"StringValue":"UTC adjustment: 5:30"}}},{"key":"log.file.name","value":{"Value":{"StringValue":"ERRORLOG"}}}]},{"time_unix_nano":1663842491500000000,"body":{"Value":{"StringValue":"2022-09-22 15:58:11.50 Server (c) Microsoft Corporation."}},"attributes":[{"key":"log.file.name","value":{"Value":{"StringValue":"ERRORLOG"}}},{"key":"timestamp","value":{"Value":{"StringValue":"2022-09-22 15:58:11.50"}}},{"key":"server","value":{"Value":{"StringValue":"Server"}}},{"key":"message","value":{"Value":{"StringValue":"(c) Microsoft Corporation."}}}]},{"time_unix_nano":1663842491500000000,"body":{"Value":{"StringValue":"2022-09-22 15:58:11.50 Server All rights reserved."}},"attributes":[{"key":"log.file.name","value":{"Value":{"StringValue":"ERRORLOG"}}},{"key":"server","value":{"Value":{"StringValue":"Server"}}},{"key":"message","value":{"Value":{"StringValue":"All rights reserved."}}},{"key":"timestamp","value":{"Value":{"StringValue":"2022-09-22 15:58:11.50"}}}]},{"time_unix_nano":1663842491500000000,"body":{"Value":{"StringValue":"2022-09-22 15:58:11.50 Server Server process ID is 6028."}},"attributes":[{"key":"message","value":{"Value":{"StringValue":"Server process ID is 6028."}}},{"key":"log.file.name","value":{"Value":{"StringValue":"ERRORLOG"}}},{"key":"timestamp","value":{"Value":{"StringValue":"2022-09-22 15:58:11.50"}}},{"key":"server","value":{"Value":{"StringValue":"Server"}}}]},{"time_unix_nano":1663842491500000000,"body":{"Value":{"StringValue":"2022-09-22 15:58:11.50 Server System Manufacturer: 'innotek GmbH', System Model: 'VirtualBox'."}},"attributes":[{"key":"log.file.name","value":{"Value":{"StringValue":"ERRORLOG"}}},{"key":"timestamp","value":{"Value":{"StringValue":"2022-09-22 15:58:11.50"}}},{"key":"server","value":{"Value":{"StringValue":"Server"}}},{"key":"message","value":{"Value":{"StringValue":"System Manufacturer: 'innotek GmbH', System Model: 'VirtualBox'."}}}]},{"time_unix_nano":1663842491500000000,"body":{"Value":{"StringValue":"2022-09-22 15:58:11.50 Server Authentication mode is WINDOWS-ONLY."}},"attributes":[{"key":"log.file.name","value":{"Value":{"StringValue":"ERRORLOG"}}},{"key":"timestamp","value":{"Value":{"StringValue":"2022-09-22 15:58:11.50"}}},{"key":"server","value":{"Value":{"StringValue":"Server"}}},{"key":"message","value":{"Value":{"StringValue":"Authentication mode is WINDOWS-ONLY."}}}]},{"time_unix_nano":1663842491500000000,"body":{"Value":{"StringValue":"2022-09-22 15:58:11.50 Server Logging SQL Server messages in file 'E:\mssql\MSSQL15.SQLEXPRESS\MSSQL\Log\ERRORLOG'."}},"attributes":[{"key":"log.file.name","value":{"Value":{"StringValue":"ERRORLOG"}}},{"key":"timestamp","value":{"Value":{"StringValue":"2022-09-22 15:58:11.50"}}},{"key":"server","value":{"Value":{"StringValue":"Server"}}},{"key":"message","value":{"Value":{"StringValue":"Logging SQL Server messages in file 'E:\mssql\MSSQL15.SQLEXPRESS\MSSQL\Log\ERRORLOG'."}}}]},{"time_unix_nano":1663842491500000000,"body":{"Value":{"StringValue":"2022-09-22 15:58:11.50 Server The service account is 'NT Service\MSSQL$SQLEXPRESS'. This is an informational message; no user action is required."}},"attributes":[{"key":"log.file.name","value":{"Value":{"StringValue":"ERRORLOG"}}},{"key":"timestamp","value":{"Value":{"StringValue":"2022-09-22 15:58:11.50"}}},{"key":"server","value":{"Value":{"StringValue":"Server"}}},{"key":"message","value":{"Value":{"StringValue":"The service account is 'NT Service\MSSQL$SQLEXPRESS'. This is an informational message; no user action is required."}}}]},{"time_unix_nano":1663842491510000000,"body":{"Value":{"StringValue":"2022-09-22 15:58:11.51 Server Registry startup parameters: "}},"attributes":[{"key":"message","value":{"Value":{"StringValue":"Registry startup parameters: "}}},{"key":"log.file.name","value":{"Value":{"StringValue":"ERRORLOG"}}},{"key":"timestamp","value":{"Value":{"StringValue":"2022-09-22 15:58:11.51"}}},{"key":"server","value":{"Value":{"StringValue":"Server"}}}]},{"body":{"Value":{"StringValue":"\t -d E:\mssql\MSSQL15.SQLEXPRESS\MSSQL\DATA\master.mdf"}},"attributes":[{"key":"log.file.name","value":{"Value":{"StringValue":"ERRORLOG"}}}]},{"body":{"Value":{"StringValue":"\t -e E:\mssql\MSSQL15.SQLEXPRESS\MSSQL\Log\ERRORLOG"}},"attributes":[{"key":"log.file.name","value":{"Value":{"StringValue":"ERRORLOG"}}}]},{"body":{"Value":{"StringValue":"\t -l E:\mssql\MSSQL15.SQLEXPRESS\MSSQL\DATA\mastlog.ldf"}},"attributes":[{"key":"log.file.name","value":{"Value":{"StringValue":"ERRORLOG"}}}]},{"time_unix_nano":1663842491510000000,"body":{"Value":{"StringValue":"2022-09-22 15:58:11.51 Server Command Line Startup Parameters:"}},"attributes":[{"key":"log.file.name","value":{"Value":{"StringValue":"ERRORLOG"}}},{"key":"timestamp","value":{"Value":{"StringValue":"2022-09-22 15:58:11.51"}}},{"key":"server","value":{"Value":{"StringValue":"Server"}}},{"key":"message","value":{"Value":{"StringValue":"Command Line Startup Parameters:"}}}]},{"body":{"Value":{"StringValue":"\t -s \"SQLEXPRESS\""}},"attributes":[{"key":"log.file.name","value":{"Value":{"StringValue":"ERRORLOG"}}}]},{"body":{"Value":{"StringValue":"\t -m \"SqlSetup\""}},"attributes":[{"key":"log.file.name","value":{"Value":{"StringValue":"ERRORLOG"}}}]},{"body":{"Value":{"StringValue":"\t -T 4022"}},"attributes":[{"key":"log.file.name","value":{"Value":{"StringValue":"ERRORLOG"}}}]},{"body":{"Value":{"StringValue":"\t -T 4010"}},"attributes":[{"key":"log.file.name","value":{"Value":{"StringValue":"ERRORLOG"}}}]},{"body":{"Value":{"StringValue":"\t -T 1905"}},"attributes":[{"key":"log.file.name","value":{"Value":{"StringValue":"ERRORLOG"}}}]},{"body":{"Value":{"StringValue":"\t -T 3701"}},"attributes":[{"key":"log.file.name","value":{"Value":{"StringValue":"ERRORLOG"}}}]},{"body":{"Value":{"StringValue":"\t -T 8015"}},"attributes":[{"key":"log.file.name","value":{"Value":{"StringValue":"ERRORLOG"}}}]},{"time_unix_nano":1663842491510000000,"body":{"Value":{"StringValue":"2022-09-22 15:58:11.51 Server SQL Server detected 1 sockets with 4 cores per socket and 4 logical processors per socket, 4 total logical processors; using 4 logical processors based on SQL Server licensing. This is an informational message; no user action is required."}},"attributes":[{"key":"log.file.name","value":{"Value":{"StringValue":"ERRORLOG"}}},{"key":"timestamp","value":{"Value":{"StringValue":"2022-09-22 15:58:11.51"}}},{"key":"server","value":{"Value":{"StringValue":"Server"}}},{"key":"message","value":{"Value":{"StringValue":"SQL Server detected 1 sockets with 4 cores per socket and 4 logical processors per socket, 4 total logical processors; using 4 logical processors based on SQL Server licensing. This is an informational message; no user action is required."}}}]},{"time_unix_nano":1663842491510000000,"body":{"Value":{"StringValue":"2022-09-22 15:58:11.51 Server SQL Server is starting at normal priority base (=7). This is an informational message only. No user action is required."}},"attributes":[{"key":"server","value":{"Value":{"StringValue":"Server"}}},{"key":"message","value":{"Value":{"StringValue":"SQL Server is starting at normal priority base (=7). This is an informational message only. No user action is required."}}},{"key":"log.file.name","value":{"Value":{"StringValue":"ERRORLOG"}}},{"key":"timestamp","value":{"Value":{"StringValue":"2022-09-22 15:58:11.51"}}}]},{"time_unix_nano":1663842491510000000,"body":{"Value":{"StringValue":"2022-09-22 15:58:11.51 Server Detected 2047 MB of RAM. This is an informational message; no user action is required."}},"attributes":[{"key":"log.file.name","value":{"Value":{"StringValue":"ERRORLOG"}}},{"key":"timestamp","value":{"Value":{"StringValue":"2022-09-22 15:58:11.51"}}},{"key":"server","value":{"Value":{"StringValue":"Server"}}},{"key":"message","value":{"Value":{"StringValue":"Detected 2047 MB of RAM. This is an informational message; no user action is required."}}}]},{"time_unix_nano":1663842491510000000,"body":{"Value":{"StringValue":"2022-09-22 15:58:11.51 Server Using conventional memory in the memory manager."}},"attributes":[{"key":"timestamp","value":{"Value":{"StringValue":"2022-09-22 15:58:11.51"}}},{"key":"server","value":{"Value":{"StringValue":"Server"}}},{"key":"message","value":{"Value":{"StringValue":"Using conventional memory in the memory manager."}}},{"key":"log.file.name","value":{"Value":{"StringValue":"ERRORLOG"}}}]},{"time_unix_nano":1663842491510000000,"body":{"Value":{"StringValue":"2022-09-22 15:58:11.51 Server Page exclusion bitmap is enabled."}}},{"key":"log.file.name","value":{"Value":{"StringValue":"ERRORLOG"}}}]}]}]}]

github-actions[bot] commented 1 year ago

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

jpkrohling commented 1 year ago

When I give the utf16-le encoded files to ocb

It there anything I need to address in ocb for this? I couldn't find anything relevant to it, but perhaps I'm missing something?

saikalyan-bhagavathula commented 1 year ago

any update on this we are still facing the challenge with latest ocb

https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/14615#issuecomment-1278843935

djaglowski commented 1 year ago

@jpkrohling, I don't believe this is related to ocb. It appears to be a concern of the filelog receiver.

djaglowski commented 1 year ago

@saikalyan-bhagavathula, no progress has been made that I am aware of. Please let us know if you want to take on this enhancement. I'd be happy to review a PR.

github-actions[bot] commented 1 year ago

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

github-actions[bot] commented 1 year ago

This issue has been closed as inactive because it has been stale for 120 days with no activity.