I do not have these fields vmw_vc_*

[kossatov1989]

Good afternoon, I have done everything according to the instructions. I am getting logs from Vcentr, but the dashboards are not working. I get this error - Unknown field: Request contains unknown field: vmw_vc_Event. I do not have these fields vmwvc*

[kossatov1989]

[s0p4L1n3]

Ok so I just test on a new Graylog server, no error on my side.

Can you check if you have this on your Graylog pipeline side > manage rule side

If yes then can you filter in your stream if you have Event data, If yes, deploy one of the messages, left click on the field message > Create extractor > Grok Pattern (just to test if you can extract data)

On the extractor copy paste this grok pattern (grok pattern that extract the fields that is missing on your side) note: on pipeline rule, as it is java language, you need to add double \\ if your grok pattern in extractors has simple \

%{DATA:vmw_vc_Event} \[%{BASE10NUM:vmw_vc_eventId}\] \[%{DATA:vmw_vc_partInfo}\] \[%{TIMESTAMP_ISO8601:vmw_vc_createdTime}\] \[vim.event.%{DATA:vmw_vc_eventType}\] \[%{DATA:vmw_vc_severity}\] \[%{DATA:vmw_vc_user}\] \[%{DATA:vmw_vc_target}\] \[%{DATA:vmw_vc_chainId}\] \[%{DATA:vmw_vc_desc}\]

Click try against example and it should show the fields extracted

If it does not work, it can means multiple things but let's confirm just this now and tell me the result.

[kossatov1989]

Hello 1 2 3 4

I don't have a Grok pattern (DATA), can you provide one?

[kossatov1989]

[kossatov1989]

[kossatov1989]

[s0p4L1n3]

How come you do not have DATA grok pattern as it is built in grok pattern provided by Graylog ? I only used the provided one.

The issue is that your field message start with message: Event while on my VCSA, it start with Event without the prefix message: It's weird

For your pipeline named: Syslog - VMWARE st12vcenter01, edit it and:

• try replace %{DATA:vmw_vc_Event} \\[%{BASE10NUM:vmw_vc_eventId}\\] \\[%{DATA:vmw_vc_partInfo}\\] \\[%{TIMESTAMP_ISO8601:vmw_vc_createdTime}\\] \\[vim.event.%{DATA:vmw_vc_eventType}\\] \\[%{DATA:vmw_vc_severity}\\] \\[%{DATA:vmw_vc_user}\\] \\[%{DATA:vmw_vc_target}\\] \\[%{DATA:vmw_vc_chainId}\\] \\[%{DATA:vmw_vc_desc}\\]

• with this message: %{DATA:vmw_vc_Event} \\[%{BASE10NUM:vmw_vc_eventId}\\] \\[%{DATA:vmw_vc_partInfo}\\] \\[%{TIMESTAMP_ISO8601:vmw_vc_createdTime}\\] \\[vim.event.%{DATA:vmw_vc_eventType}\\] \\[%{DATA:vmw_vc_severity}\\] \\[%{DATA:vmw_vc_user}\\] \\[%{DATA:vmw_vc_target}\\] \\[%{DATA:vmw_vc_chainId}\\] \\[%{DATA:vmw_vc_desc}\\]

The VMWARE docs describe the normal log format... try debugging why it append message: before your message`(maybe syslog particularity ?) https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-monitoring-performance/GUID-F92A08A1-646A-4DCD-8DB0-47E6D2B49C5D.html

[kossatov1989]

I apologize, I didn't put the message in correctly.

[kossatov1989]

If I use \\ I get an error. if \

[kossatov1989]

Now the problem is that Syslog - VMWARE s1t2vcenter01 does not receive messages

[s0p4L1n3]

If I use \ I get an error. if \

Yes it is normal, Pipeline rules uses Java language so you need to double \ but for Extractors you need one \ as it does not use java.

[s0p4L1n3]

Now the problem is that Syslog - VMWARE s1t2vcenter01 does not receive messages

It does not receive logs specific to the pipeline rules yes, but it does receive logs on the global stream. So it is not that it does not work, you just need to triggers these event to receives some logs.

If you want to check if this pipeline rule is effective, do these action:

• log in/log out from vsphere
• do action on your virtual machine (snapshot, stop start)

below, i reverted some snapshot etc:

[kossatov1989]

Hello,

As you can see logs suitable for parsing are available

but the dashboard is still empty.

because Pirelins doesn't parse logs.

[kossatov1989]

I did a simulation it's okay

but incoming messages in Streams (VCSA), Pipelines are not processed((((

[kossatov1989]

Strange, but when I select All messages, Pipeline starts working even though VCSA is running messages

[s0p4L1n3]

Check your VCSA stream rules because if the pipeline works when you select all message stream as source, it means the VCSA logs does not go throught the correct stream.

My VCSA stream rules is this one, and yours should be s1t2vcenter01 source must match exactly vcsa

Could you try again a pipeline simulation and instead of pasting Full message in RAW field, paste message field.

If the simulation give you the parsed result, then it should work and it means there is an error somewhere else.

Check also your message processing order, it should read the stream first then the pipeline.

[kossatov1989]

Good morning.

I can see that the thread rules are working and messages are put into it, but they are not processed by Pipelines. full_message - parses the message message - error output

[s0p4L1n3]

On your last image, you can see that the pipeline are processed before the Stream message.

You should setup this way:

1 | Message Filter Chain 2 | Stream Rule Processor 3 | Pipeline Processor

First thing, why do you have VCSA logs on your All message stream (is it your default stream ?), on my vcsa stream setting I ensure to have logs only on this stream and not on the default one.

Then, you should know that you are trying to simulate the parsing with the field Full message while I did not write any parser for this field. You should focuses on message field only.

• When I simulate, I choose VCSA Stream:

• The result are here and are working
• 

BUT FOR YOU, remember that you modified one of you pipeline rules because you had message: Event[xxxxx] instead of Event[] but now you do not see it because it already parse.

BUT if you try to simulate again and if you simulate by pasting the RAW Message (that is already parsed), the pipeline rules will not parse it because it is looking for message field starting by message: Event

You should try change the message processing. Clean the fact that you have VCSA logs in your All message stream because it seems to confuse you Try again to simulate and check the details/debug mode:

It's difficult to me to help you further, but I will try to edit my content pack because as I named some variable vcsa1 or vcsa2 or event pipeline rules name containing VCSA, with my requirements saying replace VCSA by your vcsa server name, it can have conflict somewhere, but I'm not sure because I got some people that told me it was working fine.

[kossatov1989]

I don't understand why parsing from Streams VCSA doesn't work for me.

[s0p4L1n3]

What is your All message streams ? Can you check what is the condition of the stream for message to be forwarded in ?

Did you applied this settings for the message processing order ?

Can you check your syslog input if it is the right one ? Because in my template, the syslog is called Syslog (vcsa) and yours is the default syslog tcp.

It look like to me that syslog TCP input is going to All message stream. but is syslog tcp is related to VCSA syslog ?

[kossatov1989]

Good afternoon, I have not solved this problem, but the logs went and I can see them, I think we can close this issue, I do not have time to deal with it further. Thank you for your help.

[s0p4L1n3]

Alright, feel free to open a new issue, I will be glad to help you at my best !