s0p4L1n3 / Graylog_Content_Pack_VMWare-8.X-forVCSA-ESXI

MIT License
5 stars 0 forks source link

[LookupTableService] Lookup table <rfc_log_level> does not exist #1

Open thefaxe opened 9 months ago

thefaxe commented 9 months ago

Hi,

in pipeline "Pipeline rule Syslog - VMWARE - log_level lookup" the function "let new_level_severity = lookup_value("rfc_log_level", to_string($message.level));"

is flooding the server.log with this message: "[LookupTableService] Lookup table does not exist"

Graylog-Version 5.2

s0p4L1n3 commented 9 months ago

Hello,

Thanks I forgot to add the file in the README !

Download the file and place it on your target folder of your graylog server and edit the Data adapter on System > Lookup Tables > Data Adapter

image

RFC_log_level.csv

thefaxe commented 9 months ago

Thanks for the file. I added the data adapter and I did the job after a complete system reboot.

Another hint for documentation. Before installing the content pack you need to replace all source-entries in json file with the names, using wildcards, of the esxi servers.

s0p4L1n3 commented 9 months ago

Thanks for the hint, I will update the documentation :)

thefaxe commented 9 months ago

Thank you for your quick replies and solution. Dashboard is working like a charm.

pankajgeek commented 5 months ago

Hello,

Thanks I forgot to add the file in the README !

Download the file and place it on your target folder of your graylog server and edit the Data adapter on System > Lookup Tables > Data Adapter

image

RFC_log_level.csv

Hello,

Thanks I forgot to add the file in the README !

Download the file and place it on your target folder of your graylog server and edit the Data adapter on System > Lookup Tables > Data Adapter

image

RFC_log_level.csv

Hello,

Thanks I forgot to add the file in the README !

Download the file and place it on your target folder of your graylog server and edit the Data adapter on System > Lookup Tables > Data Adapter

image

RFC_log_level.csv

Hi, I created the Data adapter but it throws an error :

Screenshot 2024-05-31 at 10 53 23 AM

I am using graylog version 6.0.2

s0p4L1n3 commented 5 months ago

I will try to update on Graylog 6.X to see if I have the same errors. I've tested only with Graylog 5.X for now.

s0p4L1n3 commented 5 months ago

@pankajgeek I've upgraded from 5.2.7 to 6.0.2, nothing change, no errors on my side. Could you describe the steps you did to reproduce the issue?

deeprain279 commented 5 months ago

I had the same error today. I use Graylog 6.0.2 in a Docker container. If this is also the case here, you have to pay attention to the path of the CSV file. /srv does not exist in the Docker context. My Graylog data, which I have persistently in the file system of the host, I simply put the CSV file here. I then have the path in the container: /usr/share/graylog/data/RFC_log_level.csv Otherwise, follow the instructions in the image from the first answer.

Maybe that helps?

pankajgeek commented 5 months ago

I am not using Docker my graylog server is running behind Nginx

pankajgeek commented 5 months ago

@pankajgeek I've upgraded from 5.2.7 to 6.0.2, nothing change, no errors on my side. Could you describe the steps you did to reproduce the issue? <img width="1481"

Screenshot 2024-06-07 at 8 09 19 PM Screenshot 2024-06-07 at 8 12 18 PM

I just deleted and created the Data Adapter again, it disappeared now. I have added Cache and Data lookup table. I hope this is the correct configuration ?

pankajgeek commented 5 months ago

Also, I don't see any data difference though w.r.t logs. For example. according to CSV, Warning should be displayed as level 4 but I am seeing level 6 for everything. Is this expected?

Screenshot 2024-06-07 at 9 10 57 PM

Another issue that has started poping up from yesterday is :

Screenshot 2024-06-07 at 9 15 49 PM

Could you please guide me to fix these problems

s0p4L1n3 commented 5 months ago

If you recreated the lookup table, make sure that the pipelines rules related to this lookup table is correct with the correct name.

As I explained on the requirements before importing the content pack, if you do not use /srv as the folder path for your csv files, it can be changed by editing the data adapter.

Not a problem if you you Nginx instead of docker.

The lookup table for log level is only here to enrich data visualisation and add a new field matching the log level value sent by Vcenter/esxi. If you receive only level 6 you should look onto the vcenter setting to increase log level.

For the new error, could you find the message id related to this error and copy the message and I will test it to see if there is anything to change/fix on the content pack.

pankajgeek commented 5 months ago

Hi,

I created the pipeline rule as below: `rule "VMWare Vcenter and ESXi: Level lookup with Level Category"

when has_field("level") then let update_source = lookup_value("rfc_5424_log_level",$message.level); set_field("level", update_source);

end`

Could you please let me know, which pipeline and stage I should attach this ?

s0p4L1n3 commented 5 months ago

Why don’t you use the content pack I’ve provided. You’ve created a pipeline rule without knowing how to attach it ?

You want create a new field level but it already exist. You need to change the set field(''level'',) to set field(''level_description'',)

You need to attach the pipeline rule to the vcenter stream, the stage depends if you have multiple rules.

I can not help further if you don’t use something from the content pack. If you have any issue on something you created and not related to this content pack, please open an issue on the community forum.

pankajgeek commented 5 months ago

I am sorry to bug you. I am using your content pack and it has been very useful to me having so much of information and data. It's just that i am new to Graylog and have started using it for 2 weeks. In the mean time I am looking at all the stuff that it can do, that's where I had found out that data adapter had a problem. BTW your solution has worked like charm. Have a good day!