Graylog2 / graylog2-server

Free and open log management
https://www.graylog.org
Other
7.38k stars 1.06k forks source link

Alerts and Events dashboard not loading #10252

Closed briantownjr closed 3 years ago

briantownjr commented 3 years ago

Expected Behavior

Alerts & Event dashboard loads and shows proper data

Current Behavior

Alerts & event dashboard just cycles on "Loading Events information..." and never goes further. Event Definitions and Notification buttons all properly load their data.

Possible Solution

Not sure what could be done, I have upgraded to newest Graylog version and still an issue and even moved entire system to a new OS. Possibly an issue with mongoDB somewhere? Since I did port over the mongoDB config as I did not want to lose my definitions from previous system

Steps to Reproduce (for bugs)

  1. Installed Graylog via apt-get and following instructions on main site for MongoDB/ES setup
  2. Configured system and copied over mongoDB backup from old system to new system
  3. Updated Graylog to Graylog v4.0.5+d95b909
  4. Not sure if there is a method to just blow away alert definitions from MongoDB without a full resinstall

Context

Some users complaining about this and not being able to see the Alerts and Events going on at a glance. Email notifications are working properly but this is still an issue.

Your Environment

dennisoelkers commented 3 years ago

Hey @briantownjr,

thanks for reporting this! Do you see anything in your server's log and/or browser's console when this is happening?

briantownjr commented 3 years ago

hey @dennisoelkers

I am seeing these in my server.log:

ERROR [AbstractRotationStrategy] Cannot perform rotation of index in index set with strategy at this moment 2021-03-15T16:55:28.315Z ERROR [AbstractRotationStrategy] Cannot perform rotation of index in index set with strategy at this moment

Not sure if that corresponds to the issue i'm seeing or not.

briantownjr commented 3 years ago

So looks like I had like 25 unassigned shards and my cluster was red. Found I could do this:

curl -XGET localhost:9200/_cat/shards?h=index,shard,prirep,state,unassigned.reason| grep UNASSIGNED

To get all the shard names and then run:

curl -XDELETE 'localhost:9200/index_name/'

to delete them.

Once I had deleted all 24 I restarted elasticsearch and now things seem to be back to normal. Think it was just an issue with my indexes and also trying to copy them over to the new system.