Closed hhclpsg closed 3 years ago
rtudo
commented
3 years ago @hhclpsg - from which graylog previous version did you upgrade to 4.0 ? From exception it looks like some event_definition is not able to get initialised, maybe users are not synced properly. I would manually go and cleanup those faulty event_definitions from mongodb to recover my instance and then recreate the events from ui or using api.
hhclpsg
commented
3 years ago @sowhatim Our upgrade path/history was: graylog-server-3.1.4-1.noarch graylog-server-3.2.6-1.noarch graylog-server-3.3.8-1.noarch graylog-server-4.0.0-8.noarch graylog-server-4.0.1-1.noarch
[...]From exception it looks like some event_definition is not able to get initialised, maybe users are not synced properly.[...] I have checked the existing event_definition for anomalies, but found nothing noticeable.
No event definition uses users as recipients. Only email recipients are defined.
There were no changes to the stream, alert or notification definition between Graylog server updates.
hhclpsg
commented
3 years ago Update: I've deleted all event definitions and notifications. I still get the same error when loading the streams page.
hhclpsg
commented
3 years ago I don't know why the following stream caused the problem. But after I deleted the stream via API the streams page works again.
API Call "GET /streams worked" all the time, even with the problematic stream. But "GET /streams/paginated" did not work until I deleted the stream below.
{
"id": "58dadbaeab4090047d73117f",
"creator_user_id": "xxx",
"outputs": [],
"matching_type": "AND",
"description": "system out of memory",
"created_at": "2017-03-28T21:54:54.692Z",
"disabled": false,
"rules": [
{
"field": "message",
"stream_id": "58dadbaeab4090047d73117f",
"description": "Out of memory: kill",
"id": "58dadbe4ab4090047d7311ba",
"type": 2,
"inverted": false,
"value": "^.*Out of memory: Kill.*$"
}
],
"alert_conditions": [
{
"id": "6e43c572-d7a0-401f-8b57-ea4e8750dd1e",
"type": "message_count",
"creator_user_id": "xxx",
"created_at": "2017-06-19T10:39:59.728+0000",
"parameters": {
"backlog": 5,
"repeat_notifications": false,
"grace": 0,
"threshold_type": "MORE",
"threshold": 0,
"time": 5
},
"title": "Out of memory"
}
],
"alert_receivers": {
"emails": [
"xxx@xxx.xxx"
],
"users": []
},
"title": "Out of memory",
"content_pack": null,
"remove_matches_from_default_stream": true,
"index_set_id": "5914b2baab40903f9e03f176",
"is_default": false
}ah, glad that you worked it out. Maybe that creator_user_id's user was the issue.
aosiniao
commented
3 years ago I have the same problem. How to find the broken stream?
hhclpsg
commented
3 years ago I used the API browser to find the stream.
Execute the following API call for every configured stream:
In my case I got a valid result for all streams except the stream with the title "Out of memory".
der-eismann
commented
3 years ago Thanks to the workaround from @hhclpsg I was also able to find the 16 problematic streams. My problem is that I can't delete them, because the default streams (All system events, All messages, All events) are also affected. When I try to query them via them streams/paginated endpoint I get an Error 500:
{
"type": "ApiError",
"message": "IOException encountered while reading from a byte array input stream"
}But they look pretty normal to me:
{
"_id": "000000000000000000000003",
"creator_user_id": "admin",
"is_default_stream": false,
"index_set_id": "5d5a94113f084d00106c1541",
"matching_type": "AND",
"remove_matches_from_default_stream": true,
"description": "Stream containing all system events created by Graylog",
"created_at": "2019-08-19T12:20:33.341Z",
"disabled": false,
"title": "All system events"
}Any other ideas how to fix that?
xbb
commented
3 years ago I'm having the same issue, however it seems you can still search the affected stream if you browse to the stream specific url:
/streams/<stream id>/searchThis is the problematic stream in my case, but it's not really different from my other streams:
{
"id": "57fc2182b3e1aa035cecb20b",
"creator_user_id": "admin",
"outputs": [],
"matching_type": "AND",
"description": "VDSL Status",
"created_at": "2016-10-10T23:17:22.850Z",
"disabled": false,
"rules": [
{
"field": "message",
"stream_id": "57fc2182b3e1aa035cecb20b",
"description": "",
"id": "57fc21d0b3e1aa035cecb261",
"type": 2,
"inverted": false,
"value": ".*DSL_Status.*"
},
{
"field": "source",
"stream_id": "57fc2182b3e1aa035cecb20b",
"description": "",
"id": "58e97628b3e1aa4757693016",
"type": 1,
"inverted": false,
"value": "draytek2860"
}
],
"alert_conditions": [],
"alert_receivers": {
"emails": [],
"users": [
"admin"
]
},
"title": "Draytek2860 VDSL Status",
"content_pack": null,
"remove_matches_from_default_stream": false,
"index_set_id": "58a39433b3e1aa5e03a04fbb",
"is_default": false
}Log:
2020-12-11T00:06:52.051+01:00 ERROR [AnyExceptionClassMapper] Unhandled exception in REST resource
java.lang.RuntimeException: IOException encountered while reading from a byte array input stream
at org.mongojack.internal.stream.JacksonDBDecoder.decode(JacksonDBDecoder.java:67) ~[graylog.jar:?]
at com.mongodb.DBDecoderAdapter.decode(DBDecoderAdapter.java:49) ~[graylog.jar:?]
at com.mongodb.DBDecoderAdapter.decode(DBDecoderAdapter.java:29) ~[graylog.jar:?]
at com.mongodb.operation.CommandResultArrayCodec.decode(CommandResultArrayCodec.java:52) ~[graylog.jar:?]
at com.mongodb.operation.CommandResultDocumentCodec.readValue(CommandResultDocumentCodec.java:60) ~[graylog.jar:?]
at org.bson.codecs.BsonDocumentCodec.decode(BsonDocumentCodec.java:84) ~[graylog.jar:?]
at org.bson.codecs.BsonDocumentCodec.decode(BsonDocumentCodec.java:41) ~[graylog.jar:?]
at org.bson.internal.LazyCodec.decode(LazyCodec.java:48) ~[graylog.jar:?]
at org.bson.codecs.BsonDocumentCodec.readValue(BsonDocumentCodec.java:101) ~[graylog.jar:?]
at com.mongodb.operation.CommandResultDocumentCodec.readValue(CommandResultDocumentCodec.java:63) ~[graylog.jar:?]
at org.bson.codecs.BsonDocumentCodec.decode(BsonDocumentCodec.java:84) ~[graylog.jar:?]
at org.bson.codecs.BsonDocumentCodec.decode(BsonDocumentCodec.java:41) ~[graylog.jar:?]
at com.mongodb.internal.connection.ReplyMessage.<init>(ReplyMessage.java:51) ~[graylog.jar:?]
at com.mongodb.internal.connection.InternalStreamConnection.getCommandResult(InternalStreamConnection.java:413) ~[graylog.jar:?]
at com.mongodb.internal.connection.InternalStreamConnection.receiveCommandMessageResponse(InternalStreamConnection.java:309) ~[graylog.jar:?]
at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:259) ~[graylog.jar:?]
at com.mongodb.internal.connection.UsageTrackingInternalConnection.sendAndReceive(UsageTrackingInternalConnection.java:99) ~[graylog.jar:?]
at com.mongodb.internal.connection.DefaultConnectionPool$PooledConnection.sendAndReceive(DefaultConnectionPool.java:450) ~[graylog.jar:?]
at com.mongodb.internal.connection.CommandProtocolImpl.execute(CommandProtocolImpl.java:72) ~[graylog.jar:?]
at com.mongodb.internal.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:226) ~[graylog.jar:?]
at com.mongodb.internal.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:269) ~[graylog.jar:?]
at com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:131) ~[graylog.jar:?]
at com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:123) ~[graylog.jar:?]
at com.mongodb.operation.CommandOperationHelper.executeCommand(CommandOperationHelper.java:343) ~[graylog.jar:?]
at com.mongodb.operation.CommandOperationHelper.executeCommand(CommandOperationHelper.java:334) ~[graylog.jar:?]
at com.mongodb.operation.CommandOperationHelper.executeCommandWithConnection(CommandOperationHelper.java:220) ~[graylog.jar:?]
at com.mongodb.operation.FindOperation$1.call(FindOperation.java:731) ~[graylog.jar:?]
at com.mongodb.operation.FindOperation$1.call(FindOperation.java:725) ~[graylog.jar:?]
at com.mongodb.operation.OperationHelper.withReadConnectionSource(OperationHelper.java:463) ~[graylog.jar:?]
at com.mongodb.operation.FindOperation.execute(FindOperation.java:725) ~[graylog.jar:?]
at com.mongodb.operation.FindOperation.execute(FindOperation.java:89) ~[graylog.jar:?]
at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:196) ~[graylog.jar:?]
at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:177) ~[graylog.jar:?]
at com.mongodb.DBCursor.initializeCursor(DBCursor.java:989) ~[graylog.jar:?]
at com.mongodb.DBCursor.hasNext(DBCursor.java:172) ~[graylog.jar:?]
at org.mongojack.DBCursor.hasNext(DBCursor.java:330) ~[graylog.jar:?]
at java.util.Iterator.forEachRemaining(Iterator.java:115) ~[?:1.8.0_272]
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801) ~[?:1.8.0_272]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) ~[?:1.8.0_272]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) ~[?:1.8.0_272]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[?:1.8.0_272]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_272]
at java.util.stream.LongPipeline.reduce(LongPipeline.java:461) ~[?:1.8.0_272]
at java.util.stream.LongPipeline.sum(LongPipeline.java:419) ~[?:1.8.0_272]
at java.util.stream.ReferencePipeline.count(ReferencePipeline.java:593) ~[?:1.8.0_272]
at org.graylog2.database.PaginatedDbService.findPaginatedWithQueryFilterAndSort(PaginatedDbService.java:159) ~[graylog.jar:?]
at org.graylog2.streams.PaginatedStreamService.findPaginated(PaginatedStreamService.java:48) ~[graylog.jar:?]
at org.graylog2.rest.resources.streams.StreamResource.getPage(StreamResource.java:210) ~[graylog.jar:?]
at sun.reflect.GeneratedMethodAccessor338.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_272]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_272]
at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:469) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:391) ~[graylog.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:80) ~[graylog.jar:?]
at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:253) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:292) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:274) [graylog.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:244) [graylog.jar:?]
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) [graylog.jar:?]
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:232) [graylog.jar:?]
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) [graylog.jar:?]
at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:356) [graylog.jar:?]
at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:200) [graylog.jar:?]
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180) [graylog.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_272]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_272]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_272]
Caused by: com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of `org.graylog2.rest.models.alarmcallbacks.requests.AlertReceivers`, problem: Null emails
at [Source: de.undercouch.bson4jackson.io.LittleEndianInputStream@4f9af7b3; pos: 133] (through reference chain: org.graylog2.streams.StreamDTO["alert_receivers"])
at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:67) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.DeserializationContext.instantiationException(DeserializationContext.java:1608) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.wrapAsJsonMappingException(StdValueInstantiator.java:484) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.rewrapCtorProblem(StdValueInstantiator.java:503) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createFromObjectWith(StdValueInstantiator.java:285) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.ValueInstantiator.createFromObjectWith(ValueInstantiator.java:229) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.impl.PropertyBasedCreator.build(PropertyBasedCreator.java:195) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:488) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1287) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:326) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:159) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:530) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:528) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:417) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1287) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:326) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:159) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:3985) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2343) ~[graylog.jar:?]
at org.mongojack.internal.stream.JacksonDBDecoder.decode(JacksonDBDecoder.java:80) ~[graylog.jar:?]
at org.mongojack.internal.stream.JacksonDBDecoder.decode(JacksonDBDecoder.java:64) ~[graylog.jar:?]
... 73 more
Caused by: java.lang.NullPointerException: Null emails
at org.graylog2.rest.models.alarmcallbacks.requests.$AutoValue_AlertReceivers.<init>($AutoValue_AlertReceivers.java:18) ~[graylog.jar:?]
at org.graylog2.rest.models.alarmcallbacks.requests.AutoValue_AlertReceivers.<init>(AutoValue_AlertReceivers.java:9) ~[graylog.jar:?]
at org.graylog2.rest.models.alarmcallbacks.requests.AlertReceivers.create(AlertReceivers.java:40) ~[graylog.jar:?]
at sun.reflect.GeneratedMethodAccessor339.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_272]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_272]
at com.fasterxml.jackson.databind.introspect.AnnotatedMethod.call(AnnotatedMethod.java:109) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createFromObjectWith(StdValueInstantiator.java:283) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.ValueInstantiator.createFromObjectWith(ValueInstantiator.java:229) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.impl.PropertyBasedCreator.build(PropertyBasedCreator.java:195) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:488) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1287) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:326) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:159) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:530) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:528) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:417) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1287) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:326) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:159) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:3985) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2343) ~[graylog.jar:?]
at org.mongojack.internal.stream.JacksonDBDecoder.decode(JacksonDBDecoder.java:80) ~[graylog.jar:?]
at org.mongojack.internal.stream.JacksonDBDecoder.decode(JacksonDBDecoder.java:64) ~[graylog.jar:?]
... 73 more
bernd
commented
3 years ago @hhclpsg Thank you for the report! We identified the problem and are working on a fix for the next 4.0.x bug fix release.
bernd
commented
3 years ago @xbb The problems is, that you most probably have an incomplete alert_receivers object in the database. The API response is showing a complete alert_receivers object because the backend takes care of that. The new paginated endpoint is using a different method of loading the data from the database. We created a fix for that which will be part of one of the next 4.0.x stable releases.
ghost
commented
3 years ago The "Streams" tab still did not work for us after upgrading to 4.0.2. I fixed it locally, but in case the information is of any value:
In our case we were missing the "matching_type" field in some streams with a created_at from back in 2015. The same problem affected two separate installations of graylog.
I fixed it by running:
db.streams.updateMany({'matching_type': {$exists : false}}, {$set: {'matching_type': 'AND'}})We also had problems loading the dashboard tab, the error reported contained:
Caused by: com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of `org.graylog.plugins.views.search.searchtypes.pivot.buckets.TimeUnitInterval$Builder`, problem: Time unit must be {quantity}{unit}, where quantity is a positive number and unit [smhdwM].
at [Source: de.undercouch.bson4jackson.io.LittleEndianInputStream@37d0d978; pos: 668] (through reference chain: org.graylog.plugins.views.search.AutoValue_Search$Builder["queries"]->org.graylog.plugins.views.search.AutoValue_Query$Builder["search_types"]->java.util.HashSet[0]->org.graylog.plugins.views.search.searchtypes.pivot.AutoValue_Pivot$Builder["row_groups"]->java.util.ArrayList[0]->org.graylog.plugins.views.search.searchtypes.pivot.buckets.AutoValue_Time$Builder["interval"])I didn't manage to figure out what was wrong exactly, but I "fixed" it by just deleting all the documents from the "views" collection (no real loss in our case).
mle-ii
commented
3 years ago We've recently updated and are hitting this issue as well. We're on 4.0.6, though the call stack seems to most match the workaround mentioned here above. Is there a code fix for this or do we have to also change the data in the database to make graylog work?
2021-04-26 18:01:30,255 ERROR: org.graylog2.shared.rest.exceptionmappers.AnyExceptionClassMapper - Unhandled exception in REST resource
java.lang.RuntimeException: IOException encountered while reading from a byte array input stream
at org.mongojack.internal.stream.JacksonDBDecoder.decode(JacksonDBDecoder.java:67) ~[graylog.jar:?]
at com.mongodb.DBDecoderAdapter.decode(DBDecoderAdapter.java:49) ~[graylog.jar:?]
at com.mongodb.DBDecoderAdapter.decode(DBDecoderAdapter.java:29) ~[graylog.jar:?]
at com.mongodb.operation.CommandResultArrayCodec.decode(CommandResultArrayCodec.java:52) ~[graylog.jar:?]
at com.mongodb.operation.CommandResultDocumentCodec.readValue(CommandResultDocumentCodec.java:60) ~[graylog.jar:?]
at org.bson.codecs.BsonDocumentCodec.decode(BsonDocumentCodec.java:84) ~[graylog.jar:?]
SNIP
Caused by: com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of `org.graylog2.streams.StreamDTO`, problem: Null matchingType
at [Source: de.undercouch.bson4jackson.io.LittleEndianInputStream@2955750a; pos: 653]
at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:67) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.DeserializationContext.instantiationException(DeserializationContext.java:1608) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.wrapAsJsonMappingException(StdValueInstantiator.java:484) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.rewrapCtorProblem(StdValueInstantiator.java:503) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createFromObjectWith(StdValueInstantiator.java:285) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.ValueInstantiator.createFromObjectWith(ValueInstantiator.java:229) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.impl.PropertyBasedCreator.build(PropertyBasedCreator.java:195) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:488) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1287) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:326) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:159) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:3985) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2343) ~[graylog.jar:?]
at org.mongojack.internal.stream.JacksonDBDecoder.decode(JacksonDBDecoder.java:80) ~[graylog.jar:?]
at org.mongojack.internal.stream.JacksonDBDecoder.decode(JacksonDBDecoder.java:64) ~[graylog.jar:?]
... 74 more
Caused by: java.lang.NullPointerException: Null matchingType
at org.graylog2.streams.$AutoValue_StreamDTO.<init>($AutoValue_StreamDTO.java:71) ~[graylog.jar:?]
at org.graylog2.streams.AutoValue_StreamDTO.<init>(AutoValue_StreamDTO.java:19) ~[graylog.jar:?]
at org.graylog2.streams.StreamDTO.create(StreamDTO.java:141) ~[graylog.jar:?]
at sun.reflect.GeneratedMethodAccessor1123.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_192-internal]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_192-internal]
at com.fasterxml.jackson.databind.introspect.AnnotatedMethod.call(AnnotatedMethod.java:109) ~[graylog.jar:?]Looking at the data, if I get the json back from /api/streams/ it seems to show a matching_type of AND or OR on all items. So maybe this is something else or that streams route is not returning all the data.
mle-ii
commented
3 years ago I can create a new issue if this is indeed a newer/different issue. And it is the paginated route that is failing. All others so far seem to work fine. This one fails with the error above.
/api/streams/paginated?page=1&per_page=50&query=test&sort=title&order=asc
mle-ii
commented
3 years ago It does appear that some of the records do not have the matching_type set even though the /api/streams route seems to show that they do. So perhaps modifying the db like the other person did may fix this. Though it does seem like this should be fixed at the application level. Guessing these were older streams that didn't have that value set.
> db.streams.find( { 'matching_type': {$exists : false}} ).count()
24
> db.streams.find( { 'matching_type': {$exists : true}} ).count()
118We were able to solve this failure in the same way by modifying the streams data in mongodb. But this should be handled by Graylog to support upgrades like this with old data. Some of those streams we had were from around 2014.
danotorrey
commented
3 years ago @mle-ii Thanks for the feedback. I've opened a new issue for this including a description of a potential software fix.
After updating to Graylog 4.0.1 the streams page does not work anymore. Search and message processing works without problems.
Expected Behavior
The page should load without errors.
Current Behavior
Error displayed in browser: Could not load streams Loading streams failed with status: Error: cannot GET https://graylog.XXX.de/api/streams/paginated?page=1&per_page=10 (500)
Context
Graylog server log (Loglevel set to TRACE):
Your Environment