Open S1L3NC3 opened 5 years ago
edmundoa
commented
5 years ago Hello @S1L3NC3,
We need a few more details to be able to help you. Is this happening on a new setup or did you upgrade your existing one? Also, would you be so kind as to share with us the Content Packs in your Graylog setup?
Thank you!
S1L3NC3
commented
5 years ago Thanks for ur answer @edmundoa . It's a new install via github repository. Since "loading" stage appers, I can't see the Content Packs installed by default. Under "Select Content Packs" there isn't anything, just "loading...". I had tried to get them with the following mongodb sentence:
db.content_packs.find({ "id" : { "$exists" : true } });
I attached the parse .txt
Kind Regards, thanks!
S1L3NC3
commented
5 years ago Hi again: I ckecked u pointed this issue as "bug". Can this be solved? Thanks!
kmerz
commented
5 years ago @S1L3NC3 could it be that you used your DB to test an upgrade with 3.0? The content pack collection does have output of 3.0 migrations. If yes, then all you can do now is remove the 3.0 content packs from your collection and remove the id field from your pre 3.0 content packs.
S1L3NC3
commented
5 years ago Thanks for the response @kmerz . I checked the parse .txt and i saw there are 6 content packs with reference 3.0, but this was a new installation. Strange...
Since Content Pack page is permanently loading I cannot erase them from Graylog web.
Just I did a backup to the machine and then I erased them within mongodb with the following sentence:
db.content_packs.remove({ "category" : { "$exists" : false } });
This erased 6 content packs, but problem still goes on.
I will check this again tomorrow :) Thanks in advance.
dostoevskoy
commented
5 years ago Hi,
I met the same issue, when "Content packs" stucks in "loading". But my situation was a little bit another. It appeared after I had upgraded my graylog container from 3.0.0 to 3.0.1.
Environment: OS: Docker graylog/graylog Graylog ver: 3.0.1 Mongdb ver: 3.4
Error:
graylog_1 | Caused by: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "requires" (class org.graylog2.contentpacks.model.AutoValue_ContentPackV1$Builder), not marked as ignorable (13 known properties: "entities", "rev", "summary", "vendor", "created_at", "name", "url", "v", "_id", "id", "description", "parameters", "server_version"]) graylog_1 | at [Source: de.undercouch.bson4jackson.io.LittleEndianInputStream@7e4adf0d; pos: 610] (through reference chain: org.graylog2.contentpacks.model.AutoValue_ContentPackV1$Builder["requires"])
I compeared content_packs from a fresh installation of graylog:3.0.1 and upgraded one and find, that upgrade process changed items into this collection and added additional field "requires". You can find the example below.
An item from the fresh installation:
{ "_id" : ObjectId("5cb84b513b3af5000f35f2cc"), "id" : "1794d39d-077f-7360-b92b-95411b05fbce", "rev" : 1, "v" : "1", "name" : "Whois - Threat Intel Plugin", "summary" : "The Whois Lookup Table of the Threat Intel Plugin", "description" : "This content pack is part of the [Graylog Threat Intel Plugin](http://github.com/graylog-labs/graylog-plugin-threatintel).\n\n*Please do not delete it manually if you consider to use the Threat Intel Plugin's functions. It contains important resources required by the plugin.*\n\nThis is the lookup table for the WHOIS database, listing registered users of Internet resources like IPs, Netblocks or Domain Names. This lookup table is used internally by Graylog's Threat Intel Plugin. Do not delete it manually.", "vendor" : "Graylog <hello@graylog.com>", "url" : "https://github.com/Graylog2/graylog2-server", "created_at" : ISODate("2019-04-18T10:02:57.800Z"), "server_version" : "3.0.1+de74b68", "parameters" : [ ], "entities" : [ { "id" : "5ac762873d274666e34eca83", "type" : { "name" : "lookup_adapter", "version" : "1" }, "v" : "1", "data" : { "configuration" : { "connect_timeout" : { "@type" : "integer", "@value" : 1000 }, "read_timeout" : { "@type" : "integer", "@value" : 1000 }, "registry" : { "@type" : "string", "@value" : "ARIN" }, "type" : { "@type" : "string", "@value" : "whois" } }, "description" : { "@type" : "string", "@value" : "This is the data adapter for the WHOIS database, listing registered users of Internet resources like IPs, Netblocks or Domain Names. This adapter is used internally by Graylog's Threat Intel Plugin. Do not delete it manually." }, "name" : { "@type" : "string", "@value" : "whois" }, "title" : { "@type" : "string", "@value" : "Whois" } }, "constraints" : [ { "type" : "plugin-version", "plugin" : "org.graylog.plugins.threatintel.ThreatIntelPlugin", "version" : ">=3.0.0-alpha.2" }, { "type" : "server-version", "version" : ">=3.0.0-alpha.2+af8d8e0" } ] }, { "id" : "5ac762873d274666e34eca7c", "type" : { "name" : "lookup_cache", "version" : "1" }, "v" : "1", "data" : { "configuration" : { "expire_after_access" : { "@type" : "long", "@value" : 0 }, "expire_after_access_unit" : { "@type" : "string", "@value" : "DAYS" }, "expire_after_write" : { "@type" : "long", "@value" : 1 }, "expire_after_write_unit" : { "@type" : "string", "@value" : "DAYS" }, "max_size" : { "@type" : "integer", "@value" : 1000 }, "type" : { "@type" : "string", "@value" : "guava_cache" } }, "description" : { "@type" : "string", "@value" : "This is the cache for the WHOIS database, listing registered users of Internet resources like IPs, Netblocks or Domain Names. This cache is used internally by Graylog's Threat Intel Plugin. Do not delete it manually." }, "name" : { "@type" : "string", "@value" : "whois-cache" }, "title" : { "@type" : "string", "@value" : "Whois Cache" } }, "constraints" : [ { "type" : "plugin-version", "plugin" : "org.graylog.plugins.threatintel.ThreatIntelPlugin", "version" : ">=3.0.0-alpha.2" }, { "type" : "server-version", "version" : ">=3.0.0-alpha.2+af8d8e0" } ] }, { "id" : "5ac762873d274666e34eca90", "type" : { "name" : "lookup_table", "version" : "1" }, "v" : "1", "data" : { "cache_name" : { "@type" : "string", "@value" : "5ac762873d274666e34eca7c" }, "data_adapter_name" : { "@type" : "string", "@value" : "5ac762873d274666e34eca83" }, "default_multi_value" : { "@type" : "string", "@value" : "" }, "default_multi_value_type" : { "@type" : "string", "@value" : "NULL" }, "default_single_value" : { "@type" : "string", "@value" : "" }, "default_single_value_type" : { "@type" : "string", "@value" : "NULL" }, "description" : { "@type" : "string", "@value" : "This is the lookup table for the WHOIS database, listing registered users of Internet resources like IPs, Netblocks or Domain Names. This lookup table is used internally by Graylog's Threat Intel Plugin. Do not delete it manually." }, "name" : { "@type" : "string", "@value" : "whois" }, "title" : { "@type" : "string", "@value" : "Whois" } }, "constraints" : [ { "type" : "plugin-version", "plugin" : "org.graylog.plugins.threatintel.ThreatIntelPlugin", "version" : ">=3.0.0-alpha.2" }, { "type" : "server-version", "version" : ">=3.0.0-alpha.2+af8d8e0" } ] } ] }
An item after the uprgading process:
{ "_id" : ObjectId("5c5193b3e64ada0001b56930"), "id" : "1794d39d-077f-7360-b92b-95411b05fbce", "rev" : 1, "v" : "1", "name" : "Whois - Threat Intel Plugin", "summary" : "The Whois Lookup Table of the Threat Intel Plugin", "description" : "This content pack is part of the [Graylog Threat Intel Plugin](http://github.com/graylog-labs/graylog-plugin-threatintel).\n\n*Please do not delete it manually if you consider to use the Threat Intel Plugin's functions. It contains important resources required by the plugin.*\n\nThis is the lookup table for the WHOIS database, listing registered users of Internet resources like IPs, Netblocks or Domain Names. This lookup table is used internally by Graylog's Threat Intel Plugin. Do not delete it manually.", "vendor" : "Graylog <hello@graylog.com>", "url" : "https://github.com/Graylog2/graylog2-server", "requires" : [ { "type" : "plugin-version", "plugin" : "org.graylog.plugins.threatintel.ThreatIntelPlugin", "version" : ">=3.0.0-alpha.2" }, { "type" : "server-version", "version" : ">=3.0.0-alpha.2+af8d8e0" } ], "parameters" : [ ], "entities" : [ { "id" : "5ac762873d274666e34eca83", "type" : { "name" : "lookup_adapter", "version" : "1" }, "v" : "1", "data" : { "configuration" : { "connect_timeout" : { "type" : "integer", "value" : 1000 }, "read_timeout" : { "type" : "integer", "value" : 1000 }, "registry" : { "type" : "string", "value" : "ARIN" }, "type" : { "type" : "string", "value" : "whois" } }, "description" : { "type" : "string", "value" : "This is the data adapter for the WHOIS database, listing registered users of Internet resources like IPs, Netblocks or Domain Names. This adapter is used internally by Graylog's Threat Intel Plugin. Do not delete it manually." }, "name" : { "type" : "string", "value" : "whois" }, "title" : { "type" : "string", "value" : "Whois" } } }, { "id" : "5ac762873d274666e34eca7c", "type" : { "name" : "lookup_cache", "version" : "1" }, "v" : "1", "data" : { "configuration" : { "expire_after_access" : { "type" : "long", "value" : 0 }, "expire_after_access_unit" : { "type" : "string", "value" : "DAYS" }, "expire_after_write" : { "type" : "long", "value" : 1 }, "expire_after_write_unit" : { "type" : "string", "value" : "DAYS" }, "max_size" : { "type" : "integer", "value" : 1000 }, "type" : { "type" : "string", "value" : "guava_cache" } }, "description" : { "type" : "string", "value" : "This is the cache for the WHOIS database, listing registered users of Internet resources like IPs, Netblocks or Domain Names. This cache is used internally by Graylog's Threat Intel Plugin. Do not delete it manually." }, "name" : { "type" : "string", "value" : "whois-cache" }, "title" : { "type" : "string", "value" : "Whois Cache" } } }, { "id" : "5ac762873d274666e34eca90", "type" : { "name" : "lookup_table", "version" : "1" }, "v" : "1", "data" : { "cache_name" : { "type" : "string", "value" : "5ac762873d274666e34eca7c" }, "data_adapter_name" : { "type" : "string", "value" : "5ac762873d274666e34eca83" }, "default_multi_value" : { "type" : "string", "value" : "" }, "default_multi_value_type" : { "type" : "string", "value" : "NULL" }, "default_single_value" : { "type" : "string", "value" : "" }, "default_single_value_type" : { "type" : "string", "value" : "NULL" }, "description" : { "type" : "string", "value" : "This is the lookup table for the WHOIS database, listing registered users of Internet resources like IPs, Netblocks or Domain Names. This lookup table is used internally by Graylog's Threat Intel Plugin. Do not delete it manually." }, "name" : { "type" : "string", "value" : "whois" }, "title" : { "type" : "string", "value" : "Whois" } } } ] }
I don't know if this information can be helpful, the issue is still open as I can see. Probably, I've made the upgrade in the wrong way. I've just increased the version of the graylog container and load a new one.
Environment: OS: Debian 9 stretch Graylog ver: 2.5.1 Mongdb ver: 3.6.10 Elasticsearch ver: 6.2.4 Browser: any
Description: In tab "Content packs" within graylog, it stucks in "loading". To reproduce: click on "Content packs" and "loading" never desappears.
Testcase: Checking server.log from graylog: