search

StamusNetworks / SELKS

A Suricata based IDS/IPS/NSM distro
https://www.stamus-networks.com/open-source/#selks
GNU General Public License v3.0
1.45k stars 284 forks source link

first time setup fails on fresh install #356

Closed kdy1968 closed 2 years ago

kdy1968 commented 2 years ago

err you didnt read my mind :), so sorry initially had issues with running the first time setup....but appears that is solved....got side tracked as i tried few things dash boards open to blank page, i am seeing events being logged in scirius 2 unassigned shards arkime_dstats, arkime_stats....unassigned index created, states that there is already a copy used curl -XDELETE 'localhost:9200/index_name/' to try and remove seems they are just recreated

health check seems ok, at least everything is running.....

`root@SELKS:~# selks-health-check_stamus ● suricata.service - LSB: Next Generation IDS/IPS Loaded: loaded (/etc/init.d/suricata; generated) Active: active (running) since Sat 2021-11-27 17:56:31 EST; 39min ago Docs: man:systemd-sysv-generator(8) Process: 878 ExecStart=/etc/init.d/suricata start (code=exited, status=0/SUCCESS) Tasks: 14 (limit: 4915) Memory: 485.9M CGroup: /system.slice/suricata.service └─1001 /usr/bin/suricata -c /etc/suricata/suricata.yaml --pidfile /var/run/suricata.pid --af-packet -D -v --user=logstash

Nov 27 17:56:31 SELKS systemd[1]: Starting LSB: Next Generation IDS/IPS... Nov 27 17:56:31 SELKS suricata[878]: Starting suricata in IDS (af-packet) mode... done. Nov 27 17:56:31 SELKS systemd[1]: Started LSB: Next Generation IDS/IPS. ● elasticsearch.service - Elasticsearch Loaded: loaded (/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled) Active: active (running) since Sat 2021-11-27 17:57:23 EST; 38min ago Docs: https://www.elastic.co Main PID: 883 (java) Tasks: 90 (limit: 4915) Memory: 8.9G CGroup: /system.slice/elasticsearch.service ├─ 883 /usr/share/elasticsearch/jdk/bin/java -Xshare:auto -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -… └─1537 /usr/share/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/controller

Nov 27 17:56:31 SELKS systemd[1]: Starting Elasticsearch... Nov 27 17:56:39 SELKS systemd-entrypoint[883]: WARNING: A terminally deprecated method in java.lang.System has been called Nov 27 17:56:39 SELKS systemd-entrypoint[883]: WARNING: System::setSecurityManager has been called by org.elasticsearch.bootstrap.Elasticsearch (file:/usr/share/elasticsearch/lib/elasticsearch-7.15.2.jar) Nov 27 17:56:39 SELKS systemd-entrypoint[883]: WARNING: Please consider reporting this to the maintainers of org.elasticsearch.bootstrap.Elasticsearch Nov 27 17:56:39 SELKS systemd-entrypoint[883]: WARNING: System::setSecurityManager will be removed in a future release Nov 27 17:56:44 SELKS systemd-entrypoint[883]: WARNING: A terminally deprecated method in java.lang.System has been called Nov 27 17:56:44 SELKS systemd-entrypoint[883]: WARNING: System::setSecurityManager has been called by org.elasticsearch.bootstrap.Security (file:/usr/share/elasticsearch/lib/elasticsearch-7.15.2.jar) Nov 27 17:56:44 SELKS systemd-entrypoint[883]: WARNING: Please consider reporting this to the maintainers of org.elasticsearch.bootstrap.Security Nov 27 17:56:44 SELKS systemd-entrypoint[883]: WARNING: System::setSecurityManager will be removed in a future release Nov 27 17:57:23 SELKS systemd[1]: Started Elasticsearch. ● logstash.service - logstash Loaded: loaded (/etc/systemd/system/logstash.service; enabled; vendor preset: enabled) Active: active (running) since Sat 2021-11-27 17:55:27 EST; 40min ago Main PID: 417 (java) Tasks: 43 (limit: 4915) Memory: 1.0G CGroup: /system.slice/logstash.service └─417 /usr/share/logstash/jdk/bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djruby.compile.invokedynami…

Nov 27 17:58:11 SELKS logstash[417]: [2021-11-27T17:58:11,091][INFO ][logstash.outputs.elasticsearch][main] Installing Elasticsearch template {:name=>"logstash"} Nov 27 17:58:11 SELKS logstash[417]: [2021-11-27T17:58:11,094][INFO ][logstash.outputs.elasticsearch][main] Installing Elasticsearch template {:name=>"logstash"} Nov 27 18:04:00 SELKS logstash[417]: [2021-11-27T18:04:00,455][WARN ][logstash.outputs.elasticsearch][main][e55f734d663b7fb7ca21a05c69227f334d0c6198948f303fac6e50c03be43b13] Could not index event to Elasticsearch. {:status=>404, :action… Nov 27 18:04:00 SELKS logstash[417]: [2021-11-27T18:04:00,459][WARN ][logstash.outputs.elasticsearch][main][e55f734d663b7fb7ca21a05c69227f334d0c6198948f303fac6e50c03be43b13] Could not index event to Elasticsearch. {:status=>404, :action… Nov 27 18:04:00 SELKS logstash[417]: [2021-11-27T18:04:00,461][WARN ][logstash.outputs.elasticsearch][main][e55f734d663b7fb7ca21a05c69227f334d0c6198948f303fac6e50c03be43b13] Could not index event to Elasticsearch. {:status=>404, :action… Nov 27 18:04:00 SELKS logstash[417]: [2021-11-27T18:04:00,463][WARN ][logstash.outputs.elasticsearch][main][e55f734d663b7fb7ca21a05c69227f334d0c6198948f303fac6e50c03be43b13] Could not index event to Elasticsearch. {:status=>404, :action… Nov 27 18:04:00 SELKS logstash[417]: [2021-11-27T18:04:00,464][WARN ][logstash.outputs.elasticsearch][main][e55f734d663b7fb7ca21a05c69227f334d0c6198948f303fac6e50c03be43b13] Could not index event to Elasticsearch. {:status=>404, :action… Nov 27 18:04:00 SELKS logstash[417]: [2021-11-27T18:04:00,466][WARN ][logstash.outputs.elasticsearch][main][e55f734d663b7fb7ca21a05c69227f334d0c6198948f303fac6e50c03be43b13] Could not index event to Elasticsearch. {:status=>404, :action… Nov 27 18:04:12 SELKS logstash[417]: [2021-11-27T18:04:12,509][WARN ][logstash.outputs.elasticsearch][main][e55f734d663b7fb7ca21a05c69227f334d0c6198948f303fac6e50c03be43b13] Could not index event to Elasticsearch. {:status=>400, :action… Nov 27 18:04:12 SELKS logstash[417]: [2021-11-27T18:04:12,707][WARN ][logstash.outputs.elasticsearch][main][e55f734d663b7fb7ca21a05c69227f334d0c6198948f303fac6e50c03be43b13] Could not index event to Elasticsearch. {:status=>400, :action… Hint: Some lines were ellipsized, use -l to show in full. ● kibana.service - Kibana Loaded: loaded (/etc/systemd/system/kibana.service; enabled; vendor preset: enabled) Active: active (running) since Sat 2021-11-27 17:56:31 EST; 39min ago Docs: https://www.elastic.co Main PID: 876 (node) Tasks: 11 (limit: 4915) Memory: 704.6M CGroup: /system.slice/kibana.service └─876 /usr/share/kibana/bin/../node/bin/node /usr/share/kibana/bin/../src/cli/dist --logging.dest=/var/log/kibana/kibana.log --pid.file=/run/kibana/kibana.pid

Nov 27 17:56:31 SELKS systemd[1]: Started Kibana. ● evebox.service - EveBox Server Loaded: loaded (/lib/systemd/system/evebox.service; enabled; vendor preset: enabled) Active: active (running) since Sat 2021-11-27 17:55:27 EST; 40min ago Main PID: 409 (evebox) Tasks: 8 (limit: 4915) Memory: 46.3M CGroup: /system.slice/evebox.service └─409 /usr/bin/evebox server

Nov 27 17:57:12 SELKS evebox[409]: 2021-11-27 17:57:12 (server.go:335) -- Failed to ping Elastic Search, delaying startup: : Get "http://localhost:9200/": dial tcp 127.0.0.1:9200: connect: connection refused Nov 27 17:57:15 SELKS evebox[409]: 2021-11-27 17:57:15 (server.go:335) -- Failed to ping Elastic Search, delaying startup: : Get "http://localhost:9200/": dial tcp 127.0.0.1:9200: connect: connection refused Nov 27 17:57:18 SELKS evebox[409]: 2021-11-27 17:57:18 (server.go:335) -- Failed to ping Elastic Search, delaying startup: : Get "http://localhost:9200/": dial tcp 127.0.0.1:9200: connect: connection refused Nov 27 17:57:21 SELKS evebox[409]: 2021-11-27 17:57:21 (server.go:335) -- Failed to ping Elastic Search, delaying startup: : Get "http://localhost:9200/": dial tcp 127.0.0.1:9200: connect: connection refused Nov 27 17:57:24 SELKS evebox[409]: 2021-11-27 17:57:24 (server.go:338) -- Connected to Elastic Search (version: 7.15.2) Nov 27 17:57:24 SELKS evebox[409]: 2021-11-27 17:57:24 (elasticsearch.go:177) -- Assuming Logstash style index Nov 27 17:57:24 SELKS evebox[409]: 2021-11-27 17:57:24 (server.go:131) -- Session reaper started Nov 27 17:57:24 SELKS evebox[409]: 2021-11-27 17:57:24 (server.go:165) -- Authentication disabled. Nov 27 17:57:24 SELKS evebox[409]: 2021-11-27 17:57:24 (server.go:261) -- Listening on [127.0.0.1]:5636 Nov 27 18:02:44 SELKS evebox[409]: 2021-11-27 18:02:44 (anonymous.go:64) -- Logging in anonymous user {selks-user} from 127.0.0.1:50882 ● molochviewer-selks.service - Moloch Viewer Loaded: loaded (/etc/systemd/system/molochviewer-selks.service; enabled; vendor preset: enabled) Active: active (running) since Sat 2021-11-27 17:58:06 EST; 37min ago Main PID: 1746 (sh) Tasks: 12 (limit: 4915) Memory: 41.6M CGroup: /system.slice/molochviewer-selks.service ├─1746 /bin/sh -c /data/moloch/bin/node viewer.js -c /data/moloch/etc/config.ini >> /data/moloch/logs/viewer.log 2>&1 └─1747 /data/moloch/bin/node viewer.js -c /data/moloch/etc/config.ini

Nov 27 17:58:06 SELKS systemd[1]: Started Moloch Viewer. ● molochpcapread-selks.service - Moloch Pcap Read Loaded: loaded (/etc/systemd/system/molochpcapread-selks.service; enabled; vendor preset: enabled) Active: active (running) since Sat 2021-11-27 17:58:27 EST; 37min ago Main PID: 1812 (sh) Tasks: 6 (limit: 4915) Memory: 950.6M CGroup: /system.slice/molochpcapread-selks.service ├─1812 /bin/sh -c /data/moloch/bin/moloch-capture -c /data/moloch/etc/config.ini -m --copy --delete -R /data/nsm/ >> /data/moloch/logs/capture.log 2>&1 └─1813 /data/moloch/bin/moloch-capture -c /data/moloch/etc/config.ini -m --copy --delete -R /data/nsm/

Nov 27 17:58:27 SELKS systemd[1]: Started Moloch Pcap Read. scirius RUNNING pid 1129, uptime 0:39:30 ii elasticsearch 7.15.2 amd64 Distributed RESTful search engine built for the cloud ii elasticsearch-curator 5.8.4 amd64 Have indices in Elasticsearch? This is the tool for you!\n\nLike a museum curator manages the exhibits and collections on display, \nElasticsearch Curator helps you curate, or manage your indices. ii evebox 1:0.11.1 amd64 no description given ii kibana 7.15.2 amd64 Explore and visualize your Elasticsearch data ii kibana-dashboards-stamus 2020122001 amd64 Kibana 6 dashboard templates. ii logstash 1:7.15.2-1 amd64 An extensible logging pipeline ii moloch 3.1.1-1 amd64 Moloch Full Packet System ii scirius 3.5.0-3 amd64 Django application to manage Suricata ruleset ii suricata 1:2021111201-0stamus0 amd64 Suricata open source multi-thread IDS/IPS/NSM system. Filesystem Type Size Used Avail Use% Mounted on udev devtmpfs 7.8G 0 7.8G 0% /dev tmpfs tmpfs 1.6G 8.9M 1.6G 1% /run /dev/sda1 ext4 94G 8.7G 81G 10% / tmpfs tmpfs 7.8G 140K 7.8G 1% /dev/shm tmpfs tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs tmpfs 7.8G 0 7.8G 0% /sys/fs/cgroup tmpfs tmpfs 1.6G 0 1.6G 0% /run/user/0 ` kibanna log indicates indices are missing..

{"type":"log","@timestamp":"2021-11-27T13:40:01-05:00","tags":["warning","plugins","telemetry","fetcher"],"pid":846,"message":"Error sending telemetry usage data. (FetchError: request to https://telemetry.elastic.co/xpack/v2/send failed, reason: getaddrinfo EAI_AGAIN telemetry.elastic.co)"} {"type":"log","@timestamp":"2021-11-27T16:41:30-05:00","tags":["info","plugins-system","standard"],"pid":846,"message":"Stopping all plugins."} {"type":"log","@timestamp":"2021-11-27T16:44:38-05:00","tags":["info","plugins-service"],"pid":873,"message":"Plugin \"metricsEntities\" is disabled."} {"type":"log","@timestamp":"2021-11-27T16:44:38-05:00","tags":["info","http","server","Preboot"],"pid":873,"message":"http server running at http://localhost:5601"} {"type":"log","@timestamp":"2021-11-27T16:44:38-05:00","tags":["warning","config","deprecation"],"pid":873,"message":"\"logging.dest\" has been deprecated and will be removed in 8.0. To set the destination moving forward, you can use the \"console\" appender in your logging configuration or define a custom one. For more details, see https://github.com/elastic/kibana/blob/master/src/core/server/logging/README.mdx"} {"type":"log","@timestamp":"2021-11-27T16:44:38-05:00","tags":["warning","config","deprecation"],"pid":873,"message":"plugins.scanDirs is deprecated and is no longer used"} {"type":"log","@timestamp":"2021-11-27T16:44:38-05:00","tags":["warning","config","deprecation"],"pid":873,"message":"Config key [monitoring.cluster_alerts.email_notifications.email_address] will be required for email notifications to work in 8.0.\""} {"type":"log","@timestamp":"2021-11-27T16:44:38-05:00","tags":["warning","config","deprecation"],"pid":873,"message":"\"xpack.reporting.roles\" is deprecated. Granting reporting privilege through a \"reporting_user\" role will not be supported starting in 8.0. Please set \"xpack.reporting.roles.enabled\" to \"false\" and grant reporting privileges to users using Kibana application privileges **Management > Security > Roles**."} {"type":"log","@timestamp":"2021-11-27T16:44:38-05:00","tags":["warning","config","deprecation"],"pid":873,"message":"Session idle timeout (\"xpack.security.session.idleTimeout\") will be set to 1 hour by default in the next major version (8.0)."} {"type":"log","@timestamp":"2021-11-27T16:44:38-05:00","tags":["warning","config","deprecation"],"pid":873,"message":"Session lifespan (\"xpack.security.session.lifespan\") will be set to 30 days by default in the next major version (8.0)."} {"type":"log","@timestamp":"2021-11-27T16:44:38-05:00","tags":["info","plugins-system","standard"],"pid":873,"message":"Setting up [113] plugins: [translations,licensing,globalSearch,globalSearchProviders,banners,licenseApiGuard,code,usageCollection,xpackLegacy,taskManager,telemetryCollectionManager,telemetryCollectionXpack,kibanaUsageCollection,securityOss,share,screenshotMode,telemetry,newsfeed,mapsEms,mapsLegacy,legacyExport,kibanaLegacy,embeddable,uiActionsEnhanced,fieldFormats,expressions,charts,esUiShared,bfetch,data,savedObjects,visualizations,visTypeXy,visTypeVislib,visTypeTimelion,features,visTypeTagcloud,visTypeTable,visTypePie,visTypeMetric,visTypeMarkdown,tileMap,regionMap,presentationUtil,expressionShape,expressionRevealImage,expressionRepeatImage,expressionMetric,expressionImage,timelion,indexPatternFieldEditor,home,searchprofiler,painlessLab,grokdebugger,graph,visTypeVega,management,watcher,licenseManagement,indexPatternManagement,advancedSettings,discover,discoverEnhanced,dashboard,dashboardEnhanced,visualize,visTypeTimeseries,savedObjectsManagement,spaces,security,transform,savedObjectsTagging,lens,reporting,canvas,lists,ingestPipelines,fileUpload,maps,dataVisualizer,encryptedSavedObjects,dataEnhanced,dashboardMode,cloud,snapshotRestore,fleet,indexManagement,rollup,remoteClusters,crossClusterReplication,indexLifecycleManagement,eventLog,actions,alerting,triggersActionsUi,stackAlerts,ruleRegistry,osquery,ml,cases,timelines,securitySolution,observability,uptime,infra,upgradeAssistant,monitoring,logstash,enterpriseSearch,console,apmOss,apm]"} {"type":"log","@timestamp":"2021-11-27T16:44:39-05:00","tags":["info","plugins","taskManager"],"pid":873,"message":"TaskManager is identified by the Kibana UUID: 075834e9-02d1-4514-827c-d71de14c5e92"} {"type":"log","@timestamp":"2021-11-27T16:44:40-05:00","tags":["warning","plugins","security","config"],"pid":873,"message":"Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command."} {"type":"log","@timestamp":"2021-11-27T16:44:40-05:00","tags":["warning","plugins","security","config"],"pid":873,"message":"Session cookies will be transmitted over insecure connections. This is not recommended."} {"type":"log","@timestamp":"2021-11-27T16:44:40-05:00","tags":["warning","plugins","reporting","config"],"pid":873,"message":"Generating a random key for xpack.reporting.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.reporting.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command."} {"type":"log","@timestamp":"2021-11-27T16:44:40-05:00","tags":["warning","plugins","reporting","config"],"pid":873,"message":"Chromium sandbox provides an additional layer of protection, but is not supported for Linux Debian 10.4 OS. Automatically setting 'xpack.reporting.capture.browser.chromium.disableSandbox: true'."} {"type":"log","@timestamp":"2021-11-27T16:44:40-05:00","tags":["warning","plugins","encryptedSavedObjects"],"pid":873,"message":"Saved objects encryption key is not set. This will severely limit Kibana functionality. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command."} {"type":"log","@timestamp":"2021-11-27T16:44:40-05:00","tags":["warning","plugins","actions"],"pid":873,"message":"APIs are disabled because the Encrypted Saved Objects plugin is missing encryption key. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command."} {"type":"log","@timestamp":"2021-11-27T16:44:40-05:00","tags":["warning","plugins","alerting"],"pid":873,"message":"APIs are disabled because the Encrypted Saved Objects plugin is missing encryption key. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command."} {"type":"log","@timestamp":"2021-11-27T16:44:40-05:00","tags":["info","plugins","ruleRegistry"],"pid":873,"message":"Write is disabled; not installing common resources shared between all indices"} {"type":"log","@timestamp":"2021-11-27T16:44:42-05:00","tags":["info","plugins","ruleRegistry"],"pid":873,"message":"Write is disabled; not installing resources for index .alerts-observability.uptime.alerts"} {"type":"log","@timestamp":"2021-11-27T16:44:42-05:00","tags":["info","plugins","ruleRegistry"],"pid":873,"message":"Write is disabled; not installing resources for index .alerts-observability.logs.alerts"} {"type":"log","@timestamp":"2021-11-27T16:44:42-05:00","tags":["info","plugins","ruleRegistry"],"pid":873,"message":"Write is disabled; not installing resources for index .alerts-observability.metrics.alerts"} {"type":"log","@timestamp":"2021-11-27T16:44:42-05:00","tags":["info","plugins","ruleRegistry"],"pid":873,"message":"Write is disabled; not installing resources for index .alerts-observability.apm.alerts"} {"type":"log","@timestamp":"2021-11-27T16:44:42-05:00","tags":["info","savedobjects-service"],"pid":873,"message":"Waiting until all Elasticsearch nodes are compatible with Kibana before starting saved objects migrations..."} {"type":"log","@timestamp":"2021-11-27T16:44:48-05:00","tags":["error","savedobjects-service"],"pid":873,"message":"Unable to retrieve version information from Elasticsearch nodes. connect ECONNREFUSED 127.0.0.1:9200"} {"type":"log","@timestamp":"2021-11-27T16:44:50-05:00","tags":["info","savedobjects-service"],"pid":873,"message":"Starting saved objects migrations"} {"type":"log","@timestamp":"2021-11-27T16:44:52-05:00","tags":["info","savedobjects-service"],"pid":873,"message":"[.kibana] INIT -> OUTDATED_DOCUMENTS_SEARCH_OPEN_PIT. took: 335ms."} {"type":"log","@timestamp":"2021-11-27T16:44:52-05:00","tags":["info","savedobjects-service"],"pid":873,"message":"[.kibana] OUTDATED_DOCUMENTS_SEARCH_OPEN_PIT -> OUTDATED_DOCUMENTS_SEARCH_READ. took: 99ms."} {"type":"log","@timestamp":"2021-11-27T16:44:52-05:00","tags":["info","savedobjects-service"],"pid":873,"message":"[.kibana_task_manager] INIT -> OUTDATED_DOCUMENTS_SEARCH_OPEN_PIT. took: 463ms."} {"type":"log","@timestamp":"2021-11-27T16:44:52-05:00","tags":["info","savedobjects-service"],"pid":873,"message":"[.kibana_task_manager] OUTDATED_DOCUMENTS_SEARCH_OPEN_PIT -> OUTDATED_DOCUMENTS_SEARCH_READ. took: 18ms."} {"type":"log","@timestamp":"2021-11-27T16:44:52-05:00","tags":["info","savedobjects-service"],"pid":873,"message":"[.kibana] OUTDATED_DOCUMENTS_SEARCH_READ -> OUTDATED_DOCUMENTS_SEARCH_CLOSE_PIT. took: 246ms."} {"type":"log","@timestamp":"2021-11-27T16:44:52-05:00","tags":["info","savedobjects-service"],"pid":873,"message":"[.kibana_task_manager] OUTDATED_DOCUMENTS_SEARCH_READ -> OUTDATED_DOCUMENTS_SEARCH_CLOSE_PIT. took: 202ms."} {"type":"log","@timestamp":"2021-11-27T16:44:52-05:00","tags":["info","savedobjects-service"],"pid":873,"message":"[.kibana_task_manager] OUTDATED_DOCUMENTS_SEARCH_CLOSE_PIT -> UPDATE_TARGET_MAPPINGS. took: 8ms."} {"type":"log","@timestamp":"2021-11-27T16:44:52-05:00","tags":["info","savedobjects-service"],"pid":873,"message":"[.kibana] OUTDATED_DOCUMENTS_SEARCH_CLOSE_PIT -> UPDATE_TARGET_MAPPINGS. took: 24ms."} {"type":"log","@timestamp":"2021-11-27T16:44:53-05:00","tags":["info","savedobjects-service"],"pid":873,"message":"[.kibana_task_manager] UPDATE_TARGET_MAPPINGS -> UPDATE_TARGET_MAPPINGS_WAIT_FOR_TASK. took: 800ms."} {"type":"log","@timestamp":"2021-11-27T16:44:53-05:00","tags":["info","savedobjects-service"],"pid":873,"message":"[.kibana] UPDATE_TARGET_MAPPINGS -> UPDATE_TARGET_MAPPINGS_WAIT_FOR_TASK. took: 796ms."} {"type":"log","@timestamp":"2021-11-27T16:44:54-05:00","tags":["info","savedobjects-service"],"pid":873,"message":"[.kibana_task_manager] UPDATE_TARGET_MAPPINGS_WAIT_FOR_TASK -> DONE. took: 1238ms."} {"type":"log","@timestamp":"2021-11-27T16:44:54-05:00","tags":["info","savedobjects-service"],"pid":873,"message":"[.kibana_task_manager] Migration completed after 2730ms"} {"type":"log","@timestamp":"2021-11-27T16:44:57-05:00","tags":["info","savedobjects-service"],"pid":873,"message":"[.kibana] UPDATE_TARGET_MAPPINGS_WAIT_FOR_TASK -> DONE. took: 3596ms."} {"type":"log","@timestamp":"2021-11-27T16:44:57-05:00","tags":["info","savedobjects-service"],"pid":873,"message":"[.kibana] Migration completed after 5097ms"} {"type":"log","@timestamp":"2021-11-27T16:44:57-05:00","tags":["info","plugins-system","standard"],"pid":873,"message":"Starting [113] plugins: [translations,licensing,globalSearch,globalSearchProviders,banners,licenseApiGuard,code,usageCollection,xpackLegacy,taskManager,telemetryCollectionManager,telemetryCollectionXpack,kibanaUsageCollection,securityOss,share,screenshotMode,telemetry,newsfeed,mapsEms,mapsLegacy,legacyExport,kibanaLegacy,embeddable,uiActionsEnhanced,fieldFormats,expressions,charts,esUiShared,bfetch,data,savedObjects,visualizations,visTypeXy,visTypeVislib,visTypeTimelion,features,visTypeTagcloud,visTypeTable,visTypePie,visTypeMetric,visTypeMarkdown,tileMap,regionMap,presentationUtil,expressionShape,expressionRevealImage,expressionRepeatImage,expressionMetric,expressionImage,timelion,indexPatternFieldEditor,home,searchprofiler,painlessLab,grokdebugger,graph,visTypeVega,management,watcher,licenseManagement,indexPatternManagement,advancedSettings,discover,discoverEnhanced,dashboard,dashboardEnhanced,visualize,visTypeTimeseries,savedObjectsManagement,spaces,security,transform,savedObjectsTagging,lens,reporting,canvas,lists,ingestPipelines,fileUpload,maps,dataVisualizer,encryptedSavedObjects,dataEnhanced,dashboardMode,cloud,snapshotRestore,fleet,indexManagement,rollup,remoteClusters,crossClusterReplication,indexLifecycleManagement,eventLog,actions,alerting,triggersActionsUi,stackAlerts,ruleRegistry,osquery,ml,cases,timelines,securitySolution,observability,uptime,infra,upgradeAssistant,monitoring,logstash,enterpriseSearch,console,apmOss,apm]"} {"type":"log","@timestamp":"2021-11-27T16:44:58-05:00","tags":["info","plugins","monitoring","monitoring"],"pid":873,"message":"config sourced from: production cluster"} {"type":"log","@timestamp":"2021-11-27T16:45:03-05:00","tags":["info","http","server","Kibana"],"pid":873,"message":"http server running at http://localhost:5601"} {"type":"log","@timestamp":"2021-11-27T16:45:04-05:00","tags":["info","plugins","monitoring","monitoring","kibana-monitoring"],"pid":873,"message":"Starting monitoring stats collection"} {"type":"log","@timestamp":"2021-11-27T16:45:05-05:00","tags":["error","plugins","eventLog"],"pid":873,"message":"error initializing elasticsearch resources: error creating initial index: invalid_alias_name_exception: [invalid_alias_name_exception] Reason: Invalid alias name [.kibana-event-log-7.15.2]: an index or data stream exists with the same name as the alias"} {"type":"log","@timestamp":"2021-11-27T16:45:05-05:00","tags":["error","plugins","eventLog"],"pid":873,"message":"initialization failed, events will not be indexed"} {"type":"log","@timestamp":"2021-11-27T16:45:05-05:00","tags":["info","plugins","securitySolution"],"pid":873,"message":"Dependent plugin setup complete - Starting ManifestTask"} {"type":"log","@timestamp":"2021-11-27T16:45:06-05:00","tags":["info","status"],"pid":873,"message":"Kibana is now degraded"} {"type":"log","@timestamp":"2021-11-27T16:45:06-05:00","tags":["info","plugins","reporting"],"pid":873,"message":"Browser executable: /usr/share/kibana/x-pack/plugins/reporting/chromium/headless_shell-linux_x64/headless_shell"} {"type":"log","@timestamp":"2021-11-27T16:45:06-05:00","tags":["warning","plugins","reporting"],"pid":873,"message":"Enabling the Chromium sandbox provides an additional layer of protection."} {"type":"log","@timestamp":"2021-11-27T16:45:10-05:00","tags":["info","status"],"pid":873,"message":"Kibana is now available (was degraded)"} {"type":"response","@timestamp":"2021-11-27T16:49:38-05:00","tags":[],"pid":873,"method":"get","statusCode":302,"req":{"url":"/","method":"get","headers":{"host":"localhost:5601","accept-encoding":"identity","content-length":"0","sec-fetch-user":"?1","accept-language":"en-US,en;q=0.9","sec-fetch-site":"same-origin","sec-fetch-mode":"navigate","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36","connection":"close","x-forwarded-proto":"https","referer":"https://192.168.1.229/rules/","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9","sec-ch-ua-platform":"\"Windows\"","sec-ch-ua-mobile":"?0","upgrade-insecure-requests":"1","sec-ch-ua":"\" Not A;Brand\";v=\"99\", \"Chromium\";v=\"96\", \"Google Chrome\";v=\"96\"","sec-fetch-dest":"document"},"remoteAddress":"127.0.0.1","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36","referer":"https://192.168.1.229/rules/"},"res":{"statusCode":302,"responseTime":37},"message":"GET / 302 37ms"} {"type":"response","@timestamp":"2021-11-27T16:49:38-05:00","tags":[],"pid":873,"method":"get","statusCode":302,"req":{"url":"/spaces/enter","method":"get","headers":{"host":"127.0.0.1:5601","connection":"close","upgrade-insecure-requests":"1","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9","sec-fetch-site":"same-origin","sec-fetch-mode":"navigate","sec-fetch-user":"?1","sec-fetch-dest":"document","sec-ch-ua":"\" Not A;Brand\";v=\"99\", \"Chromium\";v=\"96\", \"Google Chrome\";v=\"96\"","sec-ch-ua-mobile":"?0","sec-ch-ua-platform":"\"Windows\"","referer":"https://192.168.1.229/rules/","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9"},"remoteAddress":"127.0.0.1","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36","referer":"https://192.168.1.229/rules/"},"res":{"statusCode":302,"responseTime":18},"message":"GET /spaces/enter 302 18ms"} {"type":"response","@timestamp":"2021-11-27T16:49:38-05:00","tags":[],"pid":873,"method":"get","statusCode":200,"req":{"url":"/app/home","method":"get","headers":{"host":"localhost:5601","accept-encoding":"identity","content-length":"0","sec-fetch-user":"?1","accept-language":"en-US,en;q=0.9","sec-fetch-site":"same-origin","sec-fetch-mode":"navigate","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36","connection":"close","x-forwarded-proto":"https","referer":"https://192.168.1.229/rules/","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9","sec-ch-ua-platform":"\"Windows\"","sec-ch-ua-mobile":"?0","upgrade-insecure-requests":"1","sec-ch-ua":"\" Not A;Brand\";v=\"99\", \"Chromium\";v=\"96\", \"Google Chrome\";v=\"96\"","sec-fetch-dest":"document"},"remoteAddress":"127.0.0.1","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36","referer":"https://192.168.1.229/rules/"},"res":{"statusCode":200,"responseTime":86,"contentLength":144423},"message":"GET /app/home 200 86ms - 141.0KB"} {"type":"response","@timestamp":"2021-11-27T16:49:38-05:00","tags":[],"pid":873,"method":"get","statusCode":200,"req":{"url":"/node_modules/@kbn/ui-framework/dist/kui_light.css","method":"get","headers":{"host":"127.0.0.1:5601","connection":"close","sec-ch-ua":"\" Not A;Brand\";v=\"99\", \"Chromium\";v=\"96\", \"Google Chrome\";v=\"96\"","sec-ch-ua-mobile":"?0","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36","sec-ch-ua-platform":"\"Windows\"","accept":"text/css,*/*;q=0.1","sec-fetch-site":"same-origin","sec-fetch-mode":"no-cors","sec-fetch-dest":"style","referer":"https://192.168.1.229/app/home","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9"},"remoteAddress":"127.0.0.1","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36","referer":"https://192.168.1.229/app/home"},"res":{"statusCode":200,"responseTime":34,"contentLength":205},"message":"GET /node_modules/@kbn/ui-framework/dist/kui_light.css 200 34ms - 205.0B"} {"type":"response","@timestamp":"2021-11-27T16:49:38-05:00","tags":[],"pid":873,"method":"get","statusCode":200,"req":{"url":"/ui/legacy_light_theme.css","method":"get","headers":{"host":"127.0.0.1:5601","connection":"close","sec-ch-ua":"\" Not A;Brand\";v=\"99\", \"Chromium\";v=\"96\", \"Google Chrome\";v=\"96\"","sec-ch-ua-mobile":"?0","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36","sec-ch-ua-platform":"\"Windows\"","accept":"text/css,*/*;q=0.1","sec-fetch-site":"same-origin","sec-fetch-mode":"no-cors","sec-fetch-dest":"style","referer":"https://192.168.1.229/app/home","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9"},"remoteAddress":"127.0.0.1","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36","referer":"https://192.168.1.229/app/home"},"res":{"statusCode":200,"responseTime":27,"contentLength":206},"message":"GET /ui/legacy_light_theme.css 200 27ms - 206.0B"} {"type":"log","@timestamp":"2021-11-27T16:50:05-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":873,"message":"No matching indices found: logstash-rfb-*"} {"type":"log","@timestamp":"2021-11-27T16:50:05-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":873,"message":"No matching indices found: logstash-dnp3-*"} {"type":"log","@timestamp":"2021-11-27T16:50:05-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":873,"message":"No matching indices found: logstash-ikev2-*"} {"type":"log","@timestamp":"2021-11-27T16:50:05-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":873,"message":"No matching indices found: logstash-nfs-*"} {"type":"log","@timestamp":"2021-11-27T16:50:05-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":873,"message":"No matching indices found: logstash-tftp-*"} {"type":"log","@timestamp":"2021-11-27T16:50:05-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":873,"message":"No matching indices found: logstash-krb5-*"} {"type":"log","@timestamp":"2021-11-27T16:50:05-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":873,"message":"No matching indices found: logstash-mqtt-*"} {"type":"log","@timestamp":"2021-11-27T16:50:05-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":873,"message":"No matching indices found: logstash-sip-*"} {"type":"log","@timestamp":"2021-11-27T16:50:05-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":873,"message":"No matching indices found: logstash-rdp-*"} {"type":"log","@timestamp":"2021-11-27T16:50:05-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":873,"message":"No matching indices found: logstash-smb-*"} {"type":"log","@timestamp":"2021-11-27T16:50:05-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":873,"message":"No matching indices found: logstash-rfb-*"} {"type":"log","@timestamp":"2021-11-27T16:50:05-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":873,"message":"No matching indices found: logstash-dnp3-*"} {"type":"log","@timestamp":"2021-11-27T16:50:05-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":873,"message":"No matching indices found: logstash-nfs-*"} {"type":"log","@timestamp":"2021-11-27T16:50:05-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":873,"message":"No matching indices found: logstash-ikev2-*"} {"type":"log","@timestamp":"2021-11-27T16:50:05-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":873,"message":"No matching indices found: logstash-tftp-*"} {"type":"log","@timestamp":"2021-11-27T16:50:05-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":873,"message":"No matching indices found: logstash-mqtt-*"} {"type":"log","@timestamp":"2021-11-27T16:50:05-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":873,"message":"No matching indices found: logstash-rdp-*"} {"type":"log","@timestamp":"2021-11-27T16:50:05-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":873,"message":"No matching indices found: logstash-krb5-*"} {"type":"log","@timestamp":"2021-11-27T16:50:06-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":873,"message":"No matching indices found: logstash-sip-*"} {"type":"log","@timestamp":"2021-11-27T16:50:06-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":873,"message":"No matching indices found: logstash-smb-*"} tried running the first time setup..couple times, resetting data...

when i was initially having issues, i did update the system outside of the stamus upgrade script, apt-get update...not sur eif it might updated something that should not of been updated...

if i do run the update script...it seems evebox has a cert issue: `root@SELKS:~# selks-upgrade_stamus NOTE: Depending on the size and how busy the system is the upgrade may take a while. Starting the upgrade sequence...

Hit:1 http://deb.debian.org/debian buster InRelease Get:2 http://security.debian.org/debian-security buster/updates InRelease [65.4 kB] Get:3 http://deb.debian.org/debian buster-updates InRelease [51.9 kB] Hit:5 https://artifacts.elastic.co/packages/7.x/apt stable InRelease Hit:6 http://packages.stamus-networks.com/selks6/debian buster InRelease Ign:4 https://files.evebox.org/evebox/debian stable InRelease Hit:7 http://packages.stamus-networks.com/selks6/debian-kernel buster InRelease Hit:8 https://packages.elastic.co/curator/5/debian9 stable InRelease Hit:10 http://packages.stamus-networks.com/selks6/debian-test buster InRelease Err:9 https://files.evebox.org/evebox/debian stable Release Certificate verification failed: The certificate is NOT trusted. The certificate chain uses expired certificate. Could not handshake: Error in the certificate verification. [IP: 172.105.5.173 443] Reading package lists... Done E: The repository 'http://files.evebox.org/evebox/debian stable Release' no longer has a Release file. N: Updating from such a repository can't be done securely, and is therefore disabled by default. N: See apt-secure(8) manpage for repository creation and user configuration details. NOTE: Starting second stage upgrade sequence...

outputs.7.pcap-log.enabled = yes Hit:1 http://deb.debian.org/debian buster InRelease Hit:2 http://security.debian.org/debian-security buster/updates InRelease Hit:4 http://deb.debian.org/debian buster-updates InRelease Hit:5 http://packages.stamus-networks.com/selks6/debian buster InRelease Hit:6 http://packages.stamus-networks.com/selks6/debian-kernel buster InRelease Hit:7 https://artifacts.elastic.co/packages/7.x/apt stable InRelease Ign:3 https://files.evebox.org/evebox/debian stable InRelease Hit:8 http://packages.stamus-networks.com/selks6/debian-test buster InRelease Hit:9 https://packages.elastic.co/curator/5/debian9 stable InRelease Err:10 https://files.evebox.org/evebox/debian stable Release Certificate verification failed: The certificate is NOT trusted. The certificate chain uses expired certificate. Could not handshake: Error in the certificate verification. [IP: 172.105.5.173 443] Reading package lists... Done E: The repository 'http://files.evebox.org/evebox/debian stable Release' no longer has a Release file. N: Updating from such a repository can't be done securely, and is therefore disabled by default. N: See apt-secure(8) manpage for repository creation and user configuration details. scirius: stopped scirius: started `

pevma commented 2 years ago

Hi, What is the err ?

-- Regards, Peter Manev

On 27 Nov 2021, at 18:38, kdy1968 @.***> wrote:

 — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

kdy1968 commented 2 years ago

so i started from scratch...and everything is now working much better, except for the update withe the evebox cert error, dont think there is anything we can do about that..also on scirius, under visulations, Packets City (webGL), is there a way to get that functioning

pevma commented 2 years ago

I think after the first time setup completes you should upgrade by using the selks-upgrade-stamus script.

-- Regards, Peter Manev

On 28 Nov 2021, at 03:32, kdy1968 @.***> wrote:

 so i started from scratch...and everything is now working much better, except for the update withe the evebox cert error, dont think there is anything we can do about that..also on scirius, under visulations, Packets City (webGL), is there a way to get that functioning

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

kdy1968 commented 2 years ago

couple of notes, this is a clean install from ISO, selks-upgrade_stamus as is isnt working..its getting errors and not updating

with release of bullseye(i believe), this error:

E: Repository 'http://deb.debian.org/debian buster-updates InRelease' changed its 'Suite' value from 'stable-updates' to 'oldstable-updates'

fixed by running apt-get --allow-releaseinfo-change update

Err:8 https://files.evebox.org/evebox/debian stable Release Certificate verification failed: The certificate is NOT trusted. The certificate chain uses expired certificate. Could not handshake: Error in the certificate verification. [IP: 172.105.5.173 443]

fixed by using source listed here "https://github.com/jasonish/evebox/wiki/EveBox-Debian-Repository" changed /etc/apt/sources.list.d/evebox.list

from: deb http://files.evebox.org/evebox/debian stable main to:deb http://evebox.org/files/debian stable main

also added the gpg key listed at the above url...

after those changes the update run successful and pulled down updates there is still one error after updates are applied:

done. done. scirius: stopped scirius: started dpkg: error: --compare-versions takes three arguments:

Type dpkg --help for help about installing and deinstalling packages []; Use 'apt' or 'aptitude' for user-friendly package management; Type dpkg -Dhelp for a list of dpkg debug flag values; Type dpkg --force-help for a list of forcing options; Type dpkg-deb --help for help about manipulating .deb files;

........ and now i am back to my original issue when kibana is opened it opens to a blank page

in kibana log seeing the following: {"type":"log","@timestamp":"2021-11-28T09:44:20-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-krb5-"} {"type":"log","@timestamp":"2021-11-28T09:44:20-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-dnp3-"} {"type":"log","@timestamp":"2021-11-28T09:44:20-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-ikev2-"} {"type":"log","@timestamp":"2021-11-28T09:44:20-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-rfb-"} {"type":"log","@timestamp":"2021-11-28T09:44:20-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-nfs-"} {"type":"log","@timestamp":"2021-11-28T09:44:20-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-smb-"} {"type":"log","@timestamp":"2021-11-28T09:44:20-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-tftp-"} {"type":"log","@timestamp":"2021-11-28T09:44:20-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-sip-"} {"type":"log","@timestamp":"2021-11-28T09:44:20-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-rdp-"} {"type":"log","@timestamp":"2021-11-28T09:44:20-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-krb5-"} {"type":"log","@timestamp":"2021-11-28T09:44:20-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-dnp3-"} {"type":"log","@timestamp":"2021-11-28T09:44:20-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-ikev2-"} {"type":"log","@timestamp":"2021-11-28T09:44:20-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-smb-"} {"type":"log","@timestamp":"2021-11-28T09:44:20-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-sip-"} {"type":"log","@timestamp":"2021-11-28T09:44:21-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-tftp-"} {"type":"log","@timestamp":"2021-11-28T09:44:21-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-rdp-"} {"type":"log","@timestamp":"2021-11-28T09:44:21-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-rfb-"} {"type":"log","@timestamp":"2021-11-28T09:44:21-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-nfs-"}

output from healthcheck:

`root@mars:~# selks-health-check_stamus ● suricata.service - LSB: Next Generation IDS/IPS Loaded: loaded (/etc/init.d/suricata; generated) Active: active (running) since Sun 2021-11-28 09:38:12 EST; 14min ago Docs: man:systemd-sysv-generator(8) Process: 869 ExecStart=/etc/init.d/suricata start (code=exited, status=0/SUCCESS) Tasks: 14 (limit: 4915) Memory: 436.5M CGroup: /system.slice/suricata.service └─944 /usr/bin/suricata -c /etc/suricata/suricata.yaml --pidfile /var/run/suricata.pid --af-packet -D -v --us…

Nov 28 09:38:12 mars systemd[1]: Starting LSB: Next Generation IDS/IPS... Nov 28 09:38:12 mars suricata[869]: Starting suricata in IDS (af-packet) mode... done. Nov 28 09:38:12 mars systemd[1]: Started LSB: Next Generation IDS/IPS. ● elasticsearch.service - Elasticsearch Loaded: loaded (/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled) Active: active (running) since Sun 2021-11-28 09:39:04 EST; 13min ago Docs: https://www.elastic.co Main PID: 871 (java) Tasks: 87 (limit: 4915) Memory: 8.7G CGroup: /system.slice/elasticsearch.service ├─ 871 /usr/share/elasticsearch/jdk/bin/java -Xshare:auto -Des.networkaddress.cache.ttl=60 -Des.networkaddres… └─1538 /usr/share/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/controller

Nov 28 09:38:12 mars systemd[1]: Starting Elasticsearch... Nov 28 09:38:20 mars systemd-entrypoint[871]: WARNING: A terminally deprecated method in java.lang.System has been called Nov 28 09:38:20 mars systemd-entrypoint[871]: WARNING: System::setSecurityManager has been called by org.elastics….2.jar) Nov 28 09:38:20 mars systemd-entrypoint[871]: WARNING: Please consider reporting this to the maintainers of org.e…csearch Nov 28 09:38:20 mars systemd-entrypoint[871]: WARNING: System::setSecurityManager will be removed in a future release Nov 28 09:38:25 mars systemd-entrypoint[871]: WARNING: A terminally deprecated method in java.lang.System has been called Nov 28 09:38:25 mars systemd-entrypoint[871]: WARNING: System::setSecurityManager has been called by org.elastics….2.jar) Nov 28 09:38:25 mars systemd-entrypoint[871]: WARNING: Please consider reporting this to the maintainers of org.e…ecurity Nov 28 09:38:25 mars systemd-entrypoint[871]: WARNING: System::setSecurityManager will be removed in a future release Nov 28 09:39:04 mars systemd[1]: Started Elasticsearch. Hint: Some lines were ellipsized, use -l to show in full. ● logstash.service - logstash Loaded: loaded (/etc/systemd/system/logstash.service; enabled; vendor preset: enabled) Active: active (running) since Sun 2021-11-28 09:37:10 EST; 15min ago Main PID: 415 (java) Tasks: 44 (limit: 4915) Memory: 1.2G CGroup: /system.slice/logstash.service └─415 /usr/share/logstash/jdk/bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFracti…

Nov 28 09:39:10 mars logstash[415]: [2021-11-28T09:39:10,113][WARN ][logstash.outputs.elasticsearch][main] Resto…:9200/"} Nov 28 09:39:10 mars logstash[415]: [2021-11-28T09:39:10,146][WARN ][logstash.outputs.elasticsearch][main] Resto…:9200/"} Nov 28 09:39:10 mars logstash[415]: [2021-11-28T09:39:10,871][INFO ][logstash.outputs.elasticsearch][main] Elast…sion=>7} Nov 28 09:39:10 mars logstash[415]: [2021-11-28T09:39:10,881][INFO ][logstash.outputs.elasticsearch][main] Elast…sion=>7} Nov 28 09:39:10 mars logstash[415]: [2021-11-28T09:39:10,891][WARN ][logstash.outputs.elasticsearch][main] Detec…sion=>7} Nov 28 09:39:10 mars logstash[415]: [2021-11-28T09:39:10,912][WARN ][logstash.outputs.elasticsearch][main] Detec…sion=>7} Nov 28 09:39:54 mars logstash[415]: [2021-11-28T09:39:54,499][INFO ][logstash.outputs.elasticsearch][main] Using…e.json"} Nov 28 09:39:54 mars logstash[415]: [2021-11-28T09:39:54,520][INFO ][logstash.outputs.elasticsearch][main] Using…e.json"} Nov 28 09:39:54 mars logstash[415]: [2021-11-28T09:39:54,547][INFO ][logstash.outputs.elasticsearch][main] Insta…gstash"} Nov 28 09:39:54 mars logstash[415]: [2021-11-28T09:39:54,551][INFO ][logstash.outputs.elasticsearch][main] Insta…gstash"} Hint: Some lines were ellipsized, use -l to show in full. ● kibana.service - Kibana Loaded: loaded (/etc/systemd/system/kibana.service; enabled; vendor preset: enabled) Active: active (running) since Sun 2021-11-28 09:38:12 EST; 14min ago Docs: https://www.elastic.co Main PID: 875 (node) Tasks: 11 (limit: 4915) Memory: 693.2M CGroup: /system.slice/kibana.service └─875 /usr/share/kibana/bin/../node/bin/node /usr/share/kibana/bin/../src/cli/dist --logging.dest=/var/log/ki…

Nov 28 09:38:12 mars systemd[1]: Started Kibana. ● evebox.service - EveBox Server Loaded: loaded (/lib/systemd/system/evebox.service; enabled; vendor preset: enabled) Active: active (running) since Sun 2021-11-28 09:37:10 EST; 15min ago Main PID: 418 (evebox) Tasks: 5 (limit: 4915) Memory: 32.6M CGroup: /system.slice/evebox.service └─418 /usr/bin/evebox server

Nov 28 09:38:46 mars evebox[418]: 2021-11-28 09:38:46 WARN evebox::server::main: Failed to get Elasticsearch v…rror 111) Nov 28 09:38:49 mars evebox[418]: 2021-11-28 09:38:49 WARN evebox::server::main: Failed to get Elasticsearch v…rror 111) Nov 28 09:38:52 mars evebox[418]: 2021-11-28 09:38:52 WARN evebox::server::main: Failed to get Elasticsearch v…rror 111) Nov 28 09:38:55 mars evebox[418]: 2021-11-28 09:38:55 WARN evebox::server::main: Failed to get Elasticsearch v…rror 111) Nov 28 09:38:58 mars evebox[418]: 2021-11-28 09:38:58 WARN evebox::server::main: Failed to get Elasticsearch v…rror 111) Nov 28 09:39:01 mars evebox[418]: 2021-11-28 09:39:01 WARN evebox::server::main: Failed to get Elasticsearch v…rror 111) Nov 28 09:39:04 mars evebox[418]: 2021-11-28 09:39:04 WARN evebox::server::main: Failed to get Elasticsearch v…rror 111) Nov 28 09:39:07 mars evebox[418]: 2021-11-28 09:39:07 INFO evebox::server::main: Found Elasticsearch version 7…host:9200 Nov 28 09:39:07 mars evebox[418]: 2021-11-28 09:39:07 INFO evebox::server::main: Starting server on 127.0.0.1:…tls=false Nov 28 09:47:48 mars evebox[418]: 2021-11-28 09:47:48 INFO evebox::server::main: Creating anonymous session fo…elks-user Hint: Some lines were ellipsized, use -l to show in full. Unit molochviewer-selks.service could not be found. Unit molochpcapread-selks.service could not be found. scirius RUNNING pid 1246, uptime 0:14:35 ii elasticsearch 7.15.2 amd64 Distributed RESTful search engine bu ilt for the cloud ii elasticsearch-curator 5.8.4 amd64 Have indices in Elasticsearch? This is the tool for you!\n\nLike a museum curator manages the exhibits and collections on display, \nElasticsearch Curator he lps you curate, or manage your indices. ii evebox 1:0.14.0 amd64 no description given ii kibana 7.15.2 amd64 Explore and visualize your Elasticse arch data ii kibana-dashboards-stamus 2020122001 amd64 Kibana 6 dashboard templates. ii logstash 1:7.15.2-1 amd64 An extensible logging pipeline ii moloch 2.2.3-1 amd64 Moloch Full Packet System ii scirius 3.5.0-3 amd64 Django application to manage Suricat a ruleset ii suricata 1:2021111201-0stamus0 amd64 Suricata open source multi-thread ID S/IPS/NSM system. Filesystem Type Size Used Avail Use% Mounted on udev devtmpfs 7.8G 0 7.8G 0% /dev tmpfs tmpfs 1.6G 8.9M 1.6G 1% /run /dev/sda1 ext4 94G 8.4G 81G 10% / tmpfs tmpfs 7.8G 136K 7.8G 1% /dev/shm tmpfs tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs tmpfs 7.8G 0 7.8G 0% /sys/fs/cgroup tmpfs tmpfs 1.6G 0 1.6G 0% /run/user/0 `

let me know if you have any suggestions

pevma commented 2 years ago

Hi,

Ok. I just want to check the basics so we don’t miss something easy: After download and install did you go through the steps here (first time setup) - https://github.com/StamusNetworks/SELKS/wiki/First-time-setup

Only after those are successfully completed , you can run the upgrade command(without any distro changes etc)

We’re those the steps you followed?

-- Regards, Peter Manev

On 28 Nov 2021, at 15:55, kdy1968 @.***> wrote:

 couple of notes, this is a clean install from ISO, selks-upgrade_stamus as is isnt working..its getting errors and not updating

with release of bullseye(i believe), this error:

E: Repository 'http://deb.debian.org/debian buster-updates InRelease' changed its 'Suite' value from 'stable-updates' to 'oldstable-updates'

fixed by running apt-get --allow-releaseinfo-change update

Err:8 https://files.evebox.org/evebox/debian stable Release Certificate verification failed: The certificate is NOT trusted. The certificate chain uses expired certificate. Could not handshake: Error in the certificate verification. [IP: 172.105.5.173 443]

fixed by using source listed here "https://github.com/jasonish/evebox/wiki/EveBox-Debian-Repository" changed /etc/apt/sources.list.d/evebox.list

from: deb http://files.evebox.org/evebox/debian stable main to:deb http://evebox.org/files/debian stable main

also added the gpg key listed at the above url...

after those changes the update run successful and pulled down updates there is still one error after updates are applied:

done. done. scirius: stopped scirius: started dpkg: error: --compare-versions takes three arguments:

Type dpkg --help for help about installing and deinstalling packages []; Use 'apt' or 'aptitude' for user-friendly package management; Type dpkg -Dhelp for a list of dpkg debug flag values; Type dpkg --force-help for a list of forcing options; Type dpkg-deb --help for help about manipulating .deb files;

........ and now i am back to my original issue when kibana is opened it opens to a blank page

in kibana log seeing the following: @.":"2021-11-28T09:44:20-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-krb5-"} @.":"2021-11-28T09:44:20-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-dnp3-"} @.":"2021-11-28T09:44:20-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-ikev2-"} @.":"2021-11-28T09:44:20-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-rfb-"} @.":"2021-11-28T09:44:20-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-nfs-"} @.":"2021-11-28T09:44:20-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-smb-"} @.":"2021-11-28T09:44:20-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-tftp-"} @.":"2021-11-28T09:44:20-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-sip-"} @.":"2021-11-28T09:44:20-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-rdp-"} @.":"2021-11-28T09:44:20-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-krb5-"} @.":"2021-11-28T09:44:20-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-dnp3-"} @.":"2021-11-28T09:44:20-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-ikev2-"} @.":"2021-11-28T09:44:20-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-smb-"} @.":"2021-11-28T09:44:20-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-sip-"} @.":"2021-11-28T09:44:21-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-tftp-"} @.":"2021-11-28T09:44:21-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-rdp-"} @.":"2021-11-28T09:44:21-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-rfb-"} @.":"2021-11-28T09:44:21-05:00","tags":["warning","plugins","data","data","indexPatterns"],"pid":875,"message":"No matching indices found: logstash-nfs-"}

output from healthcheck:

@.***:~# selks-health-check_stamus ● suricata.service - LSB: Next Generation IDS/IPS Loaded: loaded (/etc/init.d/suricata; generated) Active: active (running) since Sun 2021-11-28 09:38:12 EST; 14min ago Docs: man:systemd-sysv-generator(8) Process: 869 ExecStart=/etc/init.d/suricata start (code=exited, status=0/SUCCESS) Tasks: 14 (limit: 4915) Memory: 436.5M CGroup: /system.slice/suricata.service └─944 /usr/bin/suricata -c /etc/suricata/suricata.yaml --pidfile /var/run/suricata.pid --af-packet -D -v --us…

Nov 28 09:38:12 mars systemd[1]: Starting LSB: Next Generation IDS/IPS... Nov 28 09:38:12 mars suricata[869]: Starting suricata in IDS (af-packet) mode... done. Nov 28 09:38:12 mars systemd[1]: Started LSB: Next Generation IDS/IPS. ● elasticsearch.service - Elasticsearch Loaded: loaded (/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled) Active: active (running) since Sun 2021-11-28 09:39:04 EST; 13min ago Docs: https://www.elastic.co Main PID: 871 (java) Tasks: 87 (limit: 4915) Memory: 8.7G CGroup: /system.slice/elasticsearch.service ├─ 871 /usr/share/elasticsearch/jdk/bin/java -Xshare:auto -Des.networkaddress.cache.ttl=60 -Des.networkaddres… └─1538 /usr/share/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/controller

Nov 28 09:38:12 mars systemd[1]: Starting Elasticsearch... Nov 28 09:38:20 mars systemd-entrypoint[871]: WARNING: A terminally deprecated method in java.lang.System has been called Nov 28 09:38:20 mars systemd-entrypoint[871]: WARNING: System::setSecurityManager has been called by org.elastics….2.jar) Nov 28 09:38:20 mars systemd-entrypoint[871]: WARNING: Please consider reporting this to the maintainers of org.e…csearch Nov 28 09:38:20 mars systemd-entrypoint[871]: WARNING: System::setSecurityManager will be removed in a future release Nov 28 09:38:25 mars systemd-entrypoint[871]: WARNING: A terminally deprecated method in java.lang.System has been called Nov 28 09:38:25 mars systemd-entrypoint[871]: WARNING: System::setSecurityManager has been called by org.elastics….2.jar) Nov 28 09:38:25 mars systemd-entrypoint[871]: WARNING: Please consider reporting this to the maintainers of org.e…ecurity Nov 28 09:38:25 mars systemd-entrypoint[871]: WARNING: System::setSecurityManager will be removed in a future release Nov 28 09:39:04 mars systemd[1]: Started Elasticsearch. Hint: Some lines were ellipsized, use -l to show in full. ● logstash.service - logstash Loaded: loaded (/etc/systemd/system/logstash.service; enabled; vendor preset: enabled) Active: active (running) since Sun 2021-11-28 09:37:10 EST; 15min ago Main PID: 415 (java) Tasks: 44 (limit: 4915) Memory: 1.2G CGroup: /system.slice/logstash.service └─415 /usr/share/logstash/jdk/bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFracti…

Nov 28 09:39:10 mars logstash[415]: [2021-11-28T09:39:10,113][WARN ][logstash.outputs.elasticsearch][main] Resto…:9200/"} Nov 28 09:39:10 mars logstash[415]: [2021-11-28T09:39:10,146][WARN ][logstash.outputs.elasticsearch][main] Resto…:9200/"} Nov 28 09:39:10 mars logstash[415]: [2021-11-28T09:39:10,871][INFO ][logstash.outputs.elasticsearch][main] Elast…sion=>7} Nov 28 09:39:10 mars logstash[415]: [2021-11-28T09:39:10,881][INFO ][logstash.outputs.elasticsearch][main] Elast…sion=>7} Nov 28 09:39:10 mars logstash[415]: [2021-11-28T09:39:10,891][WARN ][logstash.outputs.elasticsearch][main] Detec…sion=>7} Nov 28 09:39:10 mars logstash[415]: [2021-11-28T09:39:10,912][WARN ][logstash.outputs.elasticsearch][main] Detec…sion=>7} Nov 28 09:39:54 mars logstash[415]: [2021-11-28T09:39:54,499][INFO ][logstash.outputs.elasticsearch][main] Using…e.json"} Nov 28 09:39:54 mars logstash[415]: [2021-11-28T09:39:54,520][INFO ][logstash.outputs.elasticsearch][main] Using…e.json"} Nov 28 09:39:54 mars logstash[415]: [2021-11-28T09:39:54,547][INFO ][logstash.outputs.elasticsearch][main] Insta…gstash"} Nov 28 09:39:54 mars logstash[415]: [2021-11-28T09:39:54,551][INFO ][logstash.outputs.elasticsearch][main] Insta…gstash"} Hint: Some lines were ellipsized, use -l to show in full. ● kibana.service - Kibana Loaded: loaded (/etc/systemd/system/kibana.service; enabled; vendor preset: enabled) Active: active (running) since Sun 2021-11-28 09:38:12 EST; 14min ago Docs: https://www.elastic.co Main PID: 875 (node) Tasks: 11 (limit: 4915) Memory: 693.2M CGroup: /system.slice/kibana.service └─875 /usr/share/kibana/bin/../node/bin/node /usr/share/kibana/bin/../src/cli/dist --logging.dest=/var/log/ki…

Nov 28 09:38:12 mars systemd[1]: Started Kibana. ● evebox.service - EveBox Server Loaded: loaded (/lib/systemd/system/evebox.service; enabled; vendor preset: enabled) Active: active (running) since Sun 2021-11-28 09:37:10 EST; 15min ago Main PID: 418 (evebox) Tasks: 5 (limit: 4915) Memory: 32.6M CGroup: /system.slice/evebox.service └─418 /usr/bin/evebox server

Nov 28 09:38:46 mars evebox[418]: 2021-11-28 09:38:46 WARN evebox::server::main: Failed to get Elasticsearch v…rror 111) Nov 28 09:38:49 mars evebox[418]: 2021-11-28 09:38:49 WARN evebox::server::main: Failed to get Elasticsearch v…rror 111) Nov 28 09:38:52 mars evebox[418]: 2021-11-28 09:38:52 WARN evebox::server::main: Failed to get Elasticsearch v…rror 111) Nov 28 09:38:55 mars evebox[418]: 2021-11-28 09:38:55 WARN evebox::server::main: Failed to get Elasticsearch v…rror 111) Nov 28 09:38:58 mars evebox[418]: 2021-11-28 09:38:58 WARN evebox::server::main: Failed to get Elasticsearch v…rror 111) Nov 28 09:39:01 mars evebox[418]: 2021-11-28 09:39:01 WARN evebox::server::main: Failed to get Elasticsearch v…rror 111) Nov 28 09:39:04 mars evebox[418]: 2021-11-28 09:39:04 WARN evebox::server::main: Failed to get Elasticsearch v…rror 111) Nov 28 09:39:07 mars evebox[418]: 2021-11-28 09:39:07 INFO evebox::server::main: Found Elasticsearch version 7…host:9200 Nov 28 09:39:07 mars evebox[418]: 2021-11-28 09:39:07 INFO evebox::server::main: Starting server on 127.0.0.1:…tls=false Nov 28 09:47:48 mars evebox[418]: 2021-11-28 09:47:48 INFO evebox::server::main: Creating anonymous session fo…elks-user Hint: Some lines were ellipsized, use -l to show in full. Unit molochviewer-selks.service could not be found. Unit molochpcapread-selks.service could not be found. scirius RUNNING pid 1246, uptime 0:14:35 ii elasticsearch 7.15.2 amd64 Distributed RESTful search engine bu ilt for the cloud ii elasticsearch-curator 5.8.4 amd64 Have indices in Elasticsearch? This is the tool for you!\n\nLike a museum curator manages the exhibits and collections on display, \nElasticsearch Curator he lps you curate, or manage your indices. ii evebox 1:0.14.0 amd64 no description given ii kibana 7.15.2 amd64 Explore and visualize your Elasticse arch data ii kibana-dashboards-stamus 2020122001 amd64 Kibana 6 dashboard templates. ii logstash 1:7.15.2-1 amd64 An extensible logging pipeline ii moloch 2.2.3-1 amd64 Moloch Full Packet System ii scirius 3.5.0-3 amd64 Django application to manage Suricat a ruleset ii suricata 1:2021111201-0stamus0 amd64 Suricata open source multi-thread ID S/IPS/NSM system. Filesystem Type Size Used Avail Use% Mounted on udev devtmpfs 7.8G 0 7.8G 0% /dev tmpfs tmpfs 1.6G 8.9M 1.6G 1% /run /dev/sda1 ext4 94G 8.4G 81G 10% / tmpfs tmpfs 7.8G 136K 7.8G 1% /dev/shm tmpfs tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs tmpfs 7.8G 0 7.8G 0% /sys/fs/cgroup tmpfs tmpfs 1.6G 0 1.6G 0% /run/user/0 `

let me know if you have any suggestions

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

kdy1968 commented 2 years ago

on the reinstall i was sure to follow the steps.. however after running the "selks-first-time-setup_stamus" script, when i ran the selks-upgrade_stamus it would fail with afore mentioned errors, no apt updates were done....(this happened on both installs i did)...but everything was working, but my concern with missing updates led me to dig into the selks-upgrade_stamus and determine what going on, noted in previous post ......

appears the issue with kibana not opening was something to do with the nginx config, there was another issue that had a link to a nginx selks conf file, using that kibanna is now opening.....................

also i reran the update script and didnt get the,,, dpkg: error, oddly it tried to setup Moloch, i selected none on packet capture during first time setup, assume maybe it just saw he package and upgraded it : `root@mars:~# selks-upgrade_stamus NOTE: Depending on the size and how busy the system is the upgrade may take a while. Starting the upgrade sequence...

Get:1 http://deb.debian.org/debian buster-updates InRelease [51.9 kB] Get:2 http://security.debian.org/debian-security buster/updates InRelease [65.4 kB] Hit:3 http://evebox.org/files/debian stable InRelease Hit:4 https://artifacts.elastic.co/packages/7.x/apt stable InRelease Hit:5 http://packages.stamus-networks.com/selks6/debian buster InRelease Hit:6 https://packages.elastic.co/curator/5/debian9 stable InRelease Hit:7 http://packages.stamus-networks.com/selks6/debian-kernel buster InRelease Hit:8 http://packages.stamus-networks.com/selks6/debian-test buster InRelease Get:9 http://security.debian.org/debian-security buster/updates/main Sources [203 kB] Get:10 http://security.debian.org/debian-security buster/updates/main amd64 Packages [309 kB] Fetched 629 kB in 2s (330 kB/s) Reading package lists... Done Reading package lists... Done Building dependency tree Reading state information... Done selks-scripts-stamus is already the newest version (2020121401). The following package was automatically installed and is no longer required: liblua5.3-0 Use 'apt autoremove' to remove it. 0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded. NOTE: Starting second stage upgrade sequence...

Hit:1 http://deb.debian.org/debian buster-updates InRelease Hit:2 http://security.debian.org/debian-security buster/updates InRelease Hit:3 http://evebox.org/files/debian stable InRelease Hit:4 http://packages.stamus-networks.com/selks6/debian buster InRelease Hit:5 http://packages.stamus-networks.com/selks6/debian-kernel buster InRelease Hit:6 https://packages.elastic.co/curator/5/debian9 stable InRelease Hit:7 https://artifacts.elastic.co/packages/7.x/apt stable InRelease Hit:8 http://packages.stamus-networks.com/selks6/debian-test buster InRelease Reading package lists... Done Reading package lists... Done Building dependency tree Reading state information... Done Calculating upgrade... Done The following package was automatically installed and is no longer required: liblua5.3-0 Use 'apt autoremove' to remove it. The following packages will be upgraded: libicu63 1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. Need to get 8,300 kB of archives. After this operation, 0 B of additional disk space will be used. Get:1 http://security.debian.org/debian-security buster/updates/main amd64 libicu63 amd64 63.1-6+deb10u2 [8,300 kB] Fetched 8,300 kB in 1s (14.6 MB/s) (Reading database ... 260974 files and directories currently installed.) Preparing to unpack .../libicu63_63.1-6+deb10u2_amd64.deb ... Unpacking libicu63:amd64 (63.1-6+deb10u2) over (63.1-6+deb10u1) ... Setting up libicu63:amd64 (63.1-6+deb10u2) ... Processing triggers for libc-bin (2.28-10) ... scirius: stopped scirius: started

Upgrading Moloch..

{"cluster_name":"elasticsearch","status":"green","timed_out":false,"number_of_nodes":1,"number_of_data_nodes":1,"active_primary_shards":24,"active_shards":24,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"delayed_un(Reading database ... 260974 files and directories currently installed.)h":0,"task_max_waiting_in_queue_millis":0,"active_shards_percent_as_number":100.0}(Reading database ... Preparing to unpack .../moloch_3.1.1-1_amd64.deb ... Unpacking moloch (3.1.1-1) over (2.2.3-1) ... Setting up moloch (3.1.1-1) ... It is STRONGLY recommended that you stop ALL Arkime captures and viewers before proceeding. Use 'db.pl http://localhost:9200 backup' to backup db first.

There is 1 elastic search data node, if you expect more please fix first before proceeding.

This is a fresh Arkime install Looks like Arkime wasn't installed, must do init at /data/moloch/db/db.pl line 5721.

Starting Moloch SELKS services..

Failed to start molochpcapread-selks.service: Unit molochpcapread-selks.service not found. Failed to start molochviewer-selks.service: Unit molochviewer-selks.service not found. `

t**he last issue i have under "logstash insertion speed" it sates states no data for perio

kdy1968 commented 2 years ago

...still trying to figuree out why i have no logstash stats...under "logstash insertion speed"

i have another question.. if i look at a alert for my geoip rule,: drop ip any any -> any any (msg:"GeoIP countrydrop"; geoip:any,IR,CN,RU; threshold: type limit, track by_src, seconds 60, count 1; classtype:misc-attack; sid:55555557; rev:1;)

i access a ru site the alert looks like the following, this alert seems to be a response from the RU web server:

Capture

it shows the geo information for the destination, any way to get it to show the src ip geo info, as well?

pevma commented 2 years ago

Think that should be done inside the /etc/logstash/conf.d/logstash.conf

kdy1968 commented 2 years ago

i did verify the geoip section in the logstash.conf......what i determine was if the selks machine was between router and switch it would would respond the way i expected it show the source or destination geoip info ...depending which was external....however if the selks machine was between my modem and router it seemed abit confused and would show the geoip info for my router...i did try adjusting the $home_net variable, there may something more to understanding the the flows and such, for now i am just going to let it live on the lan......

regarding: image what stats should be populated here, should there be indexes for those?

below is a list of indices i have;

`root@mars:~# curl -XGET http://localhost:9200/_cat/indices green open logstash-smb-2021.11.29 SWOMxmgdQmezd235Iw7JVQ 1 0 1 0 22.2kb 22.2kb green open logstash-ike-2021.11.29 8uIbcijZSIyxKiv8Bv71-A 1 0 2 0 25.6kb 25.6kb green open logstash-sip-2021.11.29 LQOwfSaSQJGFrEU2oMIKFw 1 0 6 0 123.3kb 123.3kb green open logstash-ssh-2021.11.29 4Y5E6qEOQ_SdaXawENJ4OA 1 0 5 0 69.5kb 69.5kb green open logstash-ssh-2021.11.28 IMVfeVl-Toq5P4ziBSQHfg 1 0 13 0 62.3kb 62.3kb green open .kibana-event-log-7.15.2-000001 lQf9RqSpSMudV97Id8kEVQ 1 0 14 0 15.5kb 15.5kb green open logstash-2021.11.28 4QquCmvVTdmmPKls0AwBOQ 1 0 15178 0 51.7mb 51.7mb green open .kibana_task_manager_7.15.2_001 bR0a-CCTQByElSn9P37Hgw 1 0 16 118379 16.9mb 16.9mb green open logstash-alert-2021.11.29 agt3DzL1Qki5XyPqZFB76Q 1 0 1343 0 2.8mb 2.8mb green open logstash-alert-2021.11.28 OGPK6iJdT-SK7RYetiPj-A 1 0 841 0 1.7mb 1.7mb green open .apm-custom-link ioD_VCALSYOAhKULvFGEdQ 1 0 0 0 208b 208b green open .kibana_task_manager_1 hrUDRmBzQbSECDZ9nDuG6A 1 0 5 1 19.8kb 19.8kb green open logstash-smb-2021.12.02 kgQlYfaxRbWOXFgHCjMMpQ 1 0 1 0 22.9kb 22.9kb green open .kibana_7.15.2_001 oW3AgfvPSqenRCSMFdH_iw 1 0 1188 29 4.4mb 4.4mb green open logstash-smb-2021.11.30 mBz2icrSRWeLep3B4nkpEw 1 0 1 0 22.6kb 22.6kb green open logstash-snmp-2021.11.29 rk-Iwz9ZRHmQ3iPoEtDmVw 1 0 861 0 551.9kb 551.9kb green open logstash-snmp-2021.11.28 bKnyHlXASsOKKX26D9X6_g 1 0 2325 0 809.9kb 809.9kb green open logstash-alert-2021.11.30 _qn3zF_lQzO5khiEcmhwtw 1 0 1586 0 3.1mb 3.1mb green open .apm-agent-configuration f4zQVjvASnafMsuPMzWZeg 1 0 0 0 208b 208b green open logstash-alert-2021.12.02 7SdspAjeQKu0j1gCPc4kFQ 1 0 793 0 1.9mb 1.9mb green open logstash-alert-2021.12.01 LZwRn9wiQFOai2_ztJIPRQ 1 0 1480 0 3mb 3mb green open logstash-http-2021.11.30 ZxCz8yNMTw6bfN1KVRJnng 1 0 153618 0 46mb 46mb green open .tasks TaDdkwwuQSSVlKf09IwMLA 1 0 18 0 51kb 51kb green open logstash-http-2021.12.01 eCTLZ8_KQamnWc52qHh0Vw 1 0 161070 0 49.5mb 49.5mb green open logstash-http-2021.12.02 rjW__qZBSxuiT0QgzgQnhA 1 0 84908 0 29.3mb 29.3mb green open logstash-fileinfo-2021.11.29 EUR55TwLSnacK-asIr4yqg 1 0 133868 0 47.1mb 47.1mb green open logstash-anomaly-2021.11.28 _E-6k8RvRoeRt23xDp4MBg 1 0 59 0 238.2kb 238.2kb green open logstash-fileinfo-2021.11.28 h6HZo5-0T9mL7gxShIkmEg 1 0 143284 0 45.4mb 45.4mb green open logstash-http-2021.11.29 09XGdPbFSNa8T2DZNrg9lA 1 0 132137 0 41.4mb 41.4mb green open logstash-anomaly-2021.11.29 AWmzhKwgQ563aeChF8O2Mg 1 0 23 0 101.3kb 101.3kb green open logstash-http-2021.11.28 c5ouT-NDSs2hi8nK0vx8-w 1 0 141296 0 46.1mb 46.1mb green open logstash-snmp-2021.12.02 1xjmJH_fTPGrJmGXI3TznQ 1 0 124 0 151.5kb 151.5kb green open logstash-snmp-2021.12.01 ZZaXYWsERui82p-cOqGqTA 1 0 508 0 368.2kb 368.2kb green open logstash-dns-2021.12.02 xY1z8IUNTEug4PeFEN1crQ 1 0 15092 0 5.8mb 5.8mb green open logstash-dhcp-2021.11.28 ZoiSpuJDQIiWw6p5D4iJNQ 1 0 567 0 414.5kb 414.5kb green open logstash-dns-2021.12.01 iGSt3TGNSmWzLrMcZPxXPQ 1 0 34272 0 12.1mb 12.1mb green open logstash-flow-2021.11.30 qwcmGE0NTe6q5ml9tAfqTQ 1 0 455606 0 130.1mb 130.1mb green open logstash-dhcp-2021.11.29 a62nibF0TWml8kOMC5BH2g 1 0 263 0 326.1kb 326.1kb green open logstash-dns-2021.11.30 7ZcfCW-CTciVbMNJrjaimg 1 0 52141 0 19.4mb 19.4mb green open .geoip_databases G7dTak7FTVqeWI6ICUbe3Q 1 0 42 0 40.9mb 40.9mb green open logstash-flow-2021.11.28 H-Z31L17Qb2EOuB9g88ukA 1 0 407494 0 122.6mb 122.6mb green open logstash-dns-2021.11.29 v1d3RKz5RJu7wGyTpzJpow 1 0 53616 0 18.8mb 18.8mb green open logstash-flow-2021.11.29 HTLeyBI_RRiiyRhz_0J1LA 1 0 415361 0 117.4mb 117.4mb green open logstash-smtp-2021.11.28 yI71AvfpQ2GJOZY5Y_5Drg 1 0 24 0 194.8kb 194.8kb green open logstash-dhcp-2021.11.30 v0kIYbFmTgKlUBsWGQZiqQ 1 0 234 0 357.1kb 357.1kb green open logstash-smtp-2021.11.29 kzMkUq_YRr634sKyzCckng 1 0 15 0 192.3kb 192.3kb green open logstash-dhcp-2021.12.01 CdpJFNFrTau0lQz3w9dXKg 1 0 296 0 451.5kb 451.5kb green open logstash-dns-2021.11.28 rr6PxpGORKOw6Z4RrlbfRg 1 0 35174 0 12.6mb 12.6mb green open logstash-dhcp-2021.12.02 Y-Hj_wPtTE6Sz5L1PsyK4w 1 0 106 0 272.9kb 272.9kb green open logstash-snmp-2021.11.30 0XLRninETx2BO2_xHOR0CA 1 0 1668 0 982.9kb 982.9kb green open logstash-tls-2021.11.28 pctn5dFgRVC2TuRDWqcpeQ 1 0 13799 0 10.4mb 10.4mb green open logstash-tls-2021.11.29 SdUuSxT9RgScRaq9n0gp0Q 1 0 17631 0 12.2mb 12.2mb green open logstash-tls-2021.12.02 AMbHaSDBRxKB5GEoDht-cw 1 0 6285 0 5.4mb 5.4mb green open logstash-tls-2021.11.30 --m1vS-PTX-Dn4fXF4aC-w 1 0 19734 0 13.9mb 13.9mb green open logstash-2021.11.30 v0qOJO4RSPaD5ANVSWVK6g 1 0 14306 0 54.8mb 54.8mb green open logstash-tls-2021.12.01 JPnmB75PSBa4OBKS0lqEmg 1 0 15874 0 11mb 11mb green open logstash-fileinfo-2021.12.01 kfG5rjsaSTqOYb_X9LgAfQ 1 0 163533 0 52.9mb 52.9mb green open logstash-2021.12.02 J8ruNPQSSHGQX9BesI_6WA 1 0 8830 0 43.1mb 43.1mb green open logstash-anomaly-2021.12.01 WVz0YVt5QZmjTJ6B2Z0u_A 1 0 10 0 106.4kb 106.4kb green open logstash-fileinfo-2021.12.02 ACokjTnkQ1e6IjzzPqXKbg 1 0 86163 0 30.4mb 30.4mb green open logstash-anomaly-2021.11.30 YZmUfWpxToaNCVSqZt4sMQ 1 0 14 0 172.6kb 172.6kb green open logstash-2021.12.01 MaZaWT-nQ8yBDlgQcnuwZA 1 0 16556 0 68.4mb 68.4mb green open logstash-fileinfo-2021.11.30 Dgl2Jau7Q9ayj8KMOvdmKw 1 0 156056 0 55.5mb 55.5mb green open .kibana_2 SiWrAuE7TaOGdzEKM4dYrw 1 0 1151 3 496.4kb 496.4kb green open .kibana_1 Mr0Z7PvZTBGxy1GfNmlJnQ 1 0 1142 453 454.9kb 454.9kb green open logstash-anomaly-2021.12.02 d9zvEdhgSHONK-tKDmySFQ 1 0 5 0 41.2kb 41.2kb green open logstash-2021.11.29 mNlZLabVQjeeN0TMTDtsBg 1 0 16367 0 77.3mb 77.3mb green open logstash-smtp-2021.12.02 oUMRgtqLSxmkpVu7L_i4ag 1 0 4 0 108.4kb 108.4kb green open logstash-smtp-2021.12.01 3yfcyxwbTDeZQUliSeqHog 1 0 9 0 243.4kb 243.4kb green open logstash-flow-2021.12.02 d9iAMuHzSa-Icxl25EkD1Q 1 0 219408 0 59.9mb 59.9mb green open logstash-flow-2021.12.01 3GAhPxtSSFWFmE-v_bo5lA 1 0 438340 0 122mb 122mb green open logstash-smtp-2021.11.30 2r2S2-etSNad2aDFDPn_lg 1 0 9 0 243.4kb 243.4kb green open .async-search Hc9Ju-cbQS2Lg_8L6KX77w 1 0 0 11 3.7kb 3.7kb

pevma commented 2 years ago

So it seems you either have no data/ traffic for that period of time or maybe suricata was not running or not running on the correct interface ?

kdy1968 commented 2 years ago

suricata is running, it is running in inline, and there are alerts:

selks6-interfaces-config.yaml

%YAML 1.1
---
# AUTOGENERATED by Stamus SELKS set up script
# Linux high speed capture support
af-packet:
  # Put default values here. These will be used for an interface that is not
  # in the list above.
  - interface: default
    #threads: auto
    #use-mmap: no
    #rollover: yes
    #tpacket-v3: yes
  - interface: enp1s0
    threads: auto
    cluster-id: 99
    cluster-type: cluster_flow
    defrag: no
    use-mmap: yes
    #mmap-locked: yes
    #tpacket-v3: no
    ring-size: 2048
    block-size: 32768
    #block-timeout: 10
    #use-emergency-flush: yes
    #checksum-checks: kernel
    #bpf-filter: port 80 or udp
    copy-mode: ips
    copy-iface: enp2s0
  - interface: enp2s0
    threads: auto
    cluster-id: 100
    cluster-type: cluster_flow
    defrag: no
    use-mmap: yes
    #mmap-locked: yes
    #tpacket-v3: yes
    ring-size: 2048
    block-size: 32768
    #block-timeout: 10
    #use-emergency-flush: yes
    #checksum-checks: kernel
    #bpf-filter: port 80 or udp
    copy-mode: ips
    copy-iface: enp1s0

only thing i see in the logs that might indicate a issue, suricus elasticsearch.log


> 2021-12-04 12:16:46,753 POST http://localhost:9200/metricbeat-2021.12.03*,metricbeat-2021.12.04*,metricbeat-2021.12.05*/_search?ignore_unavailable=true
> -- 
> --         {
> --           "size": 0,
> --           "aggs": {
> --             "date": {
> --               "date_histogram": {
> --                 "field": "@timestamp",
> --                 "fixed_interval": "864002ms",
> --                 "min_doc_count": 0
> --               },
> --               "aggs": {
> --                 "stat": {
> --                   "avg": {
> --                     "field": "eve_insert.rate_1m"
> --                   }
> --                 }
> --               }
> --             }
> --           },
> --           "query": {
> --                 "bool": {
> --                   "must": [
> --                     {
> --                       "range": {
> --                           "@timestamp": {
> --                             "from": 1638533806549,
> --                             "to": "now"
> --                           }
> --                       }
> --                     },
> --                 {
> --                 "query_string": {
> --               
> --                   "query": "tags:metric",
> --               
> --                   "analyze_wildcard": false
> --                 }
> --               }
> --                   ]
> -- 
> --                 }
> --               }
> --         }
> --             
> 2021-12-04 12:16:46,753 GET http://localhost:9200/_stats/docs
> -- No data
> 2021-12-04 12:16:46,767 GET http://localhost:9200/_stats/store
> -- No data
> 2021-12-04 12:16:46,784 GET http://localhost:9200/_cluster/stats
> -- No data
> 2021-12-04 12:16:46,814 GET http://localhost:9200/_cluster/health
> -- No data
> 2021-12-04 12:17:16,939 GET http://localhost:9200/_cluster/health
> -- No data
> 2021-12-04 12:17:16,954 GET http://localhost:9200/_stats/docs
> -- No data
> 2021-12-04 12:17:16,955 GET http://localhost:9200/_cluster/stats
> -- No data
> 2021-12-04 12:17:16,960 GET http://localhost:9200/_stats/store
> -- No data
> 2021-12-04 12:17:16,965 GET http://localhost:9200/_cluster/health
> -- No data
> 2021-12-04 12:17:47,057 GET http://localhost:9200/_cluster/health
> -- No data
> 2021-12-04 12:17:47,062 GET http://localhost:9200/_stats/docs
> -- No data
> 2021-12-04 12:17:47,067 GET http://localhost:9200/_stats/store
> -- No data
> 2021-12-04 12:17:47,073 GET http://localhost:9200/_cluster/health
> -- No data
> 2021-12-04 12:17:47,077 GET http://localhost:9200/_cluster/stats
> -- No data
> 2021-12-04 12:18:16,842 GET http://localhost:9200/_cluster/health
> -- No data
> 2021-12-04 12:18:17,151 GET http://localhost:9200/_cluster/health
> -- No data
> 2021-12-04 12:18:17,154 GET http://localhost:9200/_stats/docs
> -- No data
> 2021-12-04 12:18:17,160 GET http://localhost:9200/_stats/store
> -- No data
> 2021-12-04 12:18:17,169 GET http://localhost:9200/_cluster/stats
> -- No data
> 2021-12-04 12:18:46,953 GET http://localhost:9200/_cluster/health
> -- No data
> 2021-12-04 12:18:47,252 GET http://localhost:9200/_cluster/health
> -- No data
> 2021-12-04 12:18:47,257 GET http://localhost:9200/_stats/docs
> -- No data
> 2021-12-04 12:18:47,263 GET http://localhost:9200/_stats/store
> -- No data
> 2021-12-04 12:18:47,270 GET http://localhost:9200/_cluster/stats
> -- No data
> 2021-12-04 12:19:17,061 GET http://localhost:9200/_cluster/health
> -- No data
> 2021-12-04 12:19:17,362 GET http://localhost:9200/_cluster/health
> -- No data
> 2021-12-04 12:19:17,367 GET http://localhost:9200/_stats/docs
> -- No data
> 2021-12-04 12:19:17,373 GET http://localhost:9200/_stats/store
> -- No data
> 2021-12-04 12:19:17,380 GET http://localhost:9200/_cluster/stats
> -- No data
> 2021-12-04 12:19:46,859 GET http://localhost:9200/_cluster/health
> -- No data
> 2021-12-04 12:19:47,469 GET http://localhost:9200/_cluster/health
> -- No data
> 2021-12-04 12:19:47,470 GET http://localhost:9200/_stats/docs
> -- No data
> 2021-12-04 12:19:47,475 GET http://localhost:9200/_stats/store
> -- No data
> 2021-12-04 12:19:47,487 GET http://localhost:9200/_cluster/stats
> -- No data
> 2021-12-04 12:20:14,750 GET http://localhost:9200/_stats/docs
> -- No data
> 2021-12-04 12:20:14,756 GET http://localhost:9200/_stats/store
> -- No data
> 2021-12-04 12:20:14,756 POST http://localhost:9200/metricbeat-2021.12.03*,metricbeat-2021.12.04*,metricbeat-2021.12.05*/_search?ignore_unavailable=true
> -- 
> --         {
> --           "size": 0,
> --           "aggs": {
> --             "date": {
> --               "date_histogram": {
> --                 "field": "@timestamp",
> --                 "fixed_interval": "864002ms",
> --                 "min_doc_count": 0
> --               },
> --               "aggs": {
> --                 "stat": {
> --                   "avg": {
> --                     "field": "eve_insert.rate_1m"
> --                   }
> --                 }
> --               }
> --             }
> --           },
> --           "query": {
> --                 "bool": {
> --                   "must": [
> --                     {
> --                       "range": {
> --                           "@timestamp": {
> --                             "from": 1638534014502,
> --                             "to": "now"
> --                           }
> --                       }
> --                     },
> --                 {
> --                 "query_string": {
> --               
> --                   "query": "tags:metric",
> --               
> --                   "analyze_wildcard": false
> --                 }
> --               }
> --                   ]
> -- 
> --                 }
> --               }
> --         }
> --             
> 2021-12-04 12:20:14,768 GET http://localhost:9200/_cluster/health
> -- No data
> 2021-12-04 12:20:14,768 GET http://localhost:9200/_cluster/stats
> -- No data
> 2021-12-04 12:20:14,780 GET http://localhost:9200/_cluster/health
> -- No data
> 2021-12-04 12:20:44,922 GET http://localhost:9200/_stats/docs
> -- No data
> 2021-12-04 12:20:44,922 GET http://localhost:9200/_cluster/health
> -- No data
> 2021-12-04 12:20:44,936 GET http://localhost:9200/_stats/store
> -- No data
> 2021-12-04 12:20:44,937 GET http://localhost:9200/_cluster/health
> -- No data
> 2021-12-04 12:20:44,949 GET http://localhost:9200/_cluster/stats
> -- No data
> 2021-12-04 12:21:15,026 GET http://localhost:9200/_cluster/health
> -- No data
> 2021-12-04 12:21:15,041 GET http://localhost:9200/_stats/docs
> -- No data
> 2021-12-04 12:21:15,047 GET http://localhost:9200/_stats/store
> -- No data
> 2021-12-04 12:21:15,048 GET http://localhost:9200/_cluster/health
> -- No data
> 2021-12-04 12:21:15,051 GET http://localhost:9200/_cluster/stats
> -- No data
> 2021-12-04 12:21:41,371 GET http://localhost:9200/_cluster/health
> -- No data
> 2021-12-04 12:21:41,374 POST http://localhost:9200/logstash-alert-2021.12.03*,logstash-alert-2021.12.04*,logstash-alert-2021.12.05*/_search?ignore_unavailable=true
> -- 
> --         {
> --           "size": 0,
> --           "query": {
> --             "bool": {
> --               "must": [ {
> --                 "query_string": {
> --                   "query": "event_type:alert ",
> --                   "analyze_wildcard": false
> --                 }
> --               },
> --                     {
> --                       "range": {
> --                         "@timestamp": {
> --                           "gte": 1638534101133,
> --                           "lte": "now",
> --                           "format": "epoch_millis"
> --                         }
> --                       }
> --                     }
> --                   ]
> -- 
> --                 }
> --           },
> --           "aggs": {
> --             "date": {
> --               "date_histogram": {
> --                 "field": "@timestamp",
> --                 "fixed_interval": "864002ms",
> --                 "min_doc_count": 0
> --               },
> --               "aggs": {
> --                 "host": {
> --                   "terms": {
> --                     "field": "host.keyword",
> --                     "size": 5,
> --                     "order": {
> --                       "_count": "desc"
> --                     }
> --                   }
> --                 }
> --               }
> --             }
> --           }
> --         }
> --             
> 2021-12-04 12:21:41,378 POST http://localhost:9200/logstash-alert-2021.12.02*,logstash-alert-2021.12.03*,logstash-alert-2021.12.04*,logstash-alert-2021.12.05*/_search?ignore_unavailable=true
> -- 
> --         {
> --           "size": 0,
> --           "aggs": {
> --             "trend": {
> --               "date_range": {
> --                 "field": "@timestamp",
> --                 "ranges": [
> --                   {
> --                     "from": 1638447700000,
> --                     "to": 1638534101133
> --                   },
> --                   {
> --                     "from": 1638534101133
> --                   }
> --                 ]
> --               }
> --             }
> --           },
> --           "query": {
> --                 "bool": {
> --                   "must": [
> --                     {
> --                       "range": {
> --                         "@timestamp": {
> --                           "gte": 1638447700000,
> --                           "lte": "now"
> -- 
> --                         }
> --                       }
> --                     }
> --                     ,{
> --                 "query_string": {
> --                   "query": "event_type:alert AND _exists_:host ",
> --                   "analyze_wildcard": true
> --                 }
> --               }
> --                   ]
> -- 
> --                 }
> --           }
> --         }
> --         
> 2021-12-04 12:21:41,378 POST http://localhost:9200/logstash-alert-2021.12.03*,logstash-alert-2021.12.04*,logstash-alert-2021.12.05*/_search?ignore_unavailable=true
> -- 
> --         {
> --           "size": 0,
> --           "aggs": {
> --             "table": {
> --               "terms": {
> --                 "field": "alert.signature_id",
> --                 "size": 20,
> --                 "order": {
> --                   "_count": "desc"
> --                 }
> --               }
> --             }
> --           },
> --           "query": {
> --             "bool": {
> --               "must": [ {
> --                 "query_string": {
> --                   "query": "event_type:alert AND  _exists_:host ",
> --                   "analyze_wildcard": false
> --                 }
> --               },
> --                     {
> --                       "range": {
> --                          "@timestamp": {
> --                             "from": 1638534101133,
> --                             "to": "now"
> --                          }
> --                       }
> --                     }
> --                   ]
> -- 
> --                 }
> --               }
> --         }
> --             
> 2021-12-04 12:21:56,496 POST http://localhost:9200/logstash-alert-2021.12.02*,logstash-alert-2021.12.03*,logstash-alert-2021.12.04*,logstash-alert-2021.12.05*/_search?ignore_unavailable=true
> -- 
> --         {
> --           "size": 0,
> --           "aggs": {
> --             "trend": {
> --               "date_range": {
> --                 "field": "@timestamp",
> --                 "ranges": [
> --                   {
> --                     "from": 1638447685000,
> --                     "to": 1638534101133
> --                   },
> --                   {
> --                     "from": 1638534101133
> --                   }
> --                 ]
> --               }
> --             }
> --           },
> --           "query": {
> --                 "bool": {
> --                   "must": [
> --                     {
> --                       "range": {
> --                         "@timestamp": {
> --                           "gte": 1638447685000,
> --                           "lte": "now"
> -- 
> --                         }
> --                       }
> --                     }
> --                     ,{
> --                 "query_string": {
> --                   "query": "event_type:alert AND _exists_:host ",
> --                   "analyze_wildcard": true
> --                 }
> --               }
> --                   ]
> -- 
> --                 }
> --           }
> --         }
> --         
> 2021-12-04 12:22:11,443 GET http://localhost:9200/_cluster/health
> -- No data
> 2021-12-04 12:22:11,454 POST http://localhost:9200/logstash-alert-2021.12.02*,logstash-alert-2021.12.03*,logstash-alert-2021.12.04*,logstash-alert-2021.12.05*/_search?ignore_unavailable=true
> -- 
> --         {
> --           "size": 0,
> --           "aggs": {
> --             "trend": {
> --               "date_range": {
> --                 "field": "@timestamp",
> --                 "ranges": [
> --                   {
> --                     "from": 1638447670000,
> --                     "to": 1638534101133
> --                   },
> --                   {
> --                     "from": 1638534101133
> --                   }
> --                 ]
> --               }
> --             }
> --           },
> --           "query": {
> --                 "bool": {
> --                   "must": [
> --                     {
> --                       "range": {
> --                         "@timestamp": {
> --                           "gte": 1638447670000,
> --                           "lte": "now"
> -- 
> --                         }
> --                       }
> --                     }
> --                     ,{
> --                 "query_string": {
> --                   "query": "event_type:alert AND _exists_:host ",
> --                   "analyze_wildcard": true
> --                 }
> --               }
> --                   ]
> -- 
> --                 }
> --           }
> --         }
> --         
> 2021-12-04 12:22:13,938 POST http://localhost:9200/logstash-alert-2021.12.03*,logstash-alert-2021.12.04*,logstash-alert-2021.12.05*/_search?ignore_unavailable=true
> -- 
> --         {
> --           "size": 0,
> --           "aggs": {
> --             "table": {
> --               "terms": {
> --                 "field": "alert.source.ip.keyword",
> --                 "size": 10,
> --                 "order": {
> --                   "_count": "desc"
> --                 }
> --               }
> --             }
> --           },
> --           "query": {
> --             "bool": {
> --               "must": [ {
> --                 "query_string": {
> --                   "query": "event_type:alert AND alert.signature_id: 2500000 AND _exists_:host ",
> --                   "analyze_wildcard": false
> --                 }
> --               },
> --                     {
> --                       "range": {
> --                          "@timestamp": {
> --                             "from": 1638534133767,
> --                             "to": "now"
> --                          }
> --                       }
> --                     }
> --                   ]
> -- 
> --                 }
> --               }
> --         }
> --             
> 2021-12-04 12:22:13,942 POST http://localhost:9200/logstash-alert-2021.12.03*,logstash-alert-2021.12.04*,logstash-alert-2021.12.05*/_search?ignore_unavailable=true
> -- 
> --         {
> --           "size": 0,
> --           "aggs": {
> --             "table": {
> --               "terms": {
> --                 "field": "src_ip.keyword",
> --                 "size": 10,
> --                 "order": {
> --                   "_count": "desc"
> --                 }
> --               }
> --             }
> --           },
> --           "query": {
> --             "bool": {
> --               "must": [ {
> --                 "query_string": {
> --                   "query": "event_type:alert AND alert.signature_id: 2500000 AND _exists_:host ",
> --                   "analyze_wildcard": false
> --                 }
> --               },
> --                     {
> --                       "range": {
> --                          "@timestamp": {
> --                             "from": 1638534133767,
> --                             "to": "now"
> --                          }
> --                       }
> --                     }
> --                   ]
> -- 
> --                 }
> --               }
> --         }
> --             
> 2021-12-04 12:22:13,943 POST http://localhost:9200/logstash-alert-2021.12.03*,logstash-alert-2021.12.04*,logstash-alert-2021.12.05*/_search?ignore_unavailable=true
> -- 
> --         {
> --           "size": 0,
> --           "aggs": {
> --             "table": {
> --               "terms": {
> --                 "field": "dest_ip.keyword",
> --                 "size": 10,
> --                 "order": {
> --                   "_count": "desc"
> --                 }
> --               }
> --             }
> --           },
> --           "query": {
> --             "bool": {
> --               "must": [ {
> --                 "query_string": {
> --                   "query": "event_type:alert AND alert.signature_id: 2500000 AND _exists_:host ",
> --                   "analyze_wildcard": false
> --                 }
> --               },
> --                     {
> --                       "range": {
> --                          "@timestamp": {
> --                             "from": 1638534133767,
> --                             "to": "now"
> --                          }
> --                       }
> --                     }
> --                   ]
> -- 
> --                 }
> --               }
> --         }
> --             
> 2021-12-04 12:22:13,944 POST http://localhost:9200/logstash-alert-2021.12.03*,logstash-alert-2021.12.04*,logstash-alert-2021.12.05*/_search?ignore_unavailable=true
> -- 
> --         {
> --           "size": 0,
> --           "aggs": {
> --             "host": {
> --               "terms": {
> --                 "field": "host.keyword",
> --                 "size": 20,
> --                 "order": {
> --                   "_count": "desc"
> --                 }
> --               }
> --             }
> --           },
> --           "query": {
> --                 "bool": {
> --                   "must": [
> --                     {
> --                       "range": {
> --                          "@timestamp": {
> --                             "from": 1638534133767,
> --                             "to": "now"
> --                          }
> --                       }
> --                     }
> --                     ,{
> --                     "query_string": {
> --                        "query": "event_type:alert AND alert.signature_id:2500000",
> --                        "analyze_wildcard": false
> --                      }
> --                     }
> --                   ]
> -- 
> --                 }
> --               }
> --         }
> --             
> 2021-12-04 12:22:13,947 POST http://localhost:9200/logstash-alert-2021.12.03*,logstash-alert-2021.12.04*,logstash-alert-2021.12.05*/_search?ignore_unavailable=true
> -- 
> --         {
> --           "size": 0,
> --           "aggs": {
> --             "table": {
> --               "terms": {
> --                 "field": "alert.target.ip.keyword",
> --                 "size": 10,
> --                 "order": {
> --                   "_count": "desc"
> --                 }
> --               }
> --             }
> --           },
> --           "query": {
> --             "bool": {
> --               "must": [ {
> --                 "query_string": {
> --                   "query": "event_type:alert AND alert.signature_id: 2500000 AND _exists_:host ",
> --                   "analyze_wildcard": false
> --                 }
> --               },
> --                     {
> --                       "range": {
> --                          "@timestamp": {
> --                             "from": 1638534133767,
> --                             "to": "now"
> --                          }
> --                       }
> --                     }
> --                   ]
> -- 
> --                 }
> --               }
> --         }
> --             
> 2021-12-04 12:22:13,994 GET http://localhost:9200/_cluster/health
> -- No data
> 2021-12-04 12:22:13,999 POST http://localhost:9200/logstash-alert-2021.12.03*,logstash-alert-2021.12.04*,logstash-alert-2021.12.05*/_search?ignore_unavailable=true
> -- 
> --         {
> --           "size": 0,
> --           "query": {
> --             "bool": {
> --               "must": [ {
> --                 "query_string": {
> --                   "query": "event_type:alert  AND alert.signature_id:2500000",
> --                   "analyze_wildcard": false
> --                 }
> --               },
> --                     {
> --                       "range": {
> --                         "@timestamp": {
> --                           "gte": 1638534133767,
> --                           "lte": "now",
> --                           "format": "epoch_millis"
> --                         }
> --                       }
> --                     }
> --                   ]
> -- 
> --                 }
> --           },
> --           "aggs": {
> --             "date": {
> --               "date_histogram": {
> --                 "field": "@timestamp",
> --                 "fixed_interval": "864002ms",
> --                 "min_doc_count": 0
> --               },
> --               "aggs": {
> --                 "host": {
> --                   "terms": {
> --                     "field": "host.keyword",
> --                     "size": 5,
> --                     "order": {
> --                       "_count": "desc"
> --                     }
> --                   }
> --                 }
> --               }
> --             }
> --           }
> --         }
> --             
> 2021-12-04 12:22:35,666 GET http://localhost:9200/_cluster/health
> -- No data
> 2021-12-04 12:22:42,305 POST http://localhost:9200/logstash-alert-2021.12.03*,logstash-alert-2021.12.04*,logstash-alert-2021.12.05*/_search?ignore_unavailable=true
> -- 
> --         {
> --           "size": 0,
> --           "aggs": {
> --             "host": {
> --               "terms": {
> --                 "field": "host.keyword",
> --                 "size": 20,
> --                 "order": {
> --                   "_count": "desc"
> --                 }
> --               }
> --             }
> --           },
> --           "query": {
> --                 "bool": {
> --                   "must": [
> --                     {
> --                       "range": {
> --                          "@timestamp": {
> --                             "from": 1638534162123,
> --                             "to": "now"
> --                          }
> --                       }
> --                     }
> --                     ,{
> --                     "query_string": {
> --                        "query": "event_type:alert AND alert.signature_id:2500000",
> --                        "analyze_wildcard": false
> --                      }
> --                     }
> --                   ]
> -- 
> --                 }
> --               }
> --         }
> --             
> 2021-12-04 12:22:42,305 POST http://localhost:9200/logstash-alert-2021.12.03*,logstash-alert-2021.12.04*,logstash-alert-2021.12.05*/_search?ignore_unavailable=true
> -- 
> --         {
> --           "size": 0,
> --           "aggs": {
> --             "table": {
> --               "terms": {
> --                 "field": "dest_ip.keyword",
> --                 "size": 10,
> --                 "order": {
> --                   "_count": "desc"
> --                 }
> --               }
> --             }
> --           },
> --           "query": {
> --             "bool": {
> --               "must": [ {
> --                 "query_string": {
> --                   "query": "event_type:alert AND alert.signature_id: 2500000 AND _exists_:host ",
> --                   "analyze_wildcard": false
> --                 }
> --               },
> --                     {
> --                       "range": {
> --                          "@timestamp": {
> --                             "from": 1638534162123,
> --                             "to": "now"
> --                          }
> --                       }
> --                     }
> --                   ]
> -- 
> --                 }
> --               }
> --         }
> --             
> 2021-12-04 12:22:42,307 POST http://localhost:9200/logstash-alert-2021.12.03*,logstash-alert-2021.12.04*,logstash-alert-2021.12.05*/_search?ignore_unavailable=true
> -- 
> --         {
> --           "size": 0,
> --           "aggs": {
> --             "table": {
> --               "terms": {
> --                 "field": "src_ip.keyword",
> --                 "size": 10,
> --                 "order": {
> --                   "_count": "desc"
> --                 }
> --               }
> --             }
> --           },
> --           "query": {
> --             "bool": {
> --               "must": [ {
> --                 "query_string": {
> --                   "query": "event_type:alert AND alert.signature_id: 2500000 AND _exists_:host ",
> --                   "analyze_wildcard": false
> --                 }
> --               },
> --                     {
> --                       "range": {
> --                          "@timestamp": {
> --                             "from": 1638534162123,
> --                             "to": "now"
> --                          }
> --                       }
> --                     }
> --                   ]
> -- 
> --                 }
> --               }
> --         }
> --             
> 2021-12-04 12:22:42,311 POST http://localhost:9200/logstash-alert-2021.12.03*,logstash-alert-2021.12.04*,logstash-alert-2021.12.05*/_search?ignore_unavailable=true
> -- 
> --         {
> --           "size": 0,
> --           "aggs": {
> --             "table": {
> --               "terms": {
> --                 "field": "alert.source.ip.keyword",
> --                 "size": 10,
> --                 "order": {
> --                   "_count": "desc"
> --                 }
> --               }
> --             }
> --           },
> --           "query": {
> --             "bool": {
> --               "must": [ {
> --                 "query_string": {
> --                   "query": "event_type:alert AND alert.signature_id: 2500000 AND _exists_:host ",
> --                   "analyze_wildcard": false
> --                 }
> --               },
> --                     {
> --                       "range": {
> --                          "@timestamp": {
> --                             "from": 1638534162123,
> --                             "to": "now"
> --                          }
> --                       }
> --                     }
> --                   ]
> -- 
> --                 }
> --               }
> --         }
> --             
> 2021-12-04 12:22:42,312 POST http://localhost:9200/logstash-alert-2021.12.03*,logstash-alert-2021.12.04*,logstash-alert-2021.12.05*/_search?ignore_unavailable=true
pevma commented 2 years ago

ok that is good that you see alerts! Do you have those lines in your logstash config https://github.com/StamusNetworks/SELKS/blob/master/staging/etc/logstash/conf.d/logstash.conf#L51 ?

kdy1968 commented 2 years ago

here is my logstash config, i had uncommented the geoip section when i was working on placing the ips on the wan side, as i mentioned previously, it works fine on he lan side so i will just leave it there...

currently just trying determine why i have no data in the logstash insertion speed

input { file { path => ["/var/log/suricata/*.json"]

sincedb_path => ["/var/lib/logstash/"]

sincedb_path => ["/var/cache/logstash/sincedbs/since.db"]
codec =>   json 
type => "SELKS"

}

}

filter { if [type] == "SELKS" {

date {
  match => [ "timestamp", "ISO8601" ]
}

ruby {
  code => "
    if event.get('[event_type]') == 'fileinfo'
      event.set('[fileinfo][type]', event.get('[fileinfo][magic]').to_s.split(',')[0])
    end
  "
}
ruby {
  code => "
    if event.get('[event_type]') == 'alert'
      sp = event.get('[alert][signature]').to_s.split(' group ')
      if (sp.length == 2) and /\A\d+\z/.match(sp[1])
        event.set('[alert][signature]', sp[0])
      end
    end
  "
 }

metrics {
  meter => [ "eve_insert" ]
  add_tag => "metric"
  flush_interval => 30
}

}

if [http] { useragent { source => "[http][http_user_agent]" target => "[http][user_agent]" } } if [src_ip] { geoip { source => "src_ip" target => "geoip" database => "/etc/Geolite2/GeoLite2-City.mmdb" add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ] add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}" ] } } if [dest_ip] { geoip { source => "dest_ip" target => "geoip" database => "/etc/Geolite2/GeoLite2-City.mmdb" add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ] add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}" ] } } }

output { if [event_type] and [event_type] != 'stats' { elasticsearch { hosts => "127.0.0.1" index => "logstash-%{event_type}-%{+YYYY.MM.dd}" template_overwrite => true template => "/etc/logstash/elasticsearch7-template.json" } } else { elasticsearch { hosts => "127.0.0.1" index => "logstash-%{+YYYY.MM.dd}" template_overwrite => true template => "/etc/logstash/elasticsearch7-template.json" } } }

mickaelmonsieur commented 2 years ago

I have the same error with the ISO "SELKS 6 ISO without desktop" from https://www.stamus-networks.com/selks downladed yesterday :

Certificate verification failed: The certificate is NOT trusted. The certificate chain uses expired certificate. Could not handshake: Error in the certificate verification. [IP: 172.105.5.173 443]

Installation on disk is therefore currently impossible.

pevma commented 2 years ago

@mickaelmonsieur - please do not mix threads questions :) The download seems to be pulling ok on my end at least on my end. The ip displayed is not that of the stamus-networks.com it seems.

bargibargi commented 2 years ago

I have the same error with the ISO "SELKS 6 ISO without desktop" from https://www.stamus-networks.com/selks downladed yesterday :

Certificate verification failed: The certificate is NOT trusted. The certificate chain uses expired certificate. Could not handshake: Error in the certificate verification. [IP: 172.105.5.173 443]

Installation on disk is therefore currently impossible.

I found the same today, looks like Lets encrypt cert issue https://stackoverflow.com/questions/21181231/server-certificate-verification-failed-cafile-etc-ssl-certs-ca-certificates-c/69403278#69403278

Following worked for me sudo dpkg-reconfigure ca-certificates and deselecting "DST Root CA X3" certificate

pevma commented 2 years ago

Thanks for the heads up

Can you retry please ? Do you have the same issue? In the mean while you can always try the docker set up if needed - https://github.com/StamusNetworks/SELKS/wiki/Docker

Thank you

-- Regards, Peter Manev

On 29 Dec 2021, at 14:32, bargibargi @.***> wrote:

 I have the same error with the ISO "SELKS 6 ISO without desktop" from https://www.stamus-networks.com/selks downladed yesterday :

Certificate verification failed: The certificate is NOT trusted. The certificate chain uses expired certificate. Could not handshake: Error in the certificate verification. [IP: 172.105.5.173 443]

Installation on disk is therefore currently impossible.

I found the same today, looks like Lets encrypt cert issue https://stackoverflow.com/questions/21181231/server-certificate-verification-failed-cafile-etc-ssl-certs-ca-certificates-c/69403278#69403278

Following worked for me sudo dpkg-reconfigure ca-certificates and deselecting "DST Root CA X3" certificate

— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you commented.