Closed AndriiOmelianenko closed 8 years ago
Hi @AndriiOmelianenko. Thank you very much for your report.
Unfortunately this is a known issue, ContainerSolutions/mesos-starter#43. The easy fix is to make sure that MESOS_RESOURCES
is unset before starting the application.
@mwl thanks for your response.
I've moved configuration from MESOS_RESOURCES=ports:[1025-2180,2182-3887,3889-5049,6000-65535];ports(logstash):[514-514,25826-25826]
to next option - /usr/sbin/mesos-slave --resources=ports(*):[1025-2180,2182-3887,3889-5049,6000-65535];ports(logstash):[514-514,25826-25826]
,
submitted logstash json to marathon, it started running (without errors), but nothing happens after that:
stdout:
Registered executor on slave-04.example.com
Starting task logstash.32f2c5b9-02eb-11e6-b828-0242000bf102
sh -c 'java -jar logstash-mesos-scheduler-0.1.0.jar --server.port=$PORT0'
Forked command at 25395
09:19:51.541 [background-preinit] INFO o.h.validator.internal.util.Version - HV000001: Hibernate Validator 5.1.3.Final
|\ /|
| \ / |
| / \ |
|/ \|
/ \ /|
/ \ / | .____ __ .__
\ | | | | ____ ____ _______/ |______ _____| |__
\ | | | | / _ \ / ___\/ ___/\ __\__ \ / ___/ | \
| | / | |__( <_> ) /_/ >___ \ | | / __ \_\___ \| Y \
| |/ |_______ \____/\___ /____ > |__| (____ /____ >___| /
| / \/ /_____/ \/ \/ \/ \/
|/ :: Running Spring Boot 0.1.0 ::
2016-04-15 09:19:51.645 INFO 25395 --- [ main] o.a.m.logstash.scheduler.Application : Starting Application v0.1.0 on slave-04.example.com with PID 25395 (/mnt/mesos/slaves/25625492-7806-42fd-af67-90f6b0fef272-S4/frameworks/25625492-7806-42fd-af67-90f6b0fef272-0001/executors/logstash.32f2c5b9-02eb-11e6-b828-0242000bf102/runs/cff5114b-0103-466c-ba97-227ca5b39c83/logstash-mesos-scheduler-0.1.0.jar started by root in /mnt/mesos/slaves/25625492-7806-42fd-af67-90f6b0fef272-S4/frameworks/25625492-7806-42fd-af67-90f6b0fef272-0001/executors/logstash.32f2c5b9-02eb-11e6-b828-0242000bf102/runs/cff5114b-0103-466c-ba97-227ca5b39c83)
2016-04-15 09:19:51.645 INFO 25395 --- [ main] o.a.m.logstash.scheduler.Application : No active profile set, falling back to default profiles: default
2016-04-15 09:19:53.885 INFO 25395 --- [ main] o.a.m.logstash.scheduler.Application : Started Application in 2.85 seconds (JVM running for 3.23)
stderr:
I0415 09:19:47.727182 25341 logging.cpp:172] INFO level logging started!
I0415 09:19:47.727496 25341 fetcher.cpp:414] Fetcher Info: {"cache_directory":"\/tmp\/mesos\/fetch\/slaves\/25625492-7806-42fd-af67-90f6b0fef272-S4\/root","items":[{"action":"BYPASS_CACHE","uri":{"extract":false,"value":"https:\/\/github.com\/mesos\/logstash\/releases\/download\/0.10\/logstash-mesos-scheduler-0.1.0.jar"}}],"sandbox_directory":"\/mnt\/mesos\/slaves\/25625492-7806-42fd-af67-90f6b0fef272-S4\/frameworks\/25625492-7806-42fd-af67-90f6b0fef272-0001\/executors\/logstash.32f2c5b9-02eb-11e6-b828-0242000bf102\/runs\/cff5114b-0103-466c-ba97-227ca5b39c83","user":"root"}
I0415 09:19:47.728615 25341 fetcher.cpp:369] Fetching URI 'https://github.com/mesos/logstash/releases/download/0.10/logstash-mesos-scheduler-0.1.0.jar'
I0415 09:19:47.728631 25341 fetcher.cpp:243] Fetching directly into the sandbox directory
I0415 09:19:47.728643 25341 fetcher.cpp:180] Fetching URI 'https://github.com/mesos/logstash/releases/download/0.10/logstash-mesos-scheduler-0.1.0.jar'
I0415 09:19:47.728667 25341 fetcher.cpp:127] Downloading resource from 'https://github.com/mesos/logstash/releases/download/0.10/logstash-mesos-scheduler-0.1.0.jar' to '/mnt/mesos/slaves/25625492-7806-42fd-af67-90f6b0fef272-S4/frameworks/25625492-7806-42fd-af67-90f6b0fef272-0001/executors/logstash.32f2c5b9-02eb-11e6-b828-0242000bf102/runs/cff5114b-0103-466c-ba97-227ca5b39c83/logstash-mesos-scheduler-0.1.0.jar'
I0415 09:19:50.521008 25341 fetcher.cpp:446] Fetched 'https://github.com/mesos/logstash/releases/download/0.10/logstash-mesos-scheduler-0.1.0.jar' to '/mnt/mesos/slaves/25625492-7806-42fd-af67-90f6b0fef272-S4/frameworks/25625492-7806-42fd-af67-90f6b0fef272-0001/executors/logstash.32f2c5b9-02eb-11e6-b828-0242000bf102/runs/cff5114b-0103-466c-ba97-227ca5b39c83/logstash-mesos-scheduler-0.1.0.jar'
I0415 09:19:50.644446 25338 logging.cpp:172] INFO level logging started!
I0415 09:19:50.645401 25338 exec.cpp:134] Version: 0.25.0
I0415 09:19:50.648433 25391 exec.cpp:208] Executor registered on slave 25625492-7806-42fd-af67-90f6b0fef272-S4
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/mnt/mesos/slaves/25625492-7806-42fd-af67-90f6b0fef272-S4/frameworks/25625492-7806-42fd-af67-90f6b0fef272-0001/executors/logstash.32f2c5b9-02eb-11e6-b828-0242000bf102/runs/cff5114b-0103-466c-ba97-227ca5b39c83/logstash-mesos-scheduler-0.1.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/mnt/mesos/slaves/25625492-7806-42fd-af67-90f6b0fef272-S4/frameworks/25625492-7806-42fd-af67-90f6b0fef272-0001/executors/logstash.32f2c5b9-02eb-11e6-b828-0242000bf102/runs/cff5114b-0103-466c-ba97-227ca5b39c83/logstash-mesos-scheduler-0.1.0.jar!/lib/logback-classic-1.1.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
2016-04-15 09:19:53,722:25395(0x7f9daec9b700):ZOO_INFO@log_env@712: Client environment:zookeeper.version=zookeeper C client 3.4.5
2016-04-15 09:19:53,722:25395(0x7f9daec9b700):ZOO_INFO@log_env@716: Client environment:host.name=slave-04.example.com
2016-04-15 09:19:53,722:25395(0x7f9daec9b700):ZOO_INFO@log_env@723: Client environment:os.name=Linux
2016-04-15 09:19:53,722:25395(0x7f9daec9b700):ZOO_INFO@log_env@724: Client environment:os.arch=3.10.0-229.20.1.el7.x86_64
2016-04-15 09:19:53,722:25395(0x7f9daec9b700):ZOO_INFO@log_env@725: Client environment:os.version=#1 SMP Tue Nov 3 19:10:07 UTC 2015
2016-04-15 09:19:53,723:25395(0x7f9daec9b700):ZOO_INFO@log_env@733: Client environment:user.name=(null)
2016-04-15 09:19:53,723:25395(0x7f9daec9b700):ZOO_INFO@log_env@741: Client environment:user.home=/root
2016-04-15 09:19:53,723:25395(0x7f9daec9b700):ZOO_INFO@log_env@753: Client environment:user.dir=/mnt/mesos/slaves/25625492-7806-42fd-af67-90f6b0fef272-S4/frameworks/25625492-7806-42fd-af67-90f6b0fef272-0001/executors/logstash.32f2c5b9-02eb-11e6-b828-0242000bf102/runs/cff5114b-0103-466c-ba97-227ca5b39c83
2016-04-15 09:19:53,723:25395(0x7f9daec9b700):ZOO_INFO@zookeeper_init@786: Initiating client connection, host=zookeeper.service.consul:2181 sessionTimeout=1000 watcher=0x7f9e1965e700 sessionId=0 sessionPasswd=<null> context=0x7f9d980031d0 flags=0
2016-04-15 09:19:53,727:25395(0x7f9da5ffb700):ZOO_INFO@check_events@1703: initiated connection to server [192.168.0.63:2181]
2016-04-15 09:19:53,731:25395(0x7f9da5ffb700):ZOO_INFO@check_events@1750: session establishment complete on server [192.168.0.63:2181], sessionId=0x154192336800099, negotiated timeout=4000
I0415 09:19:53.872167 25410 logging.cpp:172] INFO level logging started!
is it expected to start executors etc?
The scheduler isn't connecting to Mesos. Probably because you have an equals (=
) sign in your MESOS_MASTER=
environment variable. Remove it and try again.
Trying to use 0.10.0 version (was using 0.8.0 successfully before). This issue happens when mesos trying to start logstash scheduler
my mesos slave config has next configuration line:
MESOS_RESOURCES=ports:[1025-2180,2182-3887,3889-5049,6000-65535];ports(logstash):[514-514,25826-25826]
logstash marathon json: