mesosphere / marathon-example-plugins

Example Plugins for Marathon Plugin Interface
Apache License 2.0
22 stars 29 forks source link

Unable to load env plugin in Marathon (minimesos) #19

Closed kohend closed 6 years ago

kohend commented 7 years ago

I tried to add the jar file and the config file to the Marathon Docker image of minimesos and it seems to fail in loading the plugin despite the config being read properly (from what I can tell) and the file seen by Marathon.
I also added the directory that the plugin is in to the class path.


MESOS_NATIVE_LIBRARY, MESOS_NATIVE_JAVA_LIBRARY set to '/usr/lib/libmesos.so'
No start hook file found ($HOOK_MARATHON_START). Proceeding with the start script.
[2017-07-03 14:46:40,850] INFO Starting Marathon 1.3.5/unknown with --master zk://minimesos-zookeeper:2181/mesos --zk zk://minimesos-zookeeper:2181/marathon --plugin_dir /marathon/plugin --plugin_conf /marathon/plugin-conf.json --enable_features secrets (mesosphere.marathon.Main$:main)
[2017-07-03 14:46:42,042] WARN Method [public javax.ws.rs.core.Response mesosphere.marathon.api.MarathonExceptionMapper.toResponse(java.lang.Throwable)] is synthetic and is being intercepted by [mesosphere.marathon.DebugModule$MetricsBehavior@1eaf1e62]. This could indicate a bug.  The method may be intercepted twice, or may not be intercepted at all. (com.google.inject.internal.ProxyFactory:main)
[2017-07-03 14:46:42,496] INFO Logging initialized @5177ms (org.eclipse.jetty.util.log:main)
[2017-07-03 14:46:44,173] INFO Slf4jLogger started (akka.event.slf4j.Slf4jLogger:marathon-akka.actor.default-dispatcher-3)
[2017-07-03 14:46:44,466] INFO Started TaskTrackerUpdateStepsProcessorImpl with steps:
* continueOnError(notifyHealthCheckManager)
* continueOnError(notifyRateLimiter)
* continueOnError(notifyLaunchQueue)
* continueOnError(emitUpdate)
* continueOnError(postTaskStatusEvent)
* continueOnError(scaleApp) (mesosphere.marathon.core.task.tracker.impl.TaskTrackerUpdateStepProcessorImpl:main)
[2017-07-03 14:46:44,596] INFO Calling reviveOffers is enabled. Use --disable_revive_offers_for_new_apps to disable. (mesosphere.marathon.core.flow.FlowModule:main)
[2017-07-03 14:46:45,062] INFO Loading plugins implementing 'mesosphere.marathon.plugin.auth.Authenticator' from these urls: [file:/marathon/plugin/plugins_2.11-0.1-SNAPSHOT.jar] (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:main)
[2017-07-03 14:46:45,066] INFO Found 0 plugins. (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:main)
[2017-07-03 14:46:45,081] INFO Loading plugins implementing 'mesosphere.marathon.plugin.auth.Authorizer' from these urls: [file:/marathon/plugin/plugins_2.11-0.1-SNAPSHOT.jar] (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:main)
[2017-07-03 14:46:45,082] INFO Found 0 plugins. (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:main)
[2017-07-03 14:46:45,084] INFO Started status update processor (mesosphere.marathon.core.task.update.impl.TaskStatusUpdateProcessorImpl$$EnhancerByGuice$$79bb6abf:main)
[2017-07-03 14:46:45,386] INFO All actors suspended:
* Actor[akka://marathon/user/taskTracker#-572049485]
* Actor[akka://marathon/user/offerMatcherLaunchTokens#-396922016]
* Actor[akka://marathon/user/reviveOffersWhenWanted#-1270188931]
* Actor[akka://marathon/user/launchQueue#-357430807]
* Actor[akka://marathon/user/offersWantedForReconciliation#352720590]
* Actor[akka://marathon/user/groupManager#103236799]
* Actor[akka://marathon/user/taskKillServiceActor#-1484763378]
* Actor[akka://marathon/user/rateLimiter#1530916594]
* Actor[akka://marathon/user/offerMatcherManager#539319868]
* Actor[akka://marathon/user/expungeOverdueLostTasks#-520277771]
* Actor[akka://marathon/user/killOverdueStagedTasks#1286174505] (mesosphere.marathon.core.leadership.impl.LeadershipCoordinatorActor:marathon-akka.actor.default-dispatcher-3)
[2017-07-03 14:46:45,582] INFO Adding HTTP support. (mesosphere.chaos.http.HttpModule:main)
[2017-07-03 14:46:45,583] INFO No HTTPS support configured. (mesosphere.chaos.http.HttpModule:main)
[2017-07-03 14:46:45,736] INFO Starting up (mesosphere.marathon.MarathonSchedulerService$$EnhancerByGuice$$c165252e:MarathonSchedulerService$$EnhancerByGuice$$c165252e)
[2017-07-03 14:46:45,737] INFO jetty-9.3.z-SNAPSHOT (org.eclipse.jetty.server.Server:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:45,738] INFO Beginning run (mesosphere.marathon.MarathonSchedulerService$$EnhancerByGuice$$c165252e:MarathonSchedulerService$$EnhancerByGuice$$c165252e)
[2017-07-03 14:46:45,744] INFO Will offer leadership after 500 milliseconds backoff (mesosphere.marathon.core.election.impl.CuratorElectionService:MarathonSchedulerService$$EnhancerByGuice$$c165252e)
[2017-07-03 14:46:45,818] INFO Now standing by. Closing existing handles and rejecting new. (mesosphere.marathon.core.event.impl.stream.HttpEventStreamActor:marathon-akka.actor.default-dispatcher-2)
[2017-07-03 14:46:46,013] INFO Registering com.codahale.metrics.jersey.InstrumentedResourceMethodDispatchAdapter as a provider class (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:46,015] INFO Registering mesosphere.marathon.api.MarathonExceptionMapper as a provider class (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:46,017] INFO Registering mesosphere.marathon.api.v2.AppsResource as a root resource class (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:46,017] INFO Registering mesosphere.marathon.api.v2.TasksResource as a root resource class (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:46,017] INFO Registering mesosphere.marathon.api.v2.EventSubscriptionsResource as a root resource class (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:46,017] INFO Registering mesosphere.marathon.api.v2.QueueResource as a root resource class (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:46,018] INFO Registering mesosphere.marathon.api.v2.GroupsResource as a root resource class (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:46,020] INFO Registering mesosphere.marathon.api.v2.InfoResource as a root resource class (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:46,020] INFO Registering mesosphere.marathon.api.v2.LeaderResource as a root resource class (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:46,020] INFO Registering mesosphere.marathon.api.v2.DeploymentsResource as a root resource class (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:46,020] INFO Registering mesosphere.marathon.api.v2.ArtifactsResource as a root resource class (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:46,020] INFO Registering mesosphere.marathon.api.v2.SchemaResource as a root resource class (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:46,020] INFO Registering mesosphere.marathon.api.v2.PluginsResource as a root resource class (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:46,025] INFO Initiating Jersey application, version 'Jersey: 1.18.1 02/19/2014 03:28 AM' (com.sun.jersey.server.impl.application.WebApplicationImpl:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:46,138] INFO Binding com.codahale.metrics.jersey.InstrumentedResourceMethodDispatchAdapter to GuiceManagedComponentProvider with the scope "Singleton" (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:46,168] INFO Binding mesosphere.marathon.api.MarathonExceptionMapper to GuiceManagedComponentProvider with the scope "Singleton" (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:46,274] INFO Using HA and therefore offering leadership (mesosphere.marathon.core.election.impl.CuratorElectionService:ForkJoinPool-2-worker-13)
[2017-07-03 14:46:46,279] INFO Will do leader election through minimesos-zookeeper:2181 (mesosphere.marathon.core.election.impl.CuratorElectionService:ForkJoinPool-2-worker-13)
[2017-07-03 14:46:46,386] WARN session timeout [10000] is less than connection timeout [15000] (org.apache.curator.CuratorZookeeperClient:ForkJoinPool-2-worker-13)
[2017-07-03 14:46:46,442] INFO Starting (org.apache.curator.framework.imps.CuratorFrameworkImpl:ForkJoinPool-2-worker-13)
[2017-07-03 14:46:46,476] INFO Client environment:zookeeper.version=3.5.0-alpha-1615249, built on 08/01/2014 22:13 GMT (org.apache.zookeeper.ZooKeeper:ForkJoinPool-2-worker-13)
[2017-07-03 14:46:46,479] INFO Client environment:host.name=02d73624fccf (org.apache.zookeeper.ZooKeeper:ForkJoinPool-2-worker-13)
[2017-07-03 14:46:46,479] INFO Client environment:java.version=1.8.0_111 (org.apache.zookeeper.ZooKeeper:ForkJoinPool-2-worker-13)
[2017-07-03 14:46:46,479] INFO Client environment:java.vendor=Oracle Corporation (org.apache.zookeeper.ZooKeeper:ForkJoinPool-2-worker-13)
[2017-07-03 14:46:46,480] INFO Client environment:java.home=/usr/lib/jvm/java-8-openjdk-amd64/jre (org.apache.zookeeper.ZooKeeper:ForkJoinPool-2-worker-13)
[2017-07-03 14:46:46,480] INFO Client environment:java.class.path=./bin/../target/marathon-assembly-1.3.5.jar (org.apache.zookeeper.ZooKeeper:ForkJoinPool-2-worker-13)
[2017-07-03 14:46:46,480] INFO Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib (org.apache.zookeeper.ZooKeeper:ForkJoinPool-2-worker-13)
[2017-07-03 14:46:46,480] INFO Client environment:java.io.tmpdir=/tmp (org.apache.zookeeper.ZooKeeper:ForkJoinPool-2-worker-13)
[2017-07-03 14:46:46,480] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper:ForkJoinPool-2-worker-13)
[2017-07-03 14:46:46,480] INFO Client environment:os.name=Linux (org.apache.zookeeper.ZooKeeper:ForkJoinPool-2-worker-13)
[2017-07-03 14:46:46,480] INFO Client environment:os.arch=amd64 (org.apache.zookeeper.ZooKeeper:ForkJoinPool-2-worker-13)
[2017-07-03 14:46:46,480] INFO Client environment:os.version=4.4.74-boot2docker (org.apache.zookeeper.ZooKeeper:ForkJoinPool-2-worker-13)
[2017-07-03 14:46:46,480] INFO Client environment:user.name=root (org.apache.zookeeper.ZooKeeper:ForkJoinPool-2-worker-13)
[2017-07-03 14:46:46,480] INFO Client environment:user.home=/root (org.apache.zookeeper.ZooKeeper:ForkJoinPool-2-worker-13)
[2017-07-03 14:46:46,480] INFO Client environment:user.dir=/marathon (org.apache.zookeeper.ZooKeeper:ForkJoinPool-2-worker-13)
[2017-07-03 14:46:46,480] INFO Client environment:os.memory.free=26MB (org.apache.zookeeper.ZooKeeper:ForkJoinPool-2-worker-13)
[2017-07-03 14:46:46,482] INFO Client environment:os.memory.max=955MB (org.apache.zookeeper.ZooKeeper:ForkJoinPool-2-worker-13)
[2017-07-03 14:46:46,483] INFO Client environment:os.memory.total=60MB (org.apache.zookeeper.ZooKeeper:ForkJoinPool-2-worker-13)
[2017-07-03 14:46:46,484] INFO Initiating client connection, connectString=minimesos-zookeeper:2181 sessionTimeout=10000 watcher=org.apache.curator.ConnectionState@55d27918 (org.apache.zookeeper.ZooKeeper:ForkJoinPool-2-worker-13)
[2017-07-03 14:46:46,552] INFO Opening socket connection to server minimesos-zookeeper/172.17.0.3:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn:ForkJoinPool-2-worker-13-SendThread(minimesos-zookeeper:2181))
[2017-07-03 14:46:46,576] INFO Socket connection established to minimesos-zookeeper/172.17.0.3:2181, initiating session (org.apache.zookeeper.ClientCnxn:ForkJoinPool-2-worker-13-SendThread(minimesos-zookeeper:2181))
[2017-07-03 14:46:46,590] INFO Session establishment complete on server minimesos-zookeeper/172.17.0.3:2181, sessionid = 0x15d08ead42c0004, negotiated timeout = 10000 (org.apache.zookeeper.ClientCnxn:ForkJoinPool-2-worker-13-SendThread(minimesos-zookeeper:2181))
[2017-07-03 14:46:46,602] INFO State change: CONNECTED (org.apache.curator.framework.state.ConnectionStateManager:ForkJoinPool-2-worker-13-EventThread)
[2017-07-03 14:46:46,687] WARN The version of ZooKeeper being used doesn't support Container nodes. CreateMode.PERSISTENT will be used instead. (org.apache.curator.utils.ZKPaths:Curator-Framework-0)
[2017-07-03 14:46:46,723] INFO Elected (LeaderLatchListener Interface) (mesosphere.marathon.core.election.impl.CuratorElectionService:pool-1-thread-1)
[2017-07-03 14:46:46,731] INFO As new leader running the driver (mesosphere.marathon.MarathonSchedulerService$$EnhancerByGuice$$c165252e:pool-1-thread-1)
[2017-07-03 14:46:46,737] INFO Initiating client connection, connectString=minimesos-zookeeper:2181 sessionTimeout=10000 watcher=com.twitter.zk.EventBroker@55909feb (org.apache.zookeeper.ZooKeeper:pool-1-thread-1)
[2017-07-03 14:46:46,758] INFO Opening socket connection to server minimesos-zookeeper/172.17.0.3:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn:pool-1-thread-1-SendThread(minimesos-zookeeper:2181))
[2017-07-03 14:46:46,763] INFO Socket connection established to minimesos-zookeeper/172.17.0.3:2181, initiating session (org.apache.zookeeper.ClientCnxn:pool-1-thread-1-SendThread(minimesos-zookeeper:2181))
[2017-07-03 14:46:46,766] INFO Session establishment complete on server minimesos-zookeeper/172.17.0.3:2181, sessionid = 0x15d08ead42c0005, negotiated timeout = 10000 (org.apache.zookeeper.ClientCnxn:pool-1-thread-1-SendThread(minimesos-zookeeper:2181))
[2017-07-03 14:46:46,871] INFO Migration successfully applied for version Version(1, 3, 5) (mesosphere.marathon.state.Migration:ForkJoinPool-2-worker-3)
[2017-07-03 14:46:46,872] INFO Call preDriverStarts callbacks on EntityStoreCache(MarathonStore(app:)), EntityStoreCache(MarathonStore(group:)), EntityStoreCache(MarathonStore(deployment:)), EntityStoreCache(MarathonStore(framework:)), EntityStoreCache(MarathonStore(taskFailure:)), EntityStoreCache(MarathonStore(events:)) (mesosphere.marathon.MarathonSchedulerService$$EnhancerByGuice$$c165252e:pool-1-thread-1)
[2017-07-03 14:46:46,893] INFO Finished preDriverStarts callbacks (mesosphere.marathon.MarathonSchedulerService$$EnhancerByGuice$$c165252e:pool-1-thread-1)
[2017-07-03 14:46:46,932] INFO no interest in offers for reservation reconciliation anymore. (mesosphere.marathon.core.matcher.reconcile.impl.OffersWantedForReconciliationActor:marathon-akka.actor.default-dispatcher-4)
[2017-07-03 14:46:46,935] INFO ExpungeOverdueLostTasksActor has started (mesosphere.marathon.core.task.jobs.impl.ExpungeOverdueLostTasksActor:marathon-akka.actor.default-dispatcher-9)
[2017-07-03 14:46:46,943] INFO started RateLimiterActor (mesosphere.marathon.core.launchqueue.impl.RateLimiterActor:marathon-akka.actor.default-dispatcher-8)
[2017-07-03 14:46:46,959] INFO TaskTrackerActor is starting. Task loading initiated. (mesosphere.marathon.core.task.tracker.impl.TaskTrackerActor:marathon-akka.actor.default-dispatcher-5)
[2017-07-03 14:46:46,960] INFO Started. Will remain interested in offer reconciliation for 17500 milliseconds when needed. (mesosphere.marathon.core.matcher.reconcile.impl.OffersWantedForReconciliationActor:marathon-akka.actor.default-dispatcher-4)
[2017-07-03 14:46:46,966] INFO Create new Scheduler Driver with frameworkId: None and scheduler mesosphere.marathon.core.heartbeat.MesosHeartbeatMonitor$$EnhancerByGuice$$18028e64@42383cb0 (mesosphere.marathon.MarathonSchedulerDriver$:pool-1-thread-1)
[2017-07-03 14:46:46,983] INFO All actors active:
* Actor[akka://marathon/user/taskTracker#-572049485]
* Actor[akka://marathon/user/offerMatcherLaunchTokens#-396922016]
* Actor[akka://marathon/user/reviveOffersWhenWanted#-1270188931]
* Actor[akka://marathon/user/launchQueue#-357430807]
* Actor[akka://marathon/user/offersWantedForReconciliation#352720590]
* Actor[akka://marathon/user/groupManager#103236799]
* Actor[akka://marathon/user/taskKillServiceActor#-1484763378]
* Actor[akka://marathon/user/rateLimiter#1530916594]
* Actor[akka://marathon/user/offerMatcherManager#539319868]
* Actor[akka://marathon/user/expungeOverdueLostTasks#-520277771]
* Actor[akka://marathon/user/killOverdueStagedTasks#1286174505] (mesosphere.marathon.core.leadership.impl.LeadershipCoordinatorActor:marathon-akka.actor.default-dispatcher-8)
[2017-07-03 14:46:47,049] INFO About to load 0 tasks (mesosphere.marathon.core.task.tracker.impl.TaskLoaderImpl:ForkJoinPool-2-worker-5)
[2017-07-03 14:46:47,050] INFO Loaded 0 tasks (mesosphere.marathon.core.task.tracker.impl.TaskLoaderImpl:ForkJoinPool-2-worker-13)
[2017-07-03 14:46:47,082] INFO Task loading complete. (mesosphere.marathon.core.task.tracker.impl.TaskTrackerActor:marathon-akka.actor.default-dispatcher-2)
[2017-07-03 14:46:47,149] INFO interested in offers for reservation reconciliation because of becoming leader (until 2017-07-03T14:47:04.460Z) (mesosphere.marathon.core.matcher.reconcile.impl.OffersWantedForReconciliationActor:marathon-akka.actor.default-dispatcher-4)
[2017-07-03 14:46:47,156] INFO Received offers WANTED notification (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-4)
[2017-07-03 14:46:47,157] INFO => revive offers NOW, canceling any scheduled revives (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-4)
[2017-07-03 14:46:47,174] INFO 2 further revives still needed. Repeating reviveOffers according to --revive_offers_repetitions 3 (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-4)
[2017-07-03 14:46:47,178] INFO => Schedule next revive at 2017-07-03T14:46:52.157Z in 4982 milliseconds, adhering to --min_revive_offers_interval 5000 (ms) (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-4)
I0703 14:46:47.354720    51 sched.cpp:226] Version: 1.0.1
2017-07-03 14:46:47,358:1(0x7f57dd241700):ZOO_INFO@log_env@726: Client environment:zookeeper.version=zookeeper C client 3.4.8
2017-07-03 14:46:47,358:1(0x7f57dd241700):ZOO_INFO@log_env@730: Client environment:host.name=02d73624fccf
2017-07-03 14:46:47,358:1(0x7f57dd241700):ZOO_INFO@log_env@737: Client environment:os.name=Linux
2017-07-03 14:46:47,358:1(0x7f57dd241700):ZOO_INFO@log_env@738: Client environment:os.arch=4.4.74-boot2docker
2017-07-03 14:46:47,358:1(0x7f57dd241700):ZOO_INFO@log_env@739: Client environment:os.version=#1 SMP Mon Jun 26 18:01:14 UTC 2017
2017-07-03 14:46:47,358:1(0x7f57dd241700):ZOO_INFO@log_env@747: Client environment:user.name=(null)
2017-07-03 14:46:47,358:1(0x7f57dd241700):ZOO_INFO@log_env@755: Client environment:user.home=/root
2017-07-03 14:46:47,358:1(0x7f57dd241700):ZOO_INFO@log_env@767: Client environment:user.dir=/marathon
2017-07-03 14:46:47,358:1(0x7f57dd241700):ZOO_INFO@zookeeper_init@800: Initiating client connection, host=minimesos-zookeeper:2181 sessionTimeout=10000 watcher=0x7f57e6b31d80 sessionId=0 sessionPasswd=<null> context=0x7f57f4020850 flags=0
[2017-07-03 14:46:47,362] INFO Starting scheduler actor (mesosphere.marathon.MarathonSchedulerActor:marathon-akka.actor.default-dispatcher-4)
2017-07-03 14:46:47,369:1(0x7f57dba3e700):ZOO_INFO@check_events@1728: initiated connection to server [172.17.0.3:2181]
[2017-07-03 14:46:47,366] INFO Became active. Accepting event streaming requests. (mesosphere.marathon.core.event.impl.stream.HttpEventStreamActor:marathon-akka.actor.default-dispatcher-4)
2017-07-03 14:46:47,370:1(0x7f57dba3e700):ZOO_INFO@check_events@1775: session establishment complete on server [172.17.0.3:2181], sessionId=0x15d08ead42c0006, negotiated timeout=10000
I0703 14:46:47.370606    70 group.cpp:349] Group process (group(1)@172.17.0.6:45106) connected to ZooKeeper
I0703 14:46:47.370653    70 group.cpp:837] Syncing group operations: queue size (joins, cancels, datas) = (0, 0, 0)
I0703 14:46:47.370676    70 group.cpp:427] Trying to create path '/mesos' in ZooKeeper
[2017-07-03 14:46:47,384] INFO Reset offerLeadership backoff (mesosphere.marathon.core.election.impl.ExponentialBackoff:pool-1-thread-1)
[2017-07-03 14:46:47,389] INFO Scheduler actor ready (mesosphere.marathon.MarathonSchedulerActor:marathon-akka.actor.default-dispatcher-4)
I0703 14:46:47.408471    70 detector.cpp:152] Detected a new leader: (id='0')
I0703 14:46:47.409636    70 group.cpp:706] Trying to get '/mesos/json.info_0000000000' in ZooKeeper
I0703 14:46:47.410480    70 zookeeper.cpp:259] A new leading master (UPID=master@172.17.0.5:5050) is detected
I0703 14:46:47.410567    70 sched.cpp:330] New master detected at master@172.17.0.5:5050
I0703 14:46:47.411077    70 sched.cpp:341] No credentials provided. Attempting to register without authentication
I0703 14:46:47.412653    70 sched.cpp:743] Framework registered with 842438a6-613b-4b23-b86e-1a3aa63e04c6-0000
[2017-07-03 14:46:47,413] INFO Creating tombstone for old twitter commons leader election (mesosphere.marathon.core.election.impl.CuratorElectionService:pool-1-thread-1)
[2017-07-03 14:46:47,487] INFO Registered as 842438a6-613b-4b23-b86e-1a3aa63e04c6-0000 to master '842438a6-613b-4b23-b86e-1a3aa63e04c6' (mesosphere.marathon.MarathonScheduler$$EnhancerByGuice$$2a7b5c44:Thread-14)
[2017-07-03 14:46:47,699] INFO Store framework id: value: "842438a6-613b-4b23-b86e-1a3aa63e04c6-0000"
 (mesosphere.util.state.FrameworkIdUtil:Thread-14)
[2017-07-03 14:46:47,740] INFO Received reviveOffers notification: SchedulerRegisteredEvent (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-6)
[2017-07-03 14:46:47,917] INFO Binding mesosphere.marathon.api.v2.AppsResource to GuiceManagedComponentProvider with the scope "Singleton" (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:47,946] INFO Binding mesosphere.marathon.api.v2.TasksResource to GuiceManagedComponentProvider with the scope "Singleton" (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:47,950] INFO Binding mesosphere.marathon.api.v2.EventSubscriptionsResource to GuiceManagedComponentProvider with the scope "Singleton" (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:47,951] INFO Event notification disabled. (mesosphere.marathon.core.event.EventModule:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:47,961] INFO Binding mesosphere.marathon.api.v2.QueueResource to GuiceManagedComponentProvider with the scope "Singleton" (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:47,973] INFO Binding mesosphere.marathon.api.v2.GroupsResource to GuiceManagedComponentProvider with the scope "Singleton" (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:47,983] INFO Binding mesosphere.marathon.api.v2.InfoResource to GuiceManagedComponentProvider with the scope "Singleton" (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:47,983] INFO Binding mesosphere.marathon.api.v2.LeaderResource to GuiceManagedComponentProvider with the scope "Singleton" (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:47,987] INFO Binding mesosphere.marathon.api.v2.DeploymentsResource to GuiceManagedComponentProvider with the scope "Singleton" (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:48,005] INFO Binding mesosphere.marathon.api.v2.ArtifactsResource to GuiceManagedComponentProvider with the scope "Singleton" (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:48,006] INFO Binding mesosphere.marathon.api.v2.SchemaResource to GuiceManagedComponentProvider with the scope "Singleton" (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:48,012] INFO Binding mesosphere.marathon.api.v2.PluginsResource to GuiceManagedComponentProvider with the scope "Singleton" (com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:48,013] INFO Loading plugins implementing 'mesosphere.marathon.plugin.http.HttpRequestHandler' from these urls: [file:/marathon/plugin/plugins_2.11-0.1-SNAPSHOT.jar] (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:48,013] INFO Found 0 plugins. (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:48,041] INFO Started o.e.j.s.ServletContextHandler@50fb3a{/,null,AVAILABLE} (org.eclipse.jetty.server.handler.ContextHandler:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:48,067] INFO Started ServerConnector@223b9714{HTTP/1.1,[http/1.1]}{0.0.0.0:8080} (org.eclipse.jetty.server.ServerConnector:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:48,067] INFO Started @10749ms (org.eclipse.jetty.server.Server:HttpService$$EnhancerByGuice$$81207d13 STARTING)
[2017-07-03 14:46:48,068] INFO All services up and running. (mesosphere.marathon.Main$:main)
[2017-07-03 14:46:48,390] INFO 172.17.0.2 - - [03/Jul/2017:14:46:48 +0000] "GET //172.17.0.6:8080/v2/apps HTTP/1.1" 200 31 "-" "unirest-java/1.3.11" 17 (mesosphere.chaos.http.ChaosRequestLog$$EnhancerByGuice$$6066f6de:qtp397639322-30)
[2017-07-03 14:46:50,164] INFO Loading plugins implementing 'mesosphere.marathon.plugin.validation.RunSpecValidator' from these urls: [file:/marathon/plugin/plugins_2.11-0.1-SNAPSHOT.jar] (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:qtp397639322-32)
[2017-07-03 14:46:50,165] INFO Found 0 plugins. (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:qtp397639322-32)
[2017-07-03 14:46:50,173] INFO Upgrade group id:/ version:2017-07-03T14:46:50.165Z with force:false (mesosphere.marathon.core.group.impl.GroupManagerActor:marathon-akka.actor.default-dispatcher-4)
[2017-07-03 14:46:50,192] INFO Take next configured free port: 10000 (mesosphere.marathon.core.group.impl.GroupManagerActor:marathon-akka.actor.default-dispatcher-4)
[2017-07-03 14:46:50,209] INFO [/weave-scope]: new app detected (mesosphere.marathon.upgrade.GroupVersioningUtil$:marathon-akka.actor.default-dispatcher-4)
[2017-07-03 14:46:50,480] INFO Computed new deployment plan:
DeploymentPlan 2017-07-03T14:46:50.165Z
step 1:
  * Start(App(/weave-scope, image="weaveworks/scope:0.13.1", args="--probe.docker true")), instances=0)
step 2:
  * Scale(App(/weave-scope, image="weaveworks/scope:0.13.1", args="--probe.docker true")), instances=1)
 (mesosphere.marathon.core.group.impl.GroupManagerActor:marathon-akka.actor.default-dispatcher-4)
[2017-07-03 14:46:50,481] INFO Deploy plan with force=false:
DeploymentPlan 2017-07-03T14:46:50.165Z
step 1:
  * Start(App(/weave-scope, image="weaveworks/scope:0.13.1", args="--probe.docker true")), instances=0)
step 2:
  * Scale(App(/weave-scope, image="weaveworks/scope:0.13.1", args="--probe.docker true")), instances=1)
  (mesosphere.marathon.MarathonSchedulerService$$EnhancerByGuice$$c165252e:marathon-akka.actor.default-dispatcher-4)
[2017-07-03 14:46:50,506] INFO [/weave-scope] storing new app version 2017-07-03T14:46:50.165Z (mesosphere.marathon.core.group.impl.GroupManagerActor:marathon-akka.actor.default-dispatcher-6)
[2017-07-03 14:46:50,571] INFO Updated groups/apps according to deployment plan d5ecd8db-3c04-4473-bb4b-6ea366b4777c (mesosphere.marathon.core.group.impl.GroupManagerActor:marathon-akka.actor.default-dispatcher-2)
[2017-07-03 14:46:50,572] INFO Deployment acknowledged. Waiting to get processed:
DeploymentPlan 2017-07-03T14:46:50.165Z
step 1:
  * Start(App(/weave-scope, image="weaveworks/scope:0.13.1", args="--probe.docker true")), instances=0)
step 2:
  * Scale(App(/weave-scope, image="weaveworks/scope:0.13.1", args="--probe.docker true")), instances=1)
 (mesosphere.marathon.core.group.impl.GroupManagerActor:marathon-akka.actor.default-dispatcher-2)
[2017-07-03 14:46:50,592] INFO Starting app /weave-scope (mesosphere.marathon.SchedulerActions:marathon-akka.actor.default-dispatcher-6)
[2017-07-03 14:46:50,601] INFO Already running 0 instances of /weave-scope. Not scaling. (mesosphere.marathon.SchedulerActions:marathon-akka.actor.default-dispatcher-6)
[2017-07-03 14:46:50,606] INFO Successfully started 0 instances of /weave-scope (mesosphere.marathon.upgrade.AppStartActor:marathon-akka.actor.default-dispatcher-2)
[2017-07-03 14:46:50,644] INFO Started taskLaunchActor for /weave-scope version 2017-07-03T14:46:50.165Z with initial count 1 (mesosphere.marathon.core.launchqueue.impl.TaskLauncherActor:marathon-akka.actor.default-dispatcher-4)
[2017-07-03 14:46:50,666] INFO 172.17.0.2 - - [03/Jul/2017:14:46:49 +0000] "POST //172.17.0.6:8080/v2/apps HTTP/1.1" 201 1170 "-" "Java/1.8.0_111-internal" 93 (mesosphere.chaos.http.ChaosRequestLog$$EnhancerByGuice$$6066f6de:qtp397639322-32)
[2017-07-03 14:46:50,681] INFO activating matcher ActorOfferMatcher(Actor[akka://marathon/user/launchQueue/1/0-weave-scope#-1589620412]). (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-3)
[2017-07-03 14:46:50,682] INFO Received offers WANTED notification (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-3)
[2017-07-03 14:46:52,185] INFO Received TimedCheck (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-8)
[2017-07-03 14:46:52,196] INFO => revive offers NOW, canceling any scheduled revives (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-8)
[2017-07-03 14:46:52,196] INFO 2 further revives still needed. Repeating reviveOffers according to --revive_offers_repetitions 3 (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-8)
[2017-07-03 14:46:52,197] INFO => Schedule next revive at 2017-07-03T14:46:57.184Z in 4999 milliseconds, adhering to --min_revive_offers_interval 5000 (ms) (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-8)
[2017-07-03 14:46:52,200] INFO Loading plugins implementing 'mesosphere.marathon.plugin.task.RunSpecTaskProcessor' from these urls: [file:/marathon/plugin/plugins_2.11-0.1-SNAPSHOT.jar] (mesosphere.marathon.core.plugin.impl.PluginManagerImpl:marathon-akka.actor.default-dispatcher-8)
[2017-07-03 14:46:52,215] ERROR Plugin not found: PluginDefinition(envVarExtender,mesosphere.marathon.plugin.task.RunSpecTaskProcessor,mesosphere.marathon.example.plugin.env.EnvVarExtenderPlugin,None,Some({"env":{"foo":"bar","key":"value"}}),None) (akka.actor.OneForOneStrategy:marathon-akka.actor.default-dispatcher-3)
mesosphere.marathon.WrongConfigurationException: Plugin not found: PluginDefinition(envVarExtender,mesosphere.marathon.plugin.task.RunSpecTaskProcessor,mesosphere.marathon.example.plugin.env.EnvVarExtenderPlugin,None,Some({"env":{"foo":"bar","key":"value"}}),None)
    at mesosphere.marathon.core.plugin.impl.PluginManagerImpl$$anonfun$2$$anonfun$apply$3.apply(PluginManagerImpl.scala:52)
    at mesosphere.marathon.core.plugin.impl.PluginManagerImpl$$anonfun$2$$anonfun$apply$3.apply(PluginManagerImpl.scala:52)
    at scala.Option.getOrElse(Option.scala:121)
    at mesosphere.marathon.core.plugin.impl.PluginManagerImpl$$anonfun$2.apply(PluginManagerImpl.scala:52)
    at mesosphere.marathon.core.plugin.impl.PluginManagerImpl$$anonfun$2.apply(PluginManagerImpl.scala:45)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
    at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
    at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
    at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
    at scala.collection.AbstractTraversable.map(Traversable.scala:104)
    at mesosphere.marathon.core.plugin.impl.PluginManagerImpl.load(PluginManagerImpl.scala:45)
    at mesosphere.marathon.core.plugin.impl.PluginManagerImpl.mesosphere$marathon$core$plugin$impl$PluginManagerImpl$$loadAndAdd$1(PluginManagerImpl.scala:65)
    at mesosphere.marathon.core.plugin.impl.PluginManagerImpl$$anonfun$plugins$3.apply(PluginManagerImpl.scala:73)
    at mesosphere.marathon.core.plugin.impl.PluginManagerImpl$$anonfun$plugins$3.apply(PluginManagerImpl.scala:73)
    at scala.Option.getOrElse(Option.scala:121)
    at mesosphere.marathon.core.plugin.impl.PluginManagerImpl.plugins(PluginManagerImpl.scala:73)
    at mesosphere.marathon.core.launcher.impl.TaskOpFactoryImpl.appTaskProc$lzycompute(TaskOpFactoryImpl.scala:35)
    at mesosphere.marathon.core.launcher.impl.TaskOpFactoryImpl.appTaskProc(TaskOpFactoryImpl.scala:35)
    at mesosphere.marathon.core.launcher.impl.TaskOpFactoryImpl.inferNormalTaskOp(TaskOpFactoryImpl.scala:50)
    at mesosphere.marathon.core.launcher.impl.TaskOpFactoryImpl.buildTaskOp(TaskOpFactoryImpl.scala:43)
    at mesosphere.marathon.core.launchqueue.impl.TaskLauncherActor$$anonfun$receiveProcessOffers$1.applyOrElse(TaskLauncherActor.scala:361)
    at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170)
    at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
    at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
    at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
    at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
    at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
    at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
    at akka.actor.Actor$class.aroundReceive(Actor.scala:484)
    at mesosphere.marathon.core.launchqueue.impl.TaskLauncherActor.aroundReceive(TaskLauncherActor.scala:75)
    at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
    at akka.actor.ActorCell.invoke(ActorCell.scala:495)
    at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
    at akka.dispatch.Mailbox.run(Mailbox.scala:224)
    at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
    at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
    at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
    at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
    at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)```
unterstein commented 7 years ago

Hey @kohend,

thanks for you pushed docker image dkohen/debug:minimesos. I started searching in your output --plugin_dir /marathon/plugin --plugin_conf /marathon/plugin-conf.json and will sum up what I found inside your image:

root@7f3cc99cbf4e:/marathon# ls -la /marathon/plugin
total 12
drwxr-xr-x  2 root root 4096 Jul 13 07:58 .
drwxr-xr-x 17 root root 4096 Jul 13 07:58 ..
-rw-r--r--  1 root root  289 Jul  3 13:37 plugins_2.11-0.1-SNAPSHOT.jar
root@7f3cc99cbf4e:/marathon# cd plugin
root@7f3cc99cbf4e:/marathon/plugin# unzip plugins_2.11-0.1-SNAPSHOT.jar 
Archive:  plugins_2.11-0.1-SNAPSHOT.jar
  inflating: META-INF/MANIFEST.MF    
root@7f3cc99cbf4e:/marathon/plugin# ls
META-INF  plugins_2.11-0.1-SNAPSHOT.jar
root@7f3cc99cbf4e:/marathon/plugin# ls META-INF/
MANIFEST.MF
root@7f3cc99cbf4e:/marathon/plugin# cat /marathon/plugin-conf.json
{
  "plugins": {
    "envVarExtender": {
      "plugin": "mesosphere.marathon.plugin.task.RunSpecTaskProcessor",
      "implementation": "mesosphere.marathon.example.plugin.env.EnvVarExtenderPlugin",
      "configuration": {
        "env": {
            "foo": "bar",
            "key": "value"
        }
      }
    }
  }
}
root@7f3cc99cbf4e:/marathon/plugin# 

As far as I can see, the provided plugin jar file is nearly empty. So probably something with the build process is not working properly. Can you verify that this file is empty (except the MANIFEST) on your box as well and explain how you build the jar?

Thanks Johannes

kohend commented 7 years ago

I built it using sbt clean pack as the readme says, I can confirm it is indeed nearly empty on my box as well (I assumed the build to work if it doesn't visibly fail).
On the fork I checked (git@github.com:blackgold/marathon-vault-plugin.git) it does have all classes in the jar file but it is changing a lot of other things so I think getting the sample code to build properly would be best and hopefully I could take it from there myself.

sudocurse commented 7 years ago

@kohend could you post your build.sbt? When I had the same issue, I needed to add sbt pack settings to my build file so the pack command would work as intended. (Specifically, I appended the following line):

packAutoSettings
kohend commented 7 years ago

@postmillenial The packAutoSettings is in the project.sbt file, I didn't change that.

sudocurse commented 7 years ago

@kohend Not sure if "project.sbt" was intentional but I'd check if it's in a build definition file that's actually being used, and that the other parameters in that file are correct.

kohend commented 6 years ago

Turns out that the jar file I used was not the correct one for the plugin, env/target/scala-2.11/env-plugin_2.11-1.0.jar worked.

bennyyu686 commented 6 years ago

● marathon.service - Scheduler for Apache Mesos Loaded: loaded (/lib/systemd/system/marathon.service; enabled; vendor preset: enabled) Active: active (running) since 五 2017-11-24 14:52:55 CST; 1s ago Process: 28396 ExecStartPre=/bin/chmod 755 /run/marathon (code=exited, status=0/SUCCESS) Process: 28389 ExecStartPre=/bin/chown marathon:marathon /run/marathon (code=exited, status=0/SUCCESS) Process: 28382 ExecStartPre=/bin/mkdir -p /run/marathon (code=exited, status=0/SUCCESS) Main PID: 28401 (java) Tasks: 70 Memory: 170.2M CPU: 2.453s CGroup: /system.slice/marathon.service └─28401 java -cp /usr/share/marathon/lib/mesosphere.marathon.marathon-1.5.2.jar:/usr/share/marathon/lib/mesosphere.marathon.plugin-interface-1

11月 24 14:52:55 node3 systemd[1]: Starting Scheduler for Apache Mesos... 11月 24 14:52:55 node3 systemd[1]: Started Scheduler for Apache Mesos. 11月 24 14:52:55 node3 marathon[28401]: No start hook file found ($HOOK_MARATHON_START). Proceeding with the start script.

bennyyu686 commented 6 years ago

root@node3:~# service marathon status ● marathon.service - Scheduler for Apache Mesos Loaded: loaded (/lib/systemd/system/marathon.service; enabled; vendor preset: enabled) Active: activating (auto-restart) (Result: exit-code) since 五 2017-11-24 14:56:05 CST; 50s ago Process: 29029 ExecStart=/usr/share/marathon/bin/marathon (code=exited, status=1/FAILURE) Process: 29026 ExecStartPre=/bin/chmod 755 /run/marathon (code=exited, status=0/SUCCESS) Process: 29019 ExecStartPre=/bin/chown marathon:marathon /run/marathon (code=exited, status=0/SUCCESS) Process: 29011 ExecStartPre=/bin/mkdir -p /run/marathon (code=exited, status=0/SUCCESS) Main PID: 29029 (code=exited, status=1/FAILURE)

11月 24 14:56:05 node3 systemd[1]: marathon.service: Unit entered failed state. 11月 24 14:56:05 node3 systemd[1]: marathon.service: Failed with result 'exit-code'.

bennyyu686 commented 6 years ago

where is the hook file ?

bennyyu686 commented 6 years ago

when I run commond of service marathon restart