mesos / chronos

Fault tolerant job scheduler for Mesos which handles dependencies and ISO8601 based schedules
http://mesos.github.io/chronos/
Apache License 2.0
4.39k stars 529 forks source link

Unable to post jobs #853

Open fr0stxm opened 7 years ago

fr0stxm commented 7 years ago

Hi,

I have Chronos running alongside Mesos (Panteras) on the same machine. I can connect to the UI Interface from my machine, but if I attempt to use the Restful API I get Error 52 (curl: (52) Empty reply from server)

Full start options: java -Djava.library.path=/usr/local/lib:/usr/lib64:/usr/lib -Djava.util.logging.SimpleFormatter.format=%2$s %5$s%6$s%n -Xmx512m -cp /usr/bin/chronos org.apache.mesos.chronos.scheduler.Main --master zk://ip-10-6-141-18:2181,ip-10-6-188-71:2181,ip-10-6-223-62:2181/mesos --zk_hosts ip-10-6-141-18:2181,ip-10-6-188-71:2181,ip-10-6-223-62:2181 --http_address 0.0.0.0 --http_port 4400

Java version inside the panteras container java -version openjdk version "9-internal" OpenJDK Runtime Environment (build 9-internal+0-2016-04-14-195246.buildd.src) OpenJDK 64-Bit Server VM (build 9-internal+0-2016-04-14-195246.buildd.src, mixed mode)

Panteras: 0.3.5

I can access the UI on http://10.6.141.180:4400; as well as Marathon.

iptables is pretty flexible (these are stock)

Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination
DOCKER     all  --  anywhere             anywhere             ADDRTYPE match dst-type LOCAL

Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
DOCKER     all  --  anywhere            !127.0.0.0/8          ADDRTYPE match dst-type LOCAL

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination
MASQUERADE  all  --  ip-172-17-0-0.eu-west-1.compute.internal/16  anywhere

Chain DOCKER (2 references)
target     prot opt source               destination
RETURN     all  --  anywhere             anywhere
fr0stxm commented 7 years ago

Example command being used:

curl -L -X GET http://10.6.223.62:4400/scheduler/jobs

curl -L -X GET http://10.6.141.180:4400/v1/scheduler/jobs (NB: edited to reflect correct IP as per kevtaylor comment)

I've tried with /v1/ added to the initial URL also.

kevtaylor commented 7 years ago

Shouldn't it be the same consistent endpoint as the ui

http://10.6.141.180:4400/v1/scheduler/jobs
fr0stxm commented 7 years ago

Hey @kevtaylor

I have a number of machines in the same state, I copied the wrong IP down for my curl command, but the same output exists.. I'll add an amendment to the initial curl command.

kevtaylor commented 7 years ago

Perhaps --http_address = 0.0.0.0 could be the factor - you might also wish to try localhost:4400 first and see whether the port is actual available for listening.

fr0stxm commented 7 years ago

Hey @kevtaylor so I can connect to the HTTP UI from another machine, as well as curl it, telnet it.

I've attempted to run:

curl -L -X GET http://localhost:4400/v1/scheduler/jobs
curl -L -X GET http://127.0.0.1:4400/v1/scheduler/jobs
curl -L -X GET http://localhost:4400/scheduler/jobs
curl -L -X GET http://l127.0.0.1:4400/v1/scheduler/jobs

All resulted in the same outcome. To show that I can connect locally to the UI I've ran:

curl http://localhost:4400 which returns the HTML elements; coupled with -I I get a 200 response

HTTP/1.1 200 OK
Content-Type: text/html
Content-Length: 3492
Last-Modified: Wed, 07 Oct 2015 11:01:56 GMT
Server: Jetty(8.y.z-SNAPSHOT)
kevtaylor commented 7 years ago

When I perform the same approach here, I get a different version. As far as I can recall we are using v3.0 of chronos, which gives this

HTTP/1.1 200 OK
Date: Wed, 23 Aug 2017 12:35:19 GMT
Access-Control-Allow-Origin: *
Content-Type: text/html
Vary: Accept-Encoding, User-Agent
Content-Length: 698
Server: Jetty(9.3.z-SNAPSHOT)

The jetty version is quite a bit newer and this comes back from the straight curl

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="utf-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1" />
  <link rel="stylesheet" href="assets/fonts/RobotoMono-Bold.ttf" />
  <link rel="stylesheet" href="assets/css/bootstrap.min.css" />
  <link rel="stylesheet" href="assets/css/bootstrap-theme.css" />
  <link rel="stylesheet" href="assets/css/font-awesome.min.css" />
  <link rel="stylesheet" href="assets/css/react-select.min.css" />
  <link rel="stylesheet" href="assets/css/jsoneditor.min.css" />
  <title>CHRONOS</title>
</head>
<body>
  <div id="root" />
  <script type="text/javascript" src="assets/js/bundle.js" charset="utf-8"></script>
</body>
</html>
fr0stxm commented 7 years ago

v 0.3.5 of Panteras but unable to determine the version of chronos, chronos --help or --v / --version doesn't return