search

ywilkof / spark-jobs-rest-client

Fluent client for interacting with Spark Standalone Mode's Rest API for submitting, killing and monitoring the state of jobs.
Apache License 2.0
109 stars 58 forks source link

Get status response doesnt support Queued driver state #12

Closed avinashmeda closed 7 years ago

avinashmeda commented 7 years ago

Using mesos as the spark resource master. When i check for the status immediately after submitting the job the driver state in the response is "QUEUED" which is not a enum defined and hence it throws this exception

com.github.ywilkof.sparkrestclient.FailedSparkRequestException: com.fasterxml.jackson.databind.exc.InvalidFormatException: Can not construct instance of com.github.ywilkof.sparkrestclient.DriverState from String value 'QUEUED': value not one of declared Enum instance names: [FINISHED, ERROR, UNKNOWN, SUBMITTED, RELAUNCHING, FAILED, RUNNING, KILLED] at [Source: { "action" : "SubmissionStatusResponse", "driverState" : "QUEUED", "serverSparkVersion" : "2.1.1", "submissionId" : "driver-20170512125948-0019", "success" : true }; line: 2, column: 40] (through reference chain: com.github.ywilkof.sparkrestclient.JobStatusResponse["driverState"]) at com.github.ywilkof.sparkrestclient.HttpRequestUtil.executeHttpMethodAndGetResponse(HttpRequestUtil.java:21) at com.github.ywilkof.sparkrestclient.JobStatusRequestSpecificationImpl.withSubmissionIdFullResponse(JobStatusRequestSpecificationImpl.java:40) at com.github.ywilkof.sparkrestclient.JobStatusRequestSpecificationImpl.withSubmissionId(JobStatusRequestSpecificationImpl.java:24)

ywilkof commented 7 years ago

Hi @avinashmeda, A new version 1.3.3 should be available soon supporting this state. Please tell me if it works now. I have actually only tested the client with Spark Standalone so I am interested to know if it works with Mesos as well.

avinashmeda commented 7 years ago

Yes it does work on media.thanks...

avinashmeda commented 7 years ago

Mesos*