Closed hyper0x closed 9 years ago
+1. I'm running into this issue was well. I have to go into the Bamboo UI and edit and close (save is not necessary) the application before haproxy/bamboo picks up the restarted app.
This should be an easy fix. I will look into it.
On 30 Mar 2015, at 08:44, Vestaxis notifications@github.com wrote:
+1. I'm running into this issue was well. I have to go into the Bamboo UI and edit and close (save is not necessary) the application before haproxy/bamboo picks up the restarted app.
— Reply to this email directly or view it on GitHub https://github.com/QubitProducts/bamboo/issues/101#issuecomment-87581200.
@activars Great!
I found it accept a json-formatted response from [marathon_endpoint]/v2/tasks. But, the actual situation is, it will receive a plaintext after GET from the url. Is that the reason?
See: https://github.com/QubitProducts/bamboo/blob/master/services/marathon/marathon.go#L119
You have to set content type to application/json to get the json format.
@activars But the response is plaintext in Marathon 0.8.1.
I tested 0.8.1, so the default tasks API behaviour has changed.
As I remember with Marathon 0.6.* without "Accept: application/json", it returns tasks string with line breaks.
Marathon 0.7.* gives JSON response without specifying 'Accept: application/json'. I believe this tasks API behaviour has rollbacked to 0.6 in recent 0.8.1 release. I was afraid of those API behaviour changes, so added the http header in all marathon api request to enforce JSON request in previous release.
I think the line you quoted doesn't affect Bamboo:
$> curl -H "Accept: application/json" http://10.141.141.10:8080/v2/tasks
{"tasks":[]}%
Did you enable the http_callback on Marathon?
I got it, and I tried it. As you said, it returned a json response with 'Accept: application/json'. Thank you for your answer. :+1:
btw, What about current issue?
@activars I will code to resolve this problem. But may be I start Marathon w/o its callback system.
@activars I enabled Marathon callback system, but the problem is still there.
I run (Bomboo can cover it):
curl -H "Content-Type:application/json" -d "" "http://<marathon_ip>:<marathon_port>/v2/eventSubscriptions?callbackUrl=<bamboo_ip>:<bamboo_port>/api/marathon/event_callback"
And get the result:
{"clientIp":"<marathon_ip>","callbackUrl":"<bamboo_ip>:<bamboo_port>/api/marathon/event_callback","eventType":"subscribe_event","timestamp":"2015-04-01T07:24:17.592Z"}
Is that right?
But no any additional output while I restart (or scale or new) the app in Marathon. On the other hand, I run follow cmd:
curl -d "{\"EventType\": \"fake event\"}" "http://<bamboo_ip>:<bamboo_port>/api/marathon/event_callback"
Then it seems everything is ok (response: Got it!
).
I guess that is a question of Marathon about configuration.
I have a good solution for this. I would like to get chore #92 merged first and tackle this improvement next. Just being busy solving some internal issues, trying to work on this components in the evening.
:+1: Looking forward to the next release! Working around this is painful
I've worked out a solution. Please test master branch. Could you close the issue if the master branch solves the problem for you?
OK,I will try.
The problem will be resolved via reload haproxy on startup, right? What is the principle?
I resolved this problem, and submitted a pull request to marathon to resolve what it will interrupt the http event sending process if find one of the callback url's format is incorrect.
The PR you mentioned has been merged. The master branch now force an update on Bamboo start.
I konw, but I mean why do you say your solution(force an update on bamboo start) can make bamboo update acl automatically when the app/task in marathon is restarted? This feature has been ok already, isn't it? Do I make myself clear?
I think I was lost in the conversation... my bad. So there are three issues:
It's likely marathon doesn't send an event when application restarted.
Great! That would be perfect. ;-)
As you guess, the event has not been sent. But I already got it. The problem is caused by a number of reasons(me & marathon: https://github.com/mesosphere/marathon/pull/1363)
Thanks. It looks like a problem upstream. I think it's better to solve the issue in Marathon.
On 4 Apr 2015, at 14:23, Robert Hao notifications@github.com wrote:
Great! That would be perfect. ;-)
As you guess, the event has not been sent. But I already got it. The problem is caused by a number of reasons(me & marathon: mesosphere/marathon#1363 https://github.com/mesosphere/marathon/pull/1363)
— Reply to this email directly or view it on GitHub https://github.com/QubitProducts/bamboo/issues/101#issuecomment-89577114.
I agree totally. I will test the newest revision of bamboo at tomorrow and close this issue. Thank you.
Can we get a 0.2.10 release cut for this?
@bacoboy do you mean a release for the 1 & 2 in the following quote (3 is likely to be improvement in Marathon upstream):
- Ensure the http callback indeed happens (Improved by your #105 PR)
- When bamboo started, in some situation, no events happening, the only way to force haproxy configuration changes is to change some record. (this is what i'm fixing, sorry if i misunderstood what you were asking) (Improved by #106 )
- When application restart in Marathon, there bamboo should discover the restart and ensure an update happen as quickly as possible (this is what you were asking)
I'm mostly interested in #106, but there hasn't been a release since Feb. So I'm asking for a more current official release. Thanks.
@bacoboy thanks for reminding me this. I will cut a release today.
Done, here's the release note: https://github.com/QubitProducts/bamboo/releases/tag/v0.2.10
Sweet. Thanks!
As it said in title. Thanks.