search

strepsirrhini-army / chaos-lemur

An adaptation of the Chaos Monkey concept to BOSH releases
Apache License 2.0
62 stars 32 forks source link

401 Unauthorized when trying to destroy #5

Open amulyas opened 9 years ago

amulyas commented 9 years ago

I am able to create the task .. 

[{"status":"IN_PROGRESS","trigger":"MANUAL","start":{"monthValue":9,"hour":19,"minute":52,"second":9,"nano":569000000,"year":2015,"month":"SEPTEMBER","dayOfMonth":9,"dayOfWeek":"WEDNESDAY","dayOfYear":252,"chronology":{"calendarType":"iso8601","id":"ISO"}},"links":[{"rel":"self","href":"http://chaos-lemur.system.xxxx.xxx/task/0"}]}]

but it seems when task is about to get executed its getting 401 Unauthorized?

2015-09-09T12:50:30.50-0700 [App/0]      OUT [CHAOS LEMUR] INFO  Started Application in 6.763 seconds (JVM running for 7.577)
2015-09-09T12:52:09.57-0700 [App/0]      OUT [CHAOS LEMUR] INFO  7ae2c719-34e4-4680-a749-af9840f05509 Beginning run...
2015-09-09T12:52:10.63-0700 [App/0]      ERR Exception in thread "destroyer-1" org.springframework.web.client.HttpClientErrorException: 401 Unauthorized
2015-09-09T12:52:10.63-0700 [App/0]      ERR    at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:91)
2015-09-09T12:52:10.63-0700 [App/0]      ERR    at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:614)
2015-09-09T12:52:10.63-0700 [App/0]      ERR    at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:570)
2015-09-09T12:52:10.63-0700 [App/0]      ERR    at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:545)
2015-09-09T12:52:10.63-0700 [App/0]      ERR    at org.springframework.web.client.RestTemplate.getForObject(RestTemplate.java:253)
2015-09-09T12:52:10.63-0700 [App/0]      ERR    at io.pivotal.strepsirrhini.chaoslemur.infrastructure.StandardDirectorUtils.getDeployments(StandardDirectorUtils.java:99)
2015-09-09T12:52:10.63-0700 [App/0]      ERR    at io.pivotal.strepsirrhini.chaoslemur.infrastructure.AbstractDirectorUtilsInfrastructure.getMembers(AbstractDirectorUtilsInfrastructure.java:42)
2015-09-09T12:52:10.63-0700 [App/0]      ERR    at io.pivotal.strepsirrhini.chaoslemur.Destroyer.doDestroy(Destroyer.java:134)
2015-09-09T12:52:10.63-0700 [App/0]      ERR    at io.pivotal.strepsirrhini.chaoslemur.Destroyer.lambda$eventRequest$9(Destroyer.java:119)
2015-09-09T12:52:10.63-0700 [App/0]      ERR    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
2015-09-09T12:52:10.63-0700 [App/0]      ERR    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
2015-09-09T12:52:10.63-0700 [App/0]      ERR    at java.lang.Thread.run(Thread.java:745)
twoseat commented 9 years ago

I suspect your credentials aren't sufficient to destroy VMs. The credentials are probably valid (you'd get an InvalidLogin error if they weren't), but clearly being able to destroy 'random' VMs isn't something your average sysadmin encourages! Can you check (either with a sysadmin or by doing a manual test) whether your credentials allow you to destroy VMs?

cxhercules commented 8 years ago

I been looking at this to try to troubleshoot before posting. I am using vSphere 6 and I am connecting with an AD integrated account that I know has access because it is used by bosh to do the Pivotal Cloud Foundry deployment.

2016-07-29T14:00:00.02-0700 [APP/0] OUT org.springframework.web.client.HttpClientErrorException: 401 Unauthorized

Any tips on how to troubleshoot this?

matthewcosgrove commented 8 years ago

Seeing the same issue myself on a PCF install. Looking at the stack trace, this has nothing to do with permissions to destroy VMs but is instead related to permissions for a far less destructive prep operation of calling the Director api to list deployments for potential destroy candidates i.e. https://bosh.io/docs/director-api-v1.html#list-deployments

To clarify

io.pivotal.strepsirrhini.chaoslemur.Destroyer.doDestroy(Destroyer.java:134)

makes a call to this.infrastructure.getMembers() which leads us to

io.pivotal.strepsirrhini.chaoslemur.infrastructure.StandardDirectorUtils.getDeployments(StandardDirectorUtils.java:99)

which makes a REST call to the API endpoint linked above and returns the 401.

I had made the assumption this would require the standard director creds used for bosh target. WRT to troubleshooting, I SSHd onto OpsMan to see if I could curl the director with my chosen creds and I did re-produce the 401. I also tried some other Director related creds from UAA but so far every one I tried got the 401. Only other thing I can think of which could affect this is that as of recently to use bosh target you also need to specify the --ca-cert on a vanilla PCF installation.

It would certainly be useful for the README for Chaos Lemur to document specifically what creds are actually required wrt a PCF deployment

jtgammon commented 8 years ago

I am seeing the 401 Unauthorized described above, and I believe it is caused by the --ca-cert not being included in the bosh director request. Has anyone gotten this working with PCF v1.7.x or PCF v1.8.x?

cah-jeremykuhn commented 8 years ago

I am also having the same issue trying to run this in PCF 1.8.x, is there any word on this issue?

2016-11-09T14:50:00.176-05:00 [APP/0] [OUT] org.springframework.web.client.HttpClientErrorException: 401 Unauthorized
2016-11-09T14:50:00.176-05:00 [APP/0] [OUT] [CHAOS LEMUR] ERROR Unexpected error occurred in scheduled task.
allomov commented 7 years ago

This is actually funny, because there is already 2 PRs to solve this problem:

@nebhale could you take a look on those two and merge something?

nebhale commented 7 years ago

I'll take a look at this in the next couple of days and updates will be centralized here.

udaytangs commented 7 years ago

Is this solved. We are facing the same issue "401 Unauthorized"

2017-08-24T08:36:44.018-04:00 [APP/PROC/WEB/0] [ERR] Exception in thread "destroyer-1" org.springframework.web.client.HttpClientErrorException: 401 Unauthorized 2017-08-24T08:36:44.018-04:00 [APP/PROC/WEB/0] [ERR] at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:63) 2017-08-24T08:36:44.019-04:00 [APP/PROC/WEB/0] [ERR] at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:700) 2017-08-24T08:36:44.019-04:00 [APP/PROC/WEB/0] [ERR] at io.pivotal.strepsirrhini.chaoslemur.infrastructure.AbstractDirectorUtilsInfrastructure.getMembers(AbstractDirectorUtilsInfrastructure.java:42) 2017-08-24T08:36:44.019-04:00 [APP/PROC/WEB/0] [ERR] at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:653) 2017-08-24T08:36:44.019-04:00 [APP/PROC/WEB/0] [ERR] at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:628) 2017-08-24T08:36:44.019-04:00 [APP/PROC/WEB/0] [ERR] at org.springframework.web.client.RestTemplate.getForObject(RestTemplate.java:303) 2017-08-24T08:36:44.019-04:00 [APP/PROC/WEB/0] [ERR] at io.pivotal.strepsirrhini.chaoslemur.infrastructure.StandardDirectorUtils.getDeployments(StandardDirectorUtils.java:74)

SPC0nline commented 7 years ago

Any update on this, we are facing same issue. Tried to create a bosh client also. Still no go

2017-10-04T14:16:09.191+08:00 [APP/PROC/WEB/0] [ERR] at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:91) 2017-10-04T14:16:09.191+08:00 [APP/PROC/WEB/0] [ERR] Exception in thread "destroyer-3" org.springframework.web.client.HttpClientErrorException: 401 Unauthorized 2017-10-04T14:16:09.191+08:00 [APP/PROC/WEB/0] [ERR] at org.springframework.web.client.RestTemplate.getForObject(RestTemplate.java:253) 2017-10-04T14:16:09.191+08:00 [APP/PROC/WEB/0] [ERR] at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:614) 2017-10-04T14:16:09.191+08:00 [APP/PROC/WEB/0] [ERR] at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:570) 2017-10-04T14:16:09.191+08:00 [APP/PROC/WEB/0] [ERR] at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:545) 2017-10-04T14:16:09.191+08:00 [APP/PROC/WEB/0] [ERR] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 2017-10-04T14:16:09.191+08:00 [APP/PROC/WEB/0] [ERR] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 2017-10-04T14:16:09.191+08:00 [APP/PROC/WEB/0] [ERR] at io.pivotal.strepsirrhini.chaoslemur.infrastructure.StandardDirectorUtils.getDeployments(StandardDirectorUtils.java:99) 2017-10-04T14:16:09.191+08:00 [APP/PROC/WEB/0] [ERR] at io.pivotal.strepsirrhini.chaoslemur.infrastructure.AbstractDirectorUtilsInfrastructure.getMembers(AbstractDirectorUtilsInfrastructure.java:42) 2017-10-04T14:16:09.191+08:00 [APP/PROC/WEB/0] [ERR] at io.pivotal.strepsirrhini.chaoslemur.Destroyer.doDestroy(Destroyer.java:134) 2017-10-04T14:16:09.191+08:00 [APP/PROC/WEB/0] [ERR] at io.pivotal.strepsirrhini.chaoslemur.Destroyer.lambda$eventRequest$12(Destroyer.java:119) 2017-10-04T14:16:09.192+08:00 [APP/PROC/WEB/0] [ERR] at java.lang.Thread.run(Thread.java:748)

nickypike commented 6 years ago

Seeing the same issue. I see that there are two pull requests that may resolve this issue. Any estimation of when these might be incorporated?