huserben / TfsExtensions

Extensions for TFS 2015+ such as custom Widgets (require TFS 2017) and Build Tasks
MIT License
44 stars 22 forks source link

Task is not working behind a proxy. Please respect agent.proxyurl and agent.proxypassword as well as the proxy bypass list #78

Closed gereon77 closed 5 years ago

gereon77 commented 6 years ago

Unfortunately when the agent is behind a proxy the task is failing with connection refused:

2018-06-19T13:11:06.3083157Z innerError: 2018-06-19T13:11:06.3083311Z { Error: connect ECONNREFUSED ip:443 2018-06-19T13:11:06.3083560Z at Object.exports._errnoException (util.js:1018:11) 2018-06-19T13:11:06.3083749Z at exports._exceptionWithHostPort (util.js:1041:20) 2018-06-19T13:11:06.3084258Z at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1086:14) 2018-06-19T13:11:06.3084665Z code: 'ECONNREFUSED', 2018-06-19T13:11:06.3084831Z errno: 'ECONNREFUSED', 2018-06-19T13:11:06.3084981Z syscall: 'connect', 2018-06-19T13:11:06.3085143Z address: 'ip', 2018-06-19T13:11:06.3085289Z port: 443 } }

huserben commented 6 years ago

Hi @gereon77

did you setup the build agent with a proxy or what exactly is your setup? It seems like there was a similar issue (based on the error and the relation to a proxy) once which was solved by setting a variable, maybe this helps: https://github.com/huserben/TfsExtensions/issues/58#issuecomment-373416433

gereon77 commented 6 years ago

Hi @huserben

Well, to be honest I dont want to set http_proxy for the whole system. I set up my agent this way: https://docs.microsoft.com/en-us/vsts/pipelines/agents/proxy?view=vsts&tabs=windows

Now only the agent service is using the proxy and provides the credentials as the mentioned environment variables to all tasks. When system.debug is set to true each task sees agent.proxyusername, proxypassword (masked) and proxyurl.

Anyways, I found out what to do to make it work so the tasks will use the agent setup automatically. Just update your dependencies, especially vsts-node-api. The latest version is taking care of the proxy information without the need to pass those as parameter:

https://github.com/Microsoft/vsts-node-api/blob/1dc7a4af55e588ed7143b7bb563fb92ba385f291/api/WebApi.ts#L89

Best regards

huserben commented 6 years ago

Hi

thanks for the hint with the vsts-node-api. However the problem is that the task is not yet using this api. Changing to this is something that was/is foreseen. With this being built-in it gives the whole thing more incentive. However this will need a certain amount of time to fulfill. Currently I simply lack the time to do this.

So I can update you on this progress, but I don't know how long it will take. I will check a bit if there are other options with the implementation that can be used in the meanwhile.

huserben commented 6 years ago

Hi @gereon77

sadly I still did not find the time to refactor the service to use vsts-node-api. However maybe this could help you. Microsoft implemented a similar feature into VSTS lately, so you can start builds upon completion of other builds. Depending on the complexity of your use case you could achieve your goal with this: https://azure.microsoft.com/en-us/blog/visual-studio-team-services-april-2018/

LarryRothOz commented 5 years ago

Hi @huserben.

Great extension! Unfortunately we can't use it due to the proxy issue. Would love to see this resolved.

huserben commented 5 years ago

Hi @LarryRothOakton

yes, i might have time the following days to look a bit more deeply into the problem and then at least might have an idea on how long it will take to change.

I'll keep you updated on the progress

huserben commented 5 years ago

Hi @gereon77 and @LarryRothOakton

I was looking a bit into the refactoring to use the vso-node-api proposed by @gereon77 I now changed most parts of the code to start testing. I might be able to provide a preview version tomorrow for you, as I'm on holidays after it will definetly not come out of preview for the next 1.5 weeks, as I have to rewrite the tests and documentation apart from doing some proper testing of some special cases to see whether it's still working as it should.

So in case I would upload a preview version of the task I would let you know, then you could try to test and might be able to provide me some feedback whether the change helped with the proxy problem you're facing.

huserben commented 5 years ago

Hi

I just uploaded a preview version that makes use of the mentioned library. The tasks are now available in a new version and marked as preview: grafik

You have to manually switch from the Version 2 to version 3. The interface has slightly changed in case you trigger a build from a different team project, collection or server. Now you have to specify the team project in a separate field and not as part of the url. More info can be found on the overview page of the marketplace: https://marketplace.visualstudio.com/items?itemName=benjhuser.tfs-extensions-build-tasks

I would be glad if you could test it now and verify whether the proxy is now handled correctly or not.

Thanks for your assistance.

huserben commented 5 years ago

Hi again

in case you were affected by the mess I created with yesterdays version, it's fixed now and version 3.0.2 that is on the marketplace contains the tasks in the stable and the preview version and will let you switch between them in order to test whether it works now with the proxy or not.

huserben commented 5 years ago

@gereon77 @LarryRothOakton

Hi you both, do you have any update on this issue? Were you able to test it with the newest version to see whether it works?

LarryRothOz commented 5 years ago

Hi @huserben thanks for the update. Unfortunately it's still not working for me. It is getting further but we're now getting a 407 error, which I think is from the proxy. We have username/password proxy authentication, so not sure if that's the issue. Unfortunately the development team doesn't have much insight into how the proxy has been implemented (still going on that DevOps journey for this client!). The error we're getting is:

[debug]loading SECRET_AGENT_PROXYPASSWORD [debug]loaded 29 [debug]Agent.ProxyUrl=http://\<proxy>:8080 [debug]Agent.ProxyUsername=\<username> [debug]Agent.ProxyPassword=*** [debug]Agent.ProxyBypassList=undefined ... ... Using Custom Team Project Team Project: abc Using Custom Collection Url Server URL: https://abc.visualstudio.com/ Trying to fetch authentication token from system... Using OAuth Access Token [debug]task result: Failed [error]tunneling socket could not be established, statusCode=407 [debug]Processed: ##vso[task.issue type=error;]tunneling socket could not be established, statusCode=407 [debug]Processed: ##vso[task.complete result=Failed;]tunneling socket could not be established, statusCode=407

huserben commented 5 years ago

Hi @LarryRothOakton

thanks for the input. Hmm it seems there is some issue with a dependency of the vso-node-api that is used for the connection. I'll try to update the dependency and provide a new version to see if that will fix the issue. I'll post an update here once the new version is available.

Update: A new Version is now available, please let me know whether something has changed now with the updated dependencies.

LarryRothOz commented 5 years ago

Hi @huserben thanks again for the update. I'm still getting the same error. We might have an unusual proxy configuration. Is anyone else able to test this?

huserben commented 5 years ago

Hi @LarryRothOakton

ok, thanks for the info, I'm sorry that it's still not working, but I'm not sure how much more I can do here, as now I'll let the vso-node-api handle all the connection details... I found this statement on another github repo:

Authenticated proxies are still not supported by the vsts-node-api dependency.

However that was more than a year ago, but it might still be the source of your issue...

huserben commented 5 years ago

Will close due to inactivity - please reopen if there is still an issue