search

jessehouwing / azure-pipelines-tfvc-tasks

Azure Pipelines tasks for Team Foundation Version Control
https://marketplace.visualstudio.com/items?itemName=jessehouwing.jessehouwing-vsts-tfvc-tasks
MIT License
27 stars 18 forks source link

Checkin Fails with 'The service endpoint [ProjectCollectionName] was not found' #68

Closed seanmobrien closed 5 years ago

seanmobrien commented 5 years ago

As of 9/5/2018, the Checkin task started to fail on me with an error stating that "The service endpoint EAS was not found" (note that EAS is the name of my Team Collection and also the root folder of the VSTSVC tree attached to the build). These builds worked fine last Friday, but were not used on Monday (holiday) or Tuesday (VSTS outage). I have attached the build log output to this issue - note that the failed checkin happens in step 10. I am happy to provide any additional information that would be helpful - please let me know.

logs_14123.zip

Thanks! Sean

jessehouwing commented 5 years ago

My test builds are succeeding and are returning no issues. I haven't recently pushed a new version.

Can you run a build with the following variables set:

system.debug = true
agent.diagnostic = true

Things that come to mind that may have caused this:

seanmobrien commented 5 years ago

Thanks for such a quick reply!

I did try restarting the agent services, but it didn't help. I'm not positive on how to go about checking if the agent auto-updated, but I don't think so - I am using a service running out of bin.2.138.4 folder (which is dated 8/13/18), so that upgrade should have happened a couple of weeks ago.

I ran another build with the build variable above, and it looks like the failure is happening somewhere in Get-SourceProvider - I've attached the full log below.

Let me know if there's anything else I can try logs_with_debug_enabled.zip

jessehouwing commented 5 years ago

I don't see anything that stands out. Could you run a smbatch script task that outputs cmd /c set

On Thu, 6 Sep 2018, 01:05 seanmobrien, notifications@github.com wrote:

As of 9/5/2018, the Checkin task started to fail on me with an error stating that "The service endpoint EAS was not found" (note that EAS is the name of my Team Collection and also the root folder of the VSTSVC tree attached to the build). These builds worked fine last Friday, but were not used on Monday (holiday) r Tuesday (VSTS outage). I have attached the build log output to this issue - note that the failed checkin happens in step 10. I am happy to provide any additional information that would be helpful - please let me know.

logs_14123.zip https://github.com/jessehouwing/vsts-tfvc-tasks/files/2354829/logs_14123.zip

Thanks! Sean

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/jessehouwing/vsts-tfvc-tasks/issues/68, or mute the thread https://github.com/notifications/unsubscribe-auth/AD-uS87jTgDv-n8reZsc_EptissHVA-Qks5uYFjDgaJpZM4Wb1bO .

seanmobrien commented 5 years ago

Sure thing - I have a meeting I need to run to but will do it right when I get back. One thing that I did notice is it appears from the output that the error could be occurring around line 107 of vsts-tfvc-shared.psm1 - $serviceEndpoint = Get-ServiceEndpoint -Context $distributedTaskContext -Name $env:BUILD_REPOSITORY_NAME

However, if I retrieve GET https://xxxxxx.visualstudio.com/EAS/_apis/serviceendpoint/endpoints?api-version=4.1-preview.1&endpointNames=EAS I receive {"count":0,"value":[]}.

Here is what I get back for a full list of service endpoints: {"count":2,"value":[{"data":{},"id":"f578d64a-40d3-4b20-befe-815c56a97e0c","name":"WebTeam Nuget Feed","type":"externalnugetfeed","url":"http://nuget.xxxxx.org/nuget/WebTeam/","createdBy":{"displayName":"Xxxx Xxxxx","id":"2ddf8d7e-8c94-4cd7-8c16-280a6f1cab66","uniqueName":"xxxxx@xxxx.ORG"},"authorization":{"scheme":"None"},"isReady":true},{"data":{},"id":"e20292f0-d1d5-4c40-aace-3a739125630f","name":"Veracode (xxxxxx@xxxx.org)","type":"VeracodeAnalysisCenterEndpoint","url":"https://analysiscenter.veracode.com/","createdBy":{"displayName":"Xxxx Xxxxxxxx","id":"c02b39d5-b378-4eb1-b33b-b150575f6359","uniqueName":"xxxxxx@xxxx.ORG"},"authorization":{"scheme":"Token"},"isReady":true}]}

Is it possible I've lost some endpoint setup somewhere along the way? I don't see any similar endpoints configured in other projects, but that doesn't mean I just don't know where to look.

jessehouwing commented 5 years ago

The task is requesting the System Endpoint directly from the server. These are predefined endpoints. They should just "be there".

On Thu, Sep 6, 2018 at 8:27 PM seanmobrien notifications@github.com wrote:

Sure thing - I have a meeting I need to run to but will do it right when I get back. One thing that I did notice is it appears from the output that the error could be occurring around line 107 of vsts-tfvc-shared.psm1 - $serviceEndpoint = Get-ServiceEndpoint -Context $distributedTaskContext -Name $env:BUILD_REPOSITORY_NAME

However, if I retrieve GET https://xxxxxx.visualstudio.com/EAS/_apis/serviceendpoint/endpoints?api-version=4.1-preview.1&endpointNames=EAS I receive {"count":0,"value":[]}.

Here is what I get back for a full list of service endpoints: {"count":2,"value":[{"data":{},"id":"f578d64a-40d3-4b20-befe-815c56a97e0c","name":"WebTeam Nuget Feed","type":"externalnugetfeed","url":" http://nuget.xxxxx.org/nuget/WebTeam/","createdBy":{"displayName":"Xxxx Xxxxx","id":"2ddf8d7e-8c94-4cd7-8c16-280a6f1cab66","uniqueName":"xxxxx@xxxx.ORG"},"authorization":{"scheme":"None"},"isReady":true},{"data":{},"id":"e20292f0-d1d5-4c40-aace-3a739125630f","name":"Veracode (xxxxxx@xxxx.org)","type":"VeracodeAnalysisCenterEndpoint","url":" https://analysiscenter.veracode.com/","createdBy":{"displayName":"Xxxx Xxxxxxxx","id":"c02b39d5-b378-4eb1-b33b-b150575f6359","uniqueName":"xxxxxx@xxxx.ORG "},"authorization":{"scheme":"Token"},"isReady":true}]}

Is it possible I've lost some endpoint setup somewhere along the way? I don't see any similar endpoints configured in other projects, but that doesn't mean I just don't know where to look.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jessehouwing/vsts-tfvc-tasks/issues/68#issuecomment-419195333, or mute the thread https://github.com/notifications/unsubscribe-auth/AD-uS5KHwQ1h8330mQTp9dJ9TfvS6EfRks5uYWj_gaJpZM4Wb1bO .

seanmobrien commented 5 years ago

Sorry it took me so long - been a crazy day. Attached is a build including the cmd /c set output: logs_with_extra_vars.zip We've tried it across a couple of different builds and build servers, and are consistently seeing the same result. If there's anything else you can think of I can try let me know- i have direct access to the build servers themselves if need be.

grizz0blaw commented 5 years ago

Hi there! I just wanted to chime in that we've been seeing the same thing on our builds since the VSTS outage on Tues (04Sep). To clarify our last build that went through was Tues morning (0224 E/T). As Sean said, according to the logs it's not finding the endpoint which is the team project path. I ran the build using the 2 debugs mentioned. The agent_diag log shows that endpoint (aka server path) is correct & found. In fact the GetSources task would fail if the endpoint wasn't found. Is there a way to verify the method the task uses to grab the endpoint hasn't been broken by the fix that was implemented to get the VSTS servers back up on Tues night?

jessehouwing commented 5 years ago

I ran a test on the hosted pools and my local agent without issues.

A rewrite using the new api has been in the books for a long time, but I haven't had the time to do that.

Can you check whether the build has a collection scope or project scope security token?

On Sat, 8 Sep 2018, 00:44 grizz0blaw, notifications@github.com wrote:

Hi there! I just wanted to chime in that we've been seeing the same thing on our builds since the VSTS outage on Tues (04Sep). To clarify our last build that went through was Tues morning (0224 E/T). As Sean said, according to the logs it's not finding the endpoint which is the team project path. I ran the build using the 2 debugs mentioned. The agent_diag log shows that endpoint (aka server path) is correct & found. In fact the GetSources task would fail if the endpoint wasn't found. Is there a way to verify the method the task uses to grab the endpoint hasn't been broken by the fix that was implemented to get the VSTS servers back up on Tues night?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jessehouwing/vsts-tfvc-tasks/issues/68#issuecomment-419585601, or mute the thread https://github.com/notifications/unsubscribe-auth/AD-uSyzhCbRlRh21c4UI3PWSCDHrzDw7ks5uYvbUgaJpZM4Wb1bO .

seanmobrien commented 5 years ago

I won't be able to get in touch with the people who have access to that information until Monday, but I will check and let you know. Thanks again!

Get Outlook for Androidhttps://aka.ms/ghei36

From: Jesse Houwing notifications@github.com Sent: Saturday, September 8, 2018 12:54:28 AM To: jessehouwing/vsts-tfvc-tasks Cc: seanmobrien; Author Subject: Re: [jessehouwing/vsts-tfvc-tasks] Checkin Fails with 'The service endpoint [ProjectCollectionName] was not found' (#68)

I ran a test on the hosted pools and my local agent without issues.

A rewrite using the new api has been in the books for a long time, but I haven't had the time to do that.

Can you check whether the build has a collection scope or project scope security token?

On Sat, 8 Sep 2018, 00:44 grizz0blaw, notifications@github.com wrote:

Hi there! I just wanted to chime in that we've been seeing the same thing on our builds since the VSTS outage on Tues (04Sep). To clarify our last build that went through was Tues morning (0224 E/T). As Sean said, according to the logs it's not finding the endpoint which is the team project path. I ran the build using the 2 debugs mentioned. The agent_diag log shows that endpoint (aka server path) is correct & found. In fact the GetSources task would fail if the endpoint wasn't found. Is there a way to verify the method the task uses to grab the endpoint hasn't been broken by the fix that was implemented to get the VSTS servers back up on Tues night?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jessehouwing/vsts-tfvc-tasks/issues/68#issuecomment-419585601, or mute the thread https://github.com/notifications/unsubscribe-auth/AD-uSyzhCbRlRh21c4UI3PWSCDHrzDw7ks5uYvbUgaJpZM4Wb1bO .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/jessehouwing/vsts-tfvc-tasks/issues/68#issuecomment-419615849, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AIVsAw54MGrPhhwyHz9Bnno7Ps-9znrOks5uY1uUgaJpZM4Wb1bO.

jessehouwing commented 5 years ago

Can you guys force an agent update to get to 2.140?

From the folks that build the agent:

make sure they are on latest version agent. there was a bug in the agent cause the legacy endpoint not populate correctly.

—

doMConSwiss commented 5 years ago

Same problem here. Updating our build agent to 2.140 fixed the issue.

seanmobrien commented 5 years ago

Looks like that was my problem as well - thanks again for all your help getting this figured out!

grizz0blaw commented 5 years ago

I can also confirm that everything is back up & running. Thank you Jesse!