search

WikiTransformationProject / wikitraccs-releases

Releases, issues and discussions for WikiTraccs from the Wiki Transformation Project
https://www.wikitransformationproject.com
8 stars 0 forks source link

Connectivity Issues - Transformation will not run since there are issues with required endpoints #29

Closed bajahranks closed 1 year ago

bajahranks commented 1 year ago

I followed the documentation via the YouTube video to test out the migration. I created 2 SharePoint sites as required, went through the steps and everything worked until i clicked on the "Start transformation" button. I am getting the following error

Could not reach all needed endpoints, your environment might be somewhat locked down. Specifically those endpoints are not reachable: <url of sharepoint site> that is used for storing the transformation result (REQUIRED, blocks transformation). Transformation will not run since there are issues with required endpoints.

My question is why did the endpoint work for the "Store" site but not for the "Target" site. From the store site I was able to access the list of all the spaces 600+ of them, I chose only one of them for the migration, but then after clicking on the "Start transformation" button I get this error message.

I doubled-check to ensure that the path to the SharePoint site was correct.

What should I try?

heinrich-ulbricht commented 1 year ago

Hi. Could you please first try using the store site as migration target as well? The connection seems fine there.

If this doesn't work I'd like to have a look at the logs. They are in the WikiTraccs.GUI/logs folder. (You could also scan them for any obvious errors.)

And would you mind posting the two SharePoint URLs (with host name anonymized) so I can look at the URL format?

bajahranks commented 1 year ago

I tried using the store site as the target site and got the same error.

The store is /sites/BIPTarget (a bit misleading but I got the names mix up at first) The target is /sites/BannerInformationPortal

I went through the logs but don't know what to look for that would signal an error. I replaced what I perceive to be sensitive information from the logs with an *, I hope this does not render the debugging process impossible. WikiTraccs.GUI.debug20230306(Redacted).txt

heinrich-ulbricht commented 1 year ago

@bajahranks Thanks a ton for sending the log file! Unfortunately WikiTraccs does not seem to add useful logs for the connection checks, yet. :expressionless:

I added more logging. Could you please try once more with release 0.1.4?

The logs should now contain entries like this (a successful connection check):

[ 22:08:34 DBG] Entering CheckConnectionsAsync, checking categories "All" | 
[ 22:08:34 DBG] Checking connection to https://chromedriver.storage.googleapis.com by reading string response... | 
[ 22:08:34 DBG] HTTP result for https://chromedriver.storage.googleapis.com (first 300 chars): <?xml version='1.0' encoding='UTF-8'?><ListBucketResult xmlns='http://doc.s3.amazonaws.com/2006-03-01'><Name>chromedriver</Name><Prefix></Prefix><Marker></Marker><IsTruncated>false</IsTruncated><Contents><Key>100.0.4896.20/chromedriver_linux64.zip</Key><Generation>1646371839854073</Generation><MetaG | 
[ 22:08:35 DBG] Set isConnectionCheckSuccessful to: true | 
[ 22:08:35 DBG] Checking connection to https://chromedriver.chromium.org by reading string response... | 
[ 22:08:36 DBG] HTTP result for https://chromedriver.chromium.org (first 300 chars): <!DOCTYPE html><html lang="en-US" itemscope itemtype="http://schema.org/WebPage"><head><meta charset="utf-8"><script nonce="*">var DOCS_timing={}; DOCS_timing['sl']=new Date().getTime();</script><script nonce="*">function _DumpException(e) {throw e;}</script | 
[ 22:08:37 DBG] Set isConnectionCheckSuccessful to: true | 
[ 22:08:37 DBG] Checking connection to https://login.microsoftonline.com by reading string response... | 
[ 22:08:38 DBG] HTTP result for https://login.microsoftonline.com (first 300 chars): <!-- Copyright (C) Microsoft Corporation. All rights reserved. -->
<!DOCTYPE html>
<html dir="ltr" class="" lang="en">
<head>
    <title>Sign in to your account</title>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
    <meta http-equiv="X-UA-Compatible" content="IE=ed | 
[ 22:08:38 DBG] Set isConnectionCheckSuccessful to: true | 
[ 22:08:38 DBG] Checking connection to https://contoso.sharepoint.com/sites/2023-03-06-demo by reading string response... | 
[ 22:08:39 INF] Got HttpRequestException: Response status code does not indicate success: 403 (Forbidden). (Status code: "Forbidden") | 
System.Net.Http.HttpRequestException: Response status code does not indicate success: 403 (Forbidden).
   at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
   at System.Net.Http.HttpClient.GetStringAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
   at WikiTraccs.Shared.SanityChecks.PrerequisiteChecks.CheckConnectionsAsync(UrlCategory categories, UrlToCheck[] additionalUrls)
[ 22:08:39 DBG] Got Forbidden result for 'https://contoso.sharepoint.com/sites/2023-03-06-demo', set isConnectionCheckSuccessful to: true | 
[ 22:08:39 DBG] Checking connection to https://www.contoso.de/confluence/ by reading string response... | 
[ 22:08:40 DBG] HTTP result for https://www.contoso.de/confluence/ (first 300 chars): <!DOCTYPE html>
    <html>
<head>
            <title>Log In - Confluence</title>
            <meta http-equiv="X-UA-Compatible" content="IE=EDGE,chrome=IE7">
<meta charset="U | 
[ 22:08:41 DBG] Set isConnectionCheckSuccessful to: true | 
[ 22:09:12 DBG] Returning connection check result: [{"UrlToCheck":{"Url":{"Scheme":"https","UserInfo":"","Host":"chromedriver.storage.googleapis.com","Port":null,"Authority":"chromedriver.storage.googleapis.com","Root":"https://chromedriver.storage.googleapis.com","Path":"","PathSegments":[],"Query":"","QueryParams":[],"Fragment":"","IsRelative":false,"IsSecureScheme":true},"Category":1,"UsedFor":"downloading the web driver for your current Chrome version","IsOptional":true},"IsConnectionCheckSuccessful":true},{"UrlToCheck":{"Url":{"Scheme":"https","UserInfo":"","Host":"chromedriver.chromium.org","Port":null,"Authority":"chromedriver.chromium.org","Root":"https://chromedriver.chromium.org","Path":"","PathSegments":[],"Query":"","QueryParams":[],"Fragment":"","IsRelative":false,"IsSecureScheme":true},"Category":1,"UsedFor":"checking which web driver version is needed for your current Chrome version","IsOptional":true},"IsConnectionCheckSuccessful":true},{"UrlToCheck":{"Url":{"Scheme":"https","UserInfo":"","Host":"login.microsoftonline.com","Port":null,"Authority":"login.microsoftonline.com","Root":"https://login.microsoftonline.com","Path":"","PathSegments":[],"Query":"","QueryParams":[],"Fragment":"","IsRelative":false,"IsSecureScheme":true},"Category":2,"UsedFor":"authenticating you with Azure AD","IsOptional":false},"IsConnectionCheckSuccessful":true},{"UrlToCheck":{"Url":{"Scheme":"https","UserInfo":"","Host":"contoso.sharepoint.com","Port":null,"Authority":"contoso.sharepoint.com","Root":"https://contoso.sharepoint.com","Path":"/sites/2023-03-06-demo","PathSegments":["sites","2023-03-06-demo"],"Query":"","QueryParams":[],"Fragment":"","IsRelative":false,"IsSecureScheme":true},"Category":4,"UsedFor":"storing the transformation result","IsOptional":false},"IsConnectionCheckSuccessful":true},{"UrlToCheck":{"Url":{"Scheme":"https","UserInfo":"","Host":"www.contoso.de","Port":null,"Authority":"www.contoso.de","Root":"https://www.contoso.de","Path":"/confluence/","PathSegments":["confluence"],"Query":"","QueryParams":[],"Fragment":"","IsRelative":false,"IsSecureScheme":true},"Category":8,"UsedFor":"getting Confluence content to transform","IsOptional":false},"IsConnectionCheckSuccessful":true}] | 
[ 22:09:12 DBG] Leaving CheckConnectionsAsync |

When a connection check fails it should now be reflected in the logs.

bajahranks commented 1 year ago

@heinrich-ulbricht Here are the results of the logs after updating. I'm showing the areas that differ from a successful login as shown in your expected log output. Everything else is identical. [ 11:54:39 DBG] Checking connection to https://*.sharepoint.com/sites/BannerInformationPortal by reading string response... | [ 11:54:39 INF] Got HttpRequestException: Response status code does not indicate success: 401 (Unauthorized). (Status code: "Unauthorized") | System.Net.Http.HttpRequestException: Response status code does not indicate success: 401 (Unauthorized). at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode() at System.Net.Http.HttpClient.GetStringAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) at WikiTraccs.Shared.SanityChecks.PrerequisiteChecks.CheckConnectionsAsync(UrlCategory categories, UrlToCheck[] additionalUrls) in /home/runner/work/wikitraccs-code/wikitraccs-code/WikiTraccs.Shared/SanityChecks/PrerequisitesChecks.cs:line 67 [ 11:54:40 DBG] Set isConnectionCheckSuccessful to: false |

In the "Returning connection check result" section of the logs everything passes except { "UrlToCheck": { "Url": { "Scheme": "https", "UserInfo": "", "Host": "*sharepoint.com", "Port": null, "Authority": "*.sharepoint.com", "Root": "https://*.sharepoint.com", "Path": "/sites/BannerInformationPortal", "PathSegments": [ "sites", "BannerInformationPortal" ], "Query": "", "QueryParams": [], "Fragment": "", "IsRelative": false, "IsSecureScheme": true }, "Category": 4, "UsedFor": "storing the transformation result", "IsOptional": false }, "IsConnectionCheckSuccessful": false }

heinrich-ulbricht commented 1 year ago

Ha, thanks! This should be easy to fix. I get a 403 for the SP connection test - which counts as successful connection attempt. You get a 401 instead. WikiTraccs should handle this the same way.

Will push an update tomorrow and then we'll see if that is all it takes.

heinrich-ulbricht commented 1 year ago

@bajahranks Could you please try version 0.1.5? This should get you past the point where it's stuck.

bajahranks commented 1 year ago

@heinrich-ulbricht Success. Thank you for your help. Time to go pitch this product to my director.