search

kovetskiy / mark

Sync your markdown files with Confluence pages.
https://samizdat.dev
Apache License 2.0
993 stars 148 forks source link

Mark does not load/use the config file fully #297

Closed AX-LLaidley closed 1 year ago

AX-LLaidley commented 1 year ago

What happened? Mark does not appear to be using the config file. The config file is present, at the default location ~/.config/mark but even though that file contains the username, token, base URL, and settings for title-from-h1 and drop-h1, I'm still having to include all of that information in the command line itself when I run Mark. Mark functions in the expected manner when I include all of the necessary parameters on the command line.

What did you expect to happen? I expected to not have to include the username, token, base URL, title-from-h1, and drop-h1 on the command line, as Mark should be getting that info from the config file.

How can we reproduce the behavior you experienced? Steps to reproduce the behavior:

  1. Check that the config file exists at ~/.config/mark and that it contains the relevant information as described in the documentation.
  2. Try running Mark on a given directory mark -c ~/.config/mark -f "docs/test/*.md"
  3. See that Mark starts to run, and returns 404 errors due to being unable to find/access Confluence.

In case this is related to specific markdown, please provide a minimal markdown example here.

Information (please complete the following information):

Logs or other output

2023-04-28 14:37:13.019 TRACE rest: HTTP/2.0 404 Not Found
                              Atl-Confluence-Via: h:confluence-prod-us-4-3.prod.atl-paas.net
                              Atl-Traceid: beeada0fb378d683
                              Cache-Control: no-cache, no-store, must-revalidate
                              Content-Type: application/json
                              Date: Fri, 28 Apr 2023 18:37:13 GMT
                              Expires: Thu, 01 Jan 1970 00:00:00 GMT
                              Nel: {"failure_fraction": 0.001, "include_subdomains": true, "max_age": 600, "report_to": "endpoint-1"}
                              Report-To: {"endpoints": [{"url": "https://dz8aopenkvv6s.cloudfront.net"}], "group": "endpoint-1", "include_subdomains": true, "max_age": 600}
                              Server: AtlassianEdge
                              Strict-Transport-Security: max-age=63072000; includeSubDomains; preload
                              Vary: Accept-Encoding
                              X-Content-Type-Options: nosniff
                              X-Xss-Protection: 1; mode=block

                              {"statusCode":404,"data":{"authorized":true,"valid":true,"errors":[],"successful":true},"message":"com.atlassian.confluence.api.service.exceptions.NotFoundException: No space with key : TDT"}
2023-04-28 14:37:13.020 TRACE rest: GET /wiki/rest/api/space/TDT?expand=homepage HTTP/1.1
                              Host: automox.atlassian.net
                              Authorization: Basic bGF1cmllLmxhaWRsZXlAYXV0b21veC5jb206N1NuUWdja1NyMmRtVFpVUTFKSVZEM0ZC

2023-04-28 14:37:13.210 TRACE rest: HTTP/2.0 404 Not Found
                              Atl-Confluence-Via: h:confluence-prod-us-4-3.prod.atl-paas.net
                              Atl-Traceid: 8741c17e9d598dc6
                              Cache-Control: no-cache, no-store, must-revalidate
                              Content-Type: application/json
                              Date: Fri, 28 Apr 2023 18:37:13 GMT
                              Expires: Thu, 01 Jan 1970 00:00:00 GMT
                              Nel: {"failure_fraction": 0.001, "include_subdomains": true, "max_age": 600, "report_to": "endpoint-1"}
                              Report-To: {"endpoints": [{"url": "https://dz8aopenkvv6s.cloudfront.net"}], "group": "endpoint-1", "include_subdomains": true, "max_age": 600}
                              Server: AtlassianEdge
                              Strict-Transport-Security: max-age=63072000; includeSubDomains; preload
                              Vary: Accept-Encoding
                              X-Content-Type-Options: nosniff
                              X-Xss-Protection: 1; mode=block

                              {"statusCode":404,"data":{"authorized":true,"valid":true,"errors":[],"successful":true},"message":"com.atlassian.confluence.api.service.exceptions.NotFoundException: No space found with key : TDT"}
2023-04-28 14:37:13.215 FATAL unable to resolve page
                              └─ can't obtain home page from space "TDT"
                                 ├─ Confluence API returned unexpected status: 404 (Not Found)
                                 └─ title: Automated Vulnerability Remediation

Additional context Add any other context about the problem here.

mrueg commented 1 year ago

Can you show me the content of your config file (and anonymize the url, password and username)?

mrueg commented 1 year ago

A 404 also sounds like the resource is not found. NotFoundException: No space found with key : TDT"} Have you tried with a different confluence space?

AX-LLaidley commented 1 year ago

Yeah...but it works if I call out the username and password (token) as options on the command line. The space does exist, and the token is valid, with my user having the necessary permissions to create articles in the target space.

If I don't call out the username and password (token) as command line options, and just specify where Mark should find the config file, I get the above error. This tells me that Mark isn't getting the necessary information from the config file for some reason, or it isn't finding the config file at all, even though it is at the default location.

Here's the config file. I've anonymized the username, token, and URL.

username = "xxx@yyy.com"
password = "api-key-goes-here"
# If you are using Confluence Cloud add the /wiki suffix to base_url
base_url = "http://abababab.atlassian.net/wiki"
title-from-h1=true
h1_drop = true
mrueg commented 1 year ago

If you use an API key I think you can omit the username. Can you try if this works?

I tested the configuration with username + apikey against a Confluence onpremise and receive a ├─ Confluence API returned unexpected status: 401 (Unauthorized)

With 

password = "api-key-goes-here"
# If you are using Confluence Cloud add the /wiki suffix to base_url
base_url = "http://abababab.atlassian.net/wiki"
title-from-h1 = true
h1_drop = true

it works.

AX-LLaidley commented 1 year ago

I tried it this way, using the following command: mark --debug --trace -c ~/.config/mark -f "docs/Knowledge-Base/Getting-Started/Abababab-University.md"

It still doesn't work. It's still returning a 404 on the space and the space homepage, however the space DOES exist in our Confluence instance, and this command works if I provide both the username and password on the command line using the -u and -p options.

Here's what the output looks like (anonymized):

2023-05-02 13:25:47.904 INFO  processing docs/Knowledge-Base/Getting-Started/Abababab-University.md
2023-05-02 13:25:47.905 TRACE loaded macro "@\\{([^}]+)\\}"
                              ├─ template: ac:link:user
                              ├─ expr: @\{([^}]+)\}
                              └─ config: 
                                      Name: ${1} 
2023-05-02 13:25:47.905 TRACE loaded macro ":box:([^:]+):([^:]*):(.+):"
                              ├─ template: ac:box
                              ├─ expr: :box:([^:]+):([^:]*):(.+):
                              └─ config: 
                                      Icon: true
                                      Name: ${1}
                                      Title: ${2}
                                      Body: ${3} 
2023-05-02 13:25:47.905 TRACE loaded macro ":toc:"
                              ├─ template: ac:toc
                              ├─ expr: :toc:
                              └─ config: 
                                      Printable: 'false'
                                      MinLevel: 2 
2023-05-02 13:25:47.905 TRACE found a relative link: full=https://university.abababab.com/ filename=https://university.abababab.com/ hash=
2023-05-02 13:25:47.905 TRACE filepath: docs/Knowledge-Base/Getting-Started/https:/university.abababab.com
2023-05-02 13:25:47.905 TRACE found a relative link: full=https://community.abababab.com/ filename=https://community.abababab.com/ hash=
2023-05-02 13:25:47.905 TRACE filepath: docs/Knowledge-Base/Getting-Started/https:/community.abababab.com
2023-05-02 13:25:47.905 TRACE found a relative link: full=../../../static/images/Abababab-University/ax_uni.png filename=../../../static/images/Abababab-University/ax_uni.png hash=
2023-05-02 13:25:47.905 TRACE filepath: static/images/Abababab-University/ax_uni.png
2023-05-02 13:25:47.907 TRACE rest: GET /wiki/rest/api/content/?expand=ancestors%2Cversion&spaceKey=TDT&title=Abababab+University&type=page HTTP/1.1
                              Host: abababab.atlassian.net
                              Authorization: Bearer <api-key-goes-here>

2023-05-02 13:25:48.338 TRACE rest: HTTP/2.0 404 Not Found
                              Atl-Confluence-Via: h:confluence-prod-us-4-3.prod.atl-paas.net
                              Atl-Traceid: 85c90d76024679ec
                              Cache-Control: no-cache, no-store, must-revalidate
                              Content-Type: application/json
                              Date: Tue, 02 May 2023 17:25:48 GMT
                              Expires: Thu, 01 Jan 1970 00:00:00 GMT
                              Nel: {"failure_fraction": 0.001, "include_subdomains": true, "max_age": 600, "report_to": "endpoint-1"}
                              Report-To: {"endpoints": [{"url": "https://dz8aopenkvv6s.cloudfront.net"}], "group": "endpoint-1", "include_subdomains": true, "max_age": 600}
                              Server: AtlassianEdge
                              Strict-Transport-Security: max-age=63072000; includeSubDomains; preload
                              Vary: Accept-Encoding
                              X-Content-Type-Options: nosniff
                              X-Xss-Protection: 1; mode=block

                              {"statusCode":404,"data":{"authorized":true,"valid":true,"errors":[],"successful":true},"message":"com.atlassian.confluence.api.service.exceptions.NotFoundException: No space with key : TDT"}
2023-05-02 13:25:48.338 TRACE rest: GET /wiki/rest/api/space/TDT?expand=homepage HTTP/1.1
                              Host: abababab.atlassian.net
                              Authorization: Bearer <api-key-goes-here>

2023-05-02 13:25:48.523 TRACE rest: HTTP/2.0 404 Not Found
                              Atl-Confluence-Via: h:confluence-prod-us-4-3.prod.atl-paas.net
                              Atl-Traceid: 9cde85e62abf0157
                              Cache-Control: no-cache, no-store, must-revalidate
                              Content-Type: application/json
                              Date: Tue, 02 May 2023 17:25:48 GMT
                              Expires: Thu, 01 Jan 1970 00:00:00 GMT
                              Nel: {"failure_fraction": 0.001, "include_subdomains": true, "max_age": 600, "report_to": "endpoint-1"}
                              Report-To: {"endpoints": [{"url": "https://dz8aopenkvv6s.cloudfront.net"}], "group": "endpoint-1", "include_subdomains": true, "max_age": 600}
                              Server: AtlassianEdge
                              Strict-Transport-Security: max-age=63072000; includeSubDomains; preload
                              Vary: Accept-Encoding
                              X-Content-Type-Options: nosniff
                              X-Xss-Protection: 1; mode=block

                              {"statusCode":404,"data":{"authorized":true,"valid":true,"errors":[],"successful":true},"message":"com.atlassian.confluence.api.service.exceptions.NotFoundException: No space found with key : TDT"}
2023-05-02 13:25:48.525 FATAL unable to resolve page
                              └─ can't obtain home page from space "TDT"
                                 ├─ Confluence API returned unexpected status: 404 (Not Found)
                                 └─ title: Abababab University
mrueg commented 1 year ago

Please test again both cases with 9.3.0. If you use --debug there should be output printed that shows your config

joyfulrabbit commented 1 year ago

@mrueg For on prem you don't need a username, for cloud, you do. Always happy to help test either.

Another helpful troubleshooting tool is curl: https://github.com/kovetskiy/mark/issues/259#issuecomment-1466926040

I have installs on both and am in the process of an awkward migration from on prem to cloud that is stalled by a silly technical limitation on Atlassian's end that we didn't anticipate and weren't warned about.

joyfulrabbit commented 1 year ago

Just as an aside, I've confirmed my config files work fine for me on prem and in the cloud. This appears to be a user error, not a mark error. Mark has some rough edges and cloud confluence has really goofy error reporting as compared to my on prem. Lots of sharp edges.

mrueg commented 1 year ago

No response from reporter, closing. Please feel free to reopen if the issue still exists.

gautaz commented 6 months ago

Just as an aside, I've confirmed my config files work fine for me on prem and in the cloud. This appears to be a user error, not a mark error. Mark has some rough edges and cloud confluence has really goofy error reporting as compared to my on prem. Lots of sharp edges.

Hello @joyfulrabbit, do you see any way to cut out these sharp edges to help the user diagnose this type of issues, in particular when trying to use mark with cloud confluence? To be honest, after being unable to diagnose such an issue, I kind of throwed in the towel on using mark which is a shame because I really like the idea behind mark. I can open a new issue as a mean to discuss such improvements if there is a way to tackle this kind of issues more easily.