jhaals
commented
1 year ago Ping @tradcliffe-expediagroup who contributed this module for visibility :)
Closed Fuglen closed 1 year ago
jhaals
commented
1 year ago Ping @tradcliffe-expediagroup who contributed this module for visibility :)
awanlin
commented
1 year ago Just wanted to mention that I've done some work with this action recently to get it working with Confluence Cloud and I did not run into this issue. I also just ran it again and that also works fine locally and once deployed.
The way the files are written is very straightforward and there's nothing being done around what their permissions would be from what I can tell. Here's the code for creating the folder:
And here is where we write the files into it:
But now that I see that line above I'm wondering what that w is doing? The docs say: "'w': Open file for writing. The file is created (if it does not exist) or truncated (if it exists)." (https://nodejs.org/api/fs.html#file-system-flags)
awanlin
commented
1 year ago I should have mentioned my environment for reference:
yarn run v1.22.19
$ /Users/andre.wanlin/repos/github/backstage-app/node_modules/.bin/backstage-cli info
OS: Darwin 22.5.0 - darwin/arm64
node: v18.13.0
yarn: 1.22.19
cli: 0.22.8-next.1 (installed)
backstage: 1.15.0-next.1
Fuglen
commented
1 year ago Thank you for diving into the code and show what happens where, good to hear that you did not have any issues with permissions. Just tried to run Backstage from an elevated powershell and got the same error. Soon, I will hopefully get a new laptop at work which should not be as strict/locked down as my current one, then I will test this again and update here.
github-actions[bot]
commented
1 year ago This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
📜 Description
Using the confluence-to-markdown custom action fails it's cleanup if there are images on the Confluence Page.
I'm also writing my own Scaffolder custom action so that it publishes to our Bitbucket server instead of Github. My own custom action does indeed use the contents of the workspace folder that is created by the confluence-to-markdown action but does not cause the issue.
I found a closed similar issue: #169
👍 Expected behavior
I believe a folder is created when running a template/action. I expect this folder to be deleted after it's finished without errors.
👎 Actual Behavior with Screenshots
The template creates a folder with a mkdocs.yml file and a docs folder that contains the converted Confluence Page markdown file and a "img" folder that contains all images that are from this Page.
All files except the "img" folder is being deleted succesfully but it fails deleting the images as it does not have permission, neither can I view or delete the images with admin rights in a shell in Windows. Not sure if the permission issue is related to my corporate computer?
👟 Reproduction steps
I installed the confluence-to-markdown action and made my own template
The template
apiVersion: scaffolder.backstage.io/v1beta3 kind: Template metadata: name: confluence-to-markdown title: Confluence to Markdown description: This template converts a single confluence document to Markdown for Techdocs and adds it to a given GitHub repo. tags: - do-not-use - poc spec: owner: backstage-core-team type: service parameters: - title: Confluence and Github Repo Information properties: confluenceUrls: type: array description: Urls for confluence doc to be converted to markdown. In format/display//
items:
type: string
default: confluence url
ui:options:
addable: true
minItems: 1
maxItems: 5
repoUrl:
type: string
title: GitHub URL mkdocs.yaml link
description: The GitHub repo URL to your mkdocs.yaml file. Example
steps:
- id: create-docs
name: Get markdown file created and update markdown.yaml file
action: confluence:transform:markdown
input:
confluenceUrls: ${{ parameters.confluenceUrls }}
repoUrl: ${{ parameters.repoUrl }}
- id: publish
name: Publish PR to Bitbucket
action: publish:bitbucketServer:pull-request
input:
projectKey: ${{ steps['create-docs'].output.owner }}
repoSlug: ${{ steps['create-docs'].output.repo }}
branchName: confluence-to-markdown
title: Added markdown documentation from Confluence
description: PR for converting confluence page to mkdocs When I realized Bitbucket Server was not supported I made my own custom action for it.
My Bitbucket Server custom action. It will commit files to a branch and create a PR with it
import { createTemplateAction } from '@backstage/plugin-scaffolder-node'; import { z } from 'zod'; const superagent = require('superagent'); const enableProxy = require('superagent-proxy'); const fs = require('fs'); enableProxy(superagent); const baseurl = "git.company.com"; const proxy = "http://proxyurl.com"; // const user = "bac-bot" // const secret = ""; const secret = "mysecret"; const user = "mybot"; async function uploadFile(projectKey: string, repoSlug: string, branchName: string, rootDir: string, filePath: string, latestCommitHash: string): Promise {
try {
await superagent.put(`https://${(user)}:${(secret)}@${(baseurl)}/rest/api/1.0/projects/${(projectKey)}/repos/${(repoSlug)}/browse/${(filePath)}`)
.proxy(proxy)
// .attach('content', "C:/Udvikler/Test-Projekter/test-mepautorisationsoverblikjsfportlet/")
.attach('content', rootDir + filePath)
.field('message', 'Added converted markdown from Confluence')
.field('branch', branchName)
.field('sourceBranch', 'master')
.field('sourceCommitId',latestCommitHash)
} catch (err) {
console.error(err);
} finally {
return "Catalog-info.yaml file successfully pushed!"
}
}
async function getLatestCommitHash(projectKey: string, repoSlug: string): Promise {
const response = await superagent.get(`https://${(user)}:${(secret)}@${(baseurl)}/rest/api/1.0/projects/${projectKey}/repos/${repoSlug}/commits?limit=1`)
.proxy(proxy);
return response.body.values[0]?.id;
}
async function getDefaultReviewers(projectKey: string, repoSlug: string): Promise {
const response = await superagent.get(`https://${(user)}:${(secret)}@${(baseurl)}/rest/default-reviewers/1.0/projects/${(projectKey)}/repos/${(repoSlug)}/conditions`)
.proxy(proxy);
return response?.body[0]?.reviewers?.map((reviewer: any) => reviewer.name) || [];
}
async function createPullRequest(projectKey: string, repoSlug: string, branchName: string, title: string, description: string, defaultReviewers: string[]) {
try {
console.log("Default reviewers are: " + JSON.stringify(defaultReviewers));
const res = await superagent.post(`https://${(user)}:${(secret)}@${(baseurl)}/rest/api/1.0/projects/${(projectKey)}/repos/${(repoSlug)}/pull-requests`)
.send({ title: title,
description: description,
fromRef: { id: branchName },
toRef: { id: 'master' },
reviewers: defaultReviewers.map(reviewer => ({ user: { name: reviewer } })) })
.proxy(proxy);
return res;
} catch (err) {
console.error(err);
}
}
export const bitbucketCreatePullRequest = () => {
return createTemplateAction({
id: 'publish:bitbucketServer:pull-request',
schema: {
input: z.object({
repoSlug: z.string().describe(''),
projectKey: z.string().describe(''),
title: z.string().describe(''),
branchName: z.string().describe(''),
description: z.string().describe(''),
}),
},
async handler(ctx) {
// ctx.logger.info(ctx.input.stf)
// ctx.logger.info("PATH: " + ctx.workspacePath);
const filePath = ctx.workspacePath+'/';
const latestCommitHash = await getLatestCommitHash(ctx.input.projectKey, ctx.input.repoSlug)
// await uploadFile(ctx.input.projectKey, ctx.input.repoSlug, ctx.input.branchName, filePath, 'mkdocs.yml', latestCommitHash)
fs.readdirSync(filePath+"/docs/").forEach((file:string) => {
// if(file === "img") {
// console.log("Found img folder...");
// fs.readdirSync(filePath+"/docs/img/").forEach((file:string) => {
// console.log("In img folder we have: " + file);
// uploadFile(ctx.input.projectKey, ctx.input.repoSlug, ctx.input.branchName, filePath, 'docs/img/'+file, "");
// });
// }
console.log("In folder we have: " + file);
if(file !== "img" )
uploadFile(ctx.input.projectKey, ctx.input.repoSlug, ctx.input.branchName, filePath, 'docs/'+file, "");
});
// ctx.logger.info(JSON.stringify(await uploadFile(ctx.input.projectKey, ctx.input.repoSlug, ctx.input.branchName, ctx.workspacePath, latestCommitHash)));
// const defaultReviewers = await getDefaultReviewers(ctx.input.projectKey, ctx.input.repoSlug);
// ctx.logger.info(JSON.stringify(await createPullRequest(ctx.input.projectKey, ctx.input.repoSlug, ctx.input.branchName, ctx.input.title, ctx.input.description, defaultReviewers)));
// ctx.logger.info(JSON.stringify(await getDefaultReviewers(ctx.input.projectKey, ctx.input.repoSlug)));
},
});
};
Running the template without my own Bitbucket custom action it comes with the same error
📃 Provide the context for the Bug.
This issue prevents me from finishing the template and my custom action. I believe my own custom action still would work but having an error showing when using a template and the fact it does not clean up after itself is not intented.
🖥️ Your Environment
`C:\Udvikler\Projects\backstage_experiment>yarn backstage-cli info yarn run v1.22.19 $ C:\Udvikler\Projects\backstage_experiment\node_modules.bin\backstage-cli info OS: Windows_NT 10.0.19044 - win32/x64 node: v18.16.0 yarn: 1.22.19 cli: 0.22.7 (installed) backstage: 1.13.2
Dependencies: @backstage/app-defaults 1.3.1 @backstage/backend-app-api 0.4.3 @backstage/backend-common 0.18.5 @backstage/backend-dev-utils 0.1.1 @backstage/backend-plugin-api 0.5.2 @backstage/backend-tasks 0.5.2 @backstage/catalog-client 1.4.1 @backstage/catalog-model 1.3.0 @backstage/cli-common 0.1.12 @backstage/cli-node 0.1.0 @backstage/cli 0.22.7 @backstage/config-loader 1.3.0 @backstage/config 1.0.7 @backstage/core-app-api 1.8.0 @backstage/core-components 0.13.1 @backstage/core-plugin-api 1.5.1 @backstage/dev-utils 1.0.15 @backstage/errors 1.1.5 @backstage/eslint-plugin 0.1.3 @backstage/integration-aws-node 0.1.3 @backstage/integration-react 1.1.13 @backstage/integration 1.4.5 @backstage/plugin-api-docs 0.9.3 @backstage/plugin-app-backend 0.3.45 @backstage/plugin-auth-backend 0.18.3 @backstage/plugin-auth-node 0.2.14 @backstage/plugin-bitbucket-cloud-common 0.2.6 @backstage/plugin-catalog-backend-module-bitbucket-cloud 0.1.12 @backstage/plugin-catalog-backend-module-bitbucket-server 0.1.10 @backstage/plugin-catalog-backend 1.9.1 @backstage/plugin-catalog-common 1.0.13 @backstage/plugin-catalog-graph 0.2.30 @backstage/plugin-catalog-import 0.9.8 @backstage/plugin-catalog-node 1.3.6 @backstage/plugin-catalog-react 1.6.0 @backstage/plugin-catalog 1.11.0 @backstage/plugin-events-node 0.2.6 @backstage/plugin-github-actions 0.5.18 @backstage/plugin-org 0.6.8 @backstage/plugin-permission-common 0.7.5 @backstage/plugin-permission-node 0.7.8 @backstage/plugin-permission-react 0.4.12 @backstage/plugin-proxy-backend 0.2.39 @backstage/plugin-scaffolder-backend-module-confluence-to-markdown 0.1.2 @backstage/plugin-scaffolder-backend 1.14.0 @backstage/plugin-scaffolder-common 1.3.0 @backstage/plugin-scaffolder-node 0.1.3 @backstage/plugin-scaffolder-react 1.4.0 @backstage/plugin-scaffolder 1.13.1 @backstage/plugin-search-backend-module-catalog 0.1.1 @backstage/plugin-search-backend-module-pg 0.5.6 @backstage/plugin-search-backend-module-techdocs 0.1.1 @backstage/plugin-search-backend-node 1.2.1 @backstage/plugin-search-backend 1.3.1 @backstage/plugin-search-common 1.2.3 @backstage/plugin-search-react 1.6.0 @backstage/plugin-search 1.3.0 @backstage/plugin-tech-radar 0.6.4 @backstage/plugin-techdocs-backend 1.6.2 @backstage/plugin-techdocs-module-addons-contrib 1.0.13 @backstage/plugin-techdocs-node 1.7.1 @backstage/plugin-techdocs-react 1.1.6 @backstage/plugin-techdocs 1.6.2 @backstage/plugin-user-settings 0.7.3 @backstage/release-manifests 0.0.9 @backstage/test-utils 1.3.1 @backstage/theme 0.2.19, 0.3.0 @backstage/types 1.0.2 @backstage/version-bridge 1.0.4 Done in 1.92s.`
👀 Have you spent some time to check if this bug has been raised before?
🏢 Have you read the Code of Conduct?
Are you willing to submit PR?
Yes I am willing to submit a PR!