search

vmware / build-tools-for-vmware-aria

Build Tools for VMware Aria provides development and release management tools for implementing automation solutions based on the VMware Aria Suite and VMware Cloud Director. The solution enables Virtual Infrastructure Administrators and Automation Developers to use standard DevOps practices for managing and deploying content.
Other
48 stars 24 forks source link

vro:pull command for legacy archetype fails when workflow folder path name contains character & #165

Closed htourneu closed 9 months ago

htourneu commented 1 year ago

Description

When running a vro:pull command on legacy workflow archetype, if those workflows paths name contains the & character, the pull command will faill with the log below

Steps to Reproduce

  1. Have a vro package containing workflows which paths names (folder name) include the &
  2. Try a vro:pull command of that package
  3. See the maven vro:pull command fails

Preconditions: Have the "&" character in the workflow folder path name

Expected behavior: vRBT to handle this character as any other character and succeed

Actual behavior: It fails when running the vro:pull command

Reproduces how often: 100%

Component/s: maven/plugins/vropkg

Affects Build/s: 2.32.0

Environment

Client

Server

Failure Logs

[INFO] C:\Users\username\projects\windows_vra_catalog\legacy\workflows>node "C:\Users\username\projects\windows_vra_catalog\legacy\workflows\node_modules\@vmware-pscoe\vropkg\bin\\..\dist\cli.js" --in flat --srcPath C:\Users\LADM_T~1\AppData\Local\Temp\2\vro-xml-pull1906541284324290868\com.company.vrbt.winvra.legacy-1.0.0-SNAPSHOT.package --out tree --destPath C:\Users\username\projects\windows_vra_catalog\legacy\workflows --privateKeyPEM C:\.keystore\private_key.pem --certificatesPEM C:\.keystore\cert.pem --keyPass VMware1! --version 1.0.0-SNAPSHOT --packaging package --artifactId legacy --description "Build Tools for VMware Aria provides development and release management tools for implementing automation solutions                 based on the VMware Aria Suite and VMware Cloud Director. The solution enables Virtual Infrastructure Administrators and                  Automation Developers to use standard DevOps practices for managing and deploying content." --groupId com.company.vrbt.winvra
[INFO] info: Extracting package C:\Users\LADM_T~1\AppData\Local\Temp\2\vro-xml-pull1906541284324290868\com.company.vrbt.winvra.legacy-1.0.0-SNAPSHOT.package to "C:\Users\username\projects\windows_vra_catalog\legacy\workflows" folder...
[ERROR] C:\Users\username\projects\windows_vra_catalog\legacy\workflows\node_modules\@vmware-pscoe\vropkg\dist\serialize\tree.js:85
[ERROR]         : element.categoryPath.map(c => c.replace(/\./g, "/."));
[ERROR]                                           ^
[ERROR]
[ERROR] TypeError: Cannot read properties of undefined (reading 'replace')
[ERROR]     at C:\Users\username\projects\windows_vra_catalog\legacy\workflows\node_modules\@vmware-pscoe\vropkg\dist\serialize\tree.js:85:43
[ERROR]     at Array.map (<anonymous>)
[ERROR]     at C:\Users\username\projects\windows_vra_catalog\legacy\workflows\node_modules\@vmware-pscoe\vropkg\dist\serialize\tree.js:85:32
[ERROR]     at Generator.next (<anonymous>)
[ERROR]     at C:\Users\username\projects\windows_vra_catalog\legacy\workflows\node_modules\@vmware-pscoe\vropkg\dist\serialize\tree.js:8:71
[ERROR]     at new Promise (<anonymous>)
[ERROR]     at __awaiter (C:\Users\username\projects\windows_vra_catalog\legacy\workflows\node_modules\@vmware-pscoe\vropkg\dist\serialize\tree.js:4:12)
[ERROR]     at serializeTreeElement (C:\Users\username\projects\windows_vra_catalog\legacy\workflows\node_modules\@vmware-pscoe\vropkg\dist\serialize\tree.js:73:52)
[ERROR]     at C:\Users\username\projects\windows_vra_catalog\legacy\workflows\node_modules\@vmware-pscoe\vropkg\dist\serialize\tree.js:118:37
[ERROR]     at Array.forEach (<anonymous>)
[ERROR]     at Object.serializeTree (C:\Users\username\projects\windows_vra_catalog\legacy\workflows\node_modules\@vmware-pscoe\vropkg\dist\serialize\tree.js:118:18)
[ERROR]     at C:\Users\username\projects\windows_vra_catalog\legacy\workflows\node_modules\@vmware-pscoe\vropkg\dist\cli.js:124:24
[ERROR]     at Generator.next (<anonymous>)
[ERROR]     at fulfilled (C:\Users\username\projects\windows_vra_catalog\legacy\workflows\node_modules\@vmware-pscoe\vropkg\dist\cli.js:5:58)
[INFO] Running vropkg... finished
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for legacy-all 1.0.0-SNAPSHOT:
[INFO] 
[INFO] legacy.actions ..................................... SUCCESS [01:36 min]
[INFO] legacy ............................................. FAILURE [01:21 min]
[INFO] legacy-all ......................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  03:36 min
[INFO] Finished at: 2023-09-05T15:27:42+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.vmware.pscoe.maven.plugins:o11n-xml-package-maven-plugin:2.35.0:pull (default-cli) on project legacy: Running vropkg... failed with code 1 -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :legacy
htourneu commented 1 year ago

Tagging @Michaelpalacce as we discussed that issue earlier.

VenelinBakalov commented 11 months ago

Hey @htourneu , could you check if the issue still persists on your side with the latest Build Tools for Aria version? On my side I have a test project setup with XML module, Build Tools version 2.36.0 and it seems to parse the tags / path correctly and execute both local pull and push with changes and new tags added to the path: image image image

@Michaelpalacce by any chance, do you know if someone has worked on fixing the issue but have not closed the Github ticket?

Michaelpalacce commented 11 months ago

@VenelinBakalov the issue is with the symbol "&". Have you tested with that?

Michaelpalacce commented 11 months ago

I think we need to figure out characters that are universally acceptable in all operating systems And provide a descriptive error message what the issue is, cause currently we get some nonesense.

Michaelpalacce commented 11 months ago

@ognqnai will take over this one

Michaelpalacce commented 11 months ago

Taking a look at: vropkg/src/serialize/tree.ts

image

We can add a check for the acceptable characters before line 89

VenelinBakalov commented 11 months ago

@Michaelpalacce yeap, I was testing with &. Just in the screenshots above it was automatically translated to & but on the vRO screenshot it is visible and seems to be working ok at least for me. https://github.com/vmware/build-tools-for-vmware-aria/issues/165#issuecomment-1855301534 @ognqnai I can also provide access to the environment I was using for testing if that would help

Michaelpalacce commented 11 months ago

@VenelinBakalov this is also OS dependant I beleive, so it'd work on mac but maybe not on Windows. Ogi is working on adding a filter that will give out an error in such cases

VenelinBakalov commented 11 months ago

I see, ok that sounds good. Still, it would be good if someone on Windows can confirm that it is still breaking. But regardless of that, I agree that it is good to add such filter.

ognqnai commented 11 months ago

I have checked at: vropkg/src/serialize/tree.ts

When the name includes unsupported signs like "&," the name becomes "undefined."

The name was gotten from vropkg/src/parse/util.ts  e.attr.name. I assume somewhere this name is filtered, but I can't find where.

exports.xmlToCategory = (categories) => categories.children.map(e => {    return e.attr.name; });

Example log for category with correct name:

[ [INFO]   XmlElement { [INFO]     name: 'category', [INFO]     attr: { name: 'TestName' }, [INFO]     val: '', [INFO]     children: [ [XmlElement] ], [INFO]     firstChild: XmlElement { [INFO]       name: 'name', [INFO]       attr: {}, [INFO]       val: 'TestName', [INFO]       children: [Array], [INFO]       firstChild: [XmlCDataNode], [INFO]       lastChild: [XmlCDataNode], [INFO]       line: 0, [INFO]       column: 48, [INFO]       position: 48, [INFO]       startTagPosition: 43 [INFO]     }, [INFO]     lastChild: XmlElement { [INFO]       name: 'name', [INFO]       attr: {}, [INFO]       val: 'TestName', [INFO]       children: [Array], [INFO]       firstChild: [XmlCDataNode], [INFO]       lastChild: [XmlCDataNode], [INFO]       line: 0, [INFO]       column: 48, [INFO]       position: 48, [INFO]       startTagPosition: 43 [INFO]     }, [INFO]     line: 0, [INFO]     column: 42, [INFO]     position: 42, [INFO]     startTagPosition: 13 [INFO]   } [INFO] ]

Example log for a category with an incorrect name:

[ [INFO]   XmlElement { [INFO]     name: 'category', [INFO]     attr: {}, [INFO]     val: '', [INFO]     children: [ [XmlElement] ], [INFO]     firstChild: XmlElement { [INFO]       name: 'name', [INFO]       attr: {}, [INFO]       val: 'Workload & Onboarding', [INFO]       children: [Array], [INFO]       firstChild: [XmlCDataNode], [INFO]       lastChild: [XmlCDataNode], [INFO]       line: 0, [INFO]       column: 106, [INFO]       position: 106, [INFO]       startTagPosition: 101 [INFO]     }, [INFO]     lastChild: XmlElement { [INFO]       name: 'name', [INFO]       attr: {}, [INFO]       val: 'Workload & Onboarding', [INFO]       children: [Array], [INFO]       firstChild: [XmlCDataNode], [INFO]       lastChild: [XmlCDataNode], [INFO]       line: 0, [INFO]       column: 106, [INFO]       position: 106, [INFO]       startTagPosition: 101 [INFO]     }, [INFO]     line: 0, [INFO]     column: 100, [INFO]     position: 100, [INFO]     startTagPosition: 91 [INFO]   } [INFO] ]

sugeeshC commented 9 months ago

Cause for the issue :-

When there are special characters('&') in the workflow path, inside the downloading temporary package (C:\Users\xxx\AppData\Local\Temp\vro-xml-pullXXX), 'package\elements\xxx\categories' file doesn't contains the 'name' attribute in the 'category' tag.

Without special characters in the path (/character-issue/TestFolder/)

<categories>
    <category name='character-issue'>
        <name><![CDATA[character-issue]]></name>
    </category>
    <category name='TestFolder'>
        <name><![CDATA[TestFolder]]></name>
    </category>
</categories>

With special characters in the path (/character-issue/TestFolder&/)

<categories>
    <category name='character-issue'>
        <name><![CDATA[character-issue]]></name>
    </category>
    <category>
        <name><![CDATA[TestFolder&]]></name>
    </category>
</categories>

Issue is generating from the JS files, when we accessing the attribute.

Tested OS :- Both in Windows & MAC, the issue is generated.

Current Solution :- I am working on implementation adding descriptive error message with the error. (check on whether attribute have a value from the typescript and using CDATA value to generate the error message)

sugeeshC commented 9 months ago

During my recent implementation testing, I observed that prior to the implementation in the "vRO JS" type projects with the same scenario, there were no error during the project pull operation. But it's ignoring the special character folder.

/folder1/test&/folder2 -> /folder1/folder2

Given this behavior, we decided to apply a similar strategy to "XML Legacy" type projects and add warning logs in the both scenarios.