search

SAP / ui5-tooling

An open and modular toolchain to develop state of the art applications based on the UI5 framework
https://sap.github.io/ui5-tooling
Apache License 2.0
469 stars 71 forks source link

UI5 tooling rejects paths with parentheses #526

Closed HrFlorianHoffmann closed 3 years ago

HrFlorianHoffmann commented 3 years ago

This issue reopens https://github.com/SAP/ui5-tooling/issues/469.

By chance, I encountered the same problem reported there in one of our pipeline builds. The Jenkins Job was called "Developer Test Pipeline (Own Branch)", i.e. had parentheses in it. UI5 tooling rejected these special characters in the paths of files.

The other issue suggests that the bug should be fixed with @ui5/cli 2.7.2. However, the project we run uses @ui5/cli 2.8.1. This thus might be a regression, or a similar but different case. The other issue reported the problem for a character with an accent, here it is a parentheses.

matz3 commented 3 years ago

I was able to reproduce this problem with the following setup

Environment

git clone git@github.com:SAP/openui5-sample-app.git "openui5-sample-app (master)"
cd "openui5-sample-app (master)"
npm install
npm start

In order to adopt the $HOME path that is used as base for the .ui5 folder, I've patched the following file: node_modules\@ui5\cli\node_modules\@ui5\project\lib\ui5Framework\AbstractResolver.js to assign ui5HomeDir = path.join(process.cwd(), ".ui5"); in the first line of the constructor function.

Verbose log output

C:\SAPDevelop\openui5-sample-app (master)>npm start -- --verbose

> openui5-sample-app@0.2.0 start C:\SAPDevelop\openui5-sample-app (master)
> ui5 serve "--verbose"

verb cli:middlewares:base using @ui5/cli version 2.11.2 (from C:\SAPDevelop\openui5-sample-app (master)\node_modules\@ui5\cli\bin\ui5.js)
verb cli:middlewares:base using node version v12.19.0
verb normalizer:normalizer Building dependency tree...
verb normalizer:translators:npm Analyzing openui5-sample-app (C:\SAPDevelop\openui5-sample-app (master)) (dependency of nothing - root project)
verb normalizer:translators:npm [PERF] Consider defining UI5-dependencies in the package.json files of the relevant modules from the following list to improve npm translator execution time: openui5-sample-app
verb normalizer:translators:npm Treetop:
verb normalizer:translators:npm {
verb normalizer:translators:npm   id: 'openui5-sample-app',
verb normalizer:translators:npm   version: '0.2.0',
verb normalizer:translators:npm   path: 'C:\\SAPDevelop\\openui5-sample-app (master)',
verb normalizer:translators:npm   dependencies: []
verb normalizer:translators:npm }
verb normalizer:projectPreprocessor Processing project openui5-sample-app on level 0...
verb types:application:ApplicationFormatter Formatting application project openui5-sample-app...
verb types:application:ApplicationFormatter Namespace of project openui5-sample-app is sap/ui/demo/todo (from manifest.json)  
verb normalizer:projectPreprocessor Processed 1 projects in 272 ms
verb normalizer:normalizer Root project openui5-sample-app defines framework configuration. Installing UI5 dependencies...
info normalizer:translators:ui5Framework Using OpenUI5 version: 1.91.0
verb normalizer:ui5Framework:npm:Installer Installing to: C:\SAPDevelop\openui5-sample-app (master)\.ui5\framework\packages   
verb ui5Framework:AbstractResolver Processing sap.f
verb ui5Framework:AbstractResolver Processing sap.m
verb ui5Framework:AbstractResolver Processing sap.ui.core
verb ui5Framework:AbstractResolver Processing sap.ui.unified
verb ui5Framework:AbstractResolver Processing themelib_sap_fiori_3
verb normalizer:ui5Framework:npm:Installer Alrady installed: @openui5/sap.f in version 1.91.0
verb normalizer:ui5Framework:npm:Installer Alrady installed: @openui5/sap.m in version 1.91.0
verb normalizer:ui5Framework:npm:Installer Alrady installed: @openui5/sap.ui.core in version 1.91.0
verb normalizer:ui5Framework:npm:Installer Alrady installed: @openui5/sap.ui.unified in version 1.91.0
verb normalizer:ui5Framework:npm:Installer Alrady installed: @openui5/themelib_sap_fiori_3 in version 1.91.0
verb ui5Framework:AbstractResolver Processing dependencies of sap.f
verb ui5Framework:AbstractResolver Processing sap.ui.layout
verb ui5Framework:AbstractResolver Processing dependencies of sap.m
verb ui5Framework:AbstractResolver Done processing dependencies of sap.m
verb ui5Framework:AbstractResolver Processing dependencies of sap.ui.unified
verb ui5Framework:AbstractResolver Done processing dependencies of sap.ui.unified
verb normalizer:ui5Framework:npm:Installer Alrady installed: @openui5/sap.ui.layout in version 1.91.0
verb ui5Framework:AbstractResolver Processing dependencies of sap.ui.layout
verb ui5Framework:AbstractResolver Done processing dependencies of sap.ui.layout
verb ui5Framework:AbstractResolver Done processing dependencies of sap.f
verb normalizer:translators:ui5Framework OpenUI5 dependencies sap.f, sap.m, sap.ui.core, sap.ui.unified, themelib_sap_fiori_3 
resolved in 60 ms
verb normalizer:translators:ui5Framework Creating project for library sap.f...
verb normalizer:translators:ui5Framework Creating project for library sap.m...
verb normalizer:translators:ui5Framework Creating project for library sap.ui.core...
verb normalizer:translators:ui5Framework Creating project for library sap.ui.layout...
verb normalizer:translators:ui5Framework Creating project for library sap.ui.core...
verb normalizer:translators:ui5Framework Returning cached project for library sap.ui.core
verb normalizer:translators:ui5Framework Creating project for library sap.ui.unified...
verb normalizer:translators:ui5Framework Creating project for library sap.ui.core...
verb normalizer:translators:ui5Framework Returning cached project for library sap.ui.core
verb normalizer:translators:ui5Framework Creating project for library sap.ui.core...
verb normalizer:translators:ui5Framework Returning cached project for library sap.ui.core
verb normalizer:translators:ui5Framework Creating project for library sap.ui.layout...
verb normalizer:translators:ui5Framework Returning cached project for library sap.ui.layout
verb normalizer:translators:ui5Framework Creating project for library sap.m...
verb normalizer:translators:ui5Framework Returning cached project for library sap.m
verb normalizer:translators:ui5Framework Creating project for library sap.ui.core...
verb normalizer:translators:ui5Framework Returning cached project for library sap.ui.core
verb normalizer:translators:ui5Framework Creating project for library sap.ui.unified...
verb normalizer:translators:ui5Framework Returning cached project for library sap.ui.unified
verb normalizer:translators:ui5Framework Creating project for library themelib_sap_fiori_3...
verb normalizer:translators:ui5Framework Resolving optional dependency sap.f for project themelib_sap_fiori_3...
verb normalizer:translators:ui5Framework Creating project for library sap.f...
verb normalizer:translators:ui5Framework Returning cached project for library sap.f
verb normalizer:translators:ui5Framework Resolving optional dependency sap.m for project themelib_sap_fiori_3...
verb normalizer:translators:ui5Framework Creating project for library sap.m...
verb normalizer:translators:ui5Framework Returning cached project for library sap.m
verb normalizer:translators:ui5Framework Resolving optional dependency sap.ui.core for project themelib_sap_fiori_3...        
verb normalizer:translators:ui5Framework Creating project for library sap.ui.core...
verb normalizer:translators:ui5Framework Returning cached project for library sap.ui.core
verb normalizer:translators:ui5Framework Resolving optional dependency sap.ui.layout for project themelib_sap_fiori_3...      
verb normalizer:translators:ui5Framework Creating project for library sap.ui.layout...
verb normalizer:translators:ui5Framework Returning cached project for library sap.ui.layout
verb normalizer:translators:ui5Framework Resolving optional dependency sap.ui.unified for project themelib_sap_fiori_3...     
verb normalizer:translators:ui5Framework Creating project for library sap.ui.unified...
verb normalizer:translators:ui5Framework Returning cached project for library sap.ui.unified
verb normalizer:projectPreprocessor Processing project openui5-sample-app on level 0...
verb normalizer:projectPreprocessor Processing dependency lookahead for openui5-sample-app: @openui5/sap.f
verb normalizer:projectPreprocessor Processing dependency lookahead for openui5-sample-app: @openui5/sap.m
verb normalizer:projectPreprocessor Processing dependency lookahead for openui5-sample-app: @openui5/sap.ui.core
verb normalizer:projectPreprocessor Processing dependency lookahead for openui5-sample-app: @openui5/sap.ui.unified
verb normalizer:projectPreprocessor Processing dependency lookahead for openui5-sample-app: @openui5/themelib_sap_fiori_3     
verb normalizer:projectPreprocessor Processing project @openui5/sap.f on level 1...
verb normalizer:projectPreprocessor Dependency of project @openui5/sap.f, "@openui5/sap.m": Distance to root of 2. Will be replaced by project with same ID and distance to root of undefined.
verb normalizer:projectPreprocessor Dependency of project @openui5/sap.f, "@openui5/sap.ui.core": Distance to root of 2. Will 
be replaced by project with same ID and distance to root of undefined.
verb normalizer:projectPreprocessor Processing dependency lookahead for @openui5/sap.f: @openui5/sap.ui.layout
verb normalizer:projectPreprocessor Processing project @openui5/sap.m on level 1...
verb normalizer:projectPreprocessor Dependency of project @openui5/sap.m, "@openui5/sap.ui.core": Distance to root of 2. Will 
be replaced by project with same ID and distance to root of undefined.
verb normalizer:projectPreprocessor Dependency of project @openui5/sap.m, "@openui5/sap.ui.layout": Distance to root of 2. Will be replaced by project with same ID and distance to root of undefined.
verb normalizer:projectPreprocessor Dependency of project @openui5/sap.m, "@openui5/sap.ui.unified": Distance to root of 2. Will be replaced by project with same ID and distance to root of undefined.
verb normalizer:projectPreprocessor Processing project @openui5/sap.ui.core on level 1...
verb normalizer:projectPreprocessor Processing project @openui5/sap.ui.unified on level 1...
verb normalizer:projectPreprocessor Dependency of project @openui5/sap.ui.unified, "@openui5/sap.ui.core": Distance to root of 2. Will be replaced by project with same ID and distance to root of 1.
verb normalizer:projectPreprocessor Processing project @openui5/themelib_sap_fiori_3 on level 1...
verb normalizer:projectPreprocessor Dependency of project @openui5/themelib_sap_fiori_3, "@openui5/sap.f": Distance to root of 2. Will be replaced by project with same ID and distance to root of 1.
verb normalizer:projectPreprocessor Dependency of project @openui5/themelib_sap_fiori_3, "@openui5/sap.m": Distance to root of 2. Will be replaced by project with same ID and distance to root of 1.
verb normalizer:projectPreprocessor Dependency of project @openui5/themelib_sap_fiori_3, "@openui5/sap.ui.core": Distance to root of 2. Will be replaced by project with same ID and distance to root of 1.
verb normalizer:projectPreprocessor Dependency of project @openui5/themelib_sap_fiori_3, "@openui5/sap.ui.layout": Distance to root of 2. Will be replaced by project with same ID and distance to root of undefined.
verb normalizer:projectPreprocessor Dependency of project @openui5/themelib_sap_fiori_3, "@openui5/sap.ui.unified": Distance to root of 2. Will be replaced by project with same ID and distance to root of 1.
verb types:library:LibraryFormatter Could not find (optional) test directory of project @openui5/sap.ui.core: C:\SAPDevelop\openui5-sample-app (master)\.ui5\framework\packages\@openui5\sap.ui.core\1.91.0\test
verb types:library:LibraryFormatter Formatting library project sap.ui.core...
verb types:library:LibraryFormatter Ignoring 'test' directory for project sap.ui.core. Either no setting was provided or the path not found.
verb types:library:LibraryFormatter Could not find (optional) test directory of project @openui5/sap.m: C:\SAPDevelop\openui5-sample-app (master)\.ui5\framework\packages\@openui5\sap.m\1.91.0\test
verb types:library:LibraryFormatter Formatting library project sap.m...
verb types:library:LibraryFormatter Ignoring 'test' directory for project sap.m. Either no setting was provided or the path not found.
verb types:library:LibraryFormatter Could not find (optional) test directory of project @openui5/sap.ui.unified: C:\SAPDevelop\openui5-sample-app (master)\.ui5\framework\packages\@openui5\sap.ui.unified\1.91.0\test
verb types:library:LibraryFormatter Formatting library project sap.ui.unified...
verb types:library:LibraryFormatter Ignoring 'test' directory for project sap.ui.unified. Either no setting was provided or the path not found.
verb types:themeLibrary:ThemeLibraryFormatter Could not find (optional) test directory of project @openui5/themelib_sap_fiori_3: C:\SAPDevelop\openui5-sample-app (master)\.ui5\framework\packages\@openui5\themelib_sap_fiori_3\1.91.0\test
verb types:themeLibrary:ThemeLibraryFormatter Formatting theme-library project themelib_sap_fiori_3...
verb types:themeLibrary:ThemeLibraryFormatter Ignoring 'test' directory for project themelib_sap_fiori_3. Either no setting was provided or the path not found.
verb types:library:LibraryFormatter Could not find (optional) test directory of project @openui5/sap.f: C:\SAPDevelop\openui5-sample-app (master)\.ui5\framework\packages\@openui5\sap.f\1.91.0\test
verb types:library:LibraryFormatter Formatting library project sap.f...
verb types:library:LibraryFormatter Ignoring 'test' directory for project sap.f. Either no setting was provided or the path not found.
verb types:library:LibraryFormatter Namespace resolution from manifest.json failed for project sap.m: Could not find manifest.json file for project sap.m
verb types:library:LibraryFormatter Namespace resolution from manifest.json failed for project sap.ui.unified: Could not find 
manifest.json file for project sap.ui.unified
verb types:library:LibraryFormatter Found namespace sap.m in .library file of project sap.m at C:\SAPDevelop\openui5-sample-app (master)\.ui5\framework\packages\@openui5\sap.m\1.91.0\src\sap\m\.library
verb types:library:LibraryFormatter Resolving namespace of project sap.m from .library...

⚠️  Process Failed With Error

Error Message:
Given file system path C:/SAPDevelop/openui5-sample-app (master)/.ui5/framework/packages/@openui5/sap.m/1.91.0/src/sap/m is not based on source base path C:/SAPDevelop/openui5-sample-app (master)/.ui5/framework/packages/@openui5/sap.m/1.91.0/src.      

Stack Trace:
Error: Given file system path C:/SAPDevelop/openui5-sample-app (master)/.ui5/framework/packages/@openui5/sap.m/1.91.0/src/sap/m is not based on source base path C:/SAPDevelop/openui5-sample-app (master)/.ui5/framework/packages/@openui5/sap.m/1.91.0/src.
    at LibraryFormatter.getNamespaceFromFsPath (C:\SAPDevelop\openui5-sample-app (master)\node_modules\@ui5\cli\node_modules\@ui5\builder\lib\types\library\LibraryFormatter.js:289:10)
    at LibraryFormatter.getNamespace (C:\SAPDevelop\openui5-sample-app (master)\node_modules\@ui5\cli\node_modules\@ui5\builder\lib\types\library\LibraryFormatter.js:194:31)
    at async LibraryFormatter.format (C:\SAPDevelop\openui5-sample-app (master)\node_modules\@ui5\cli\node_modules\@ui5\builder\lib\types\library\LibraryFormatter.js:39:33)
    at async ProjectPreprocessor.applyType (C:\SAPDevelop\openui5-sample-app (master)\node_modules\@ui5\cli\node_modules\@ui5\project\lib\projectPreprocessor.js:333:3)
    at async C:\SAPDevelop\openui5-sample-app (master)\node_modules\@ui5\cli\node_modules\@ui5\project\lib\projectPreprocessor.js:92:7
    at async Promise.all (index 1)
    at async ProjectPreprocessor.processTree (C:\SAPDevelop\openui5-sample-app (master)\node_modules\@ui5\cli\node_modules\@ui5\project\lib\projectPreprocessor.js:62:4)
    at async Object.generateProjectTree (C:\SAPDevelop\openui5-sample-app (master)\node_modules\@ui5\cli\node_modules\@ui5\project\lib\normalizer.js:42:21)
    at async Object.serve.handler (C:\SAPDevelop\openui5-sample-app (master)\node_modules\@ui5\cli\lib\cli\commands\serve.js:91:15)

If you think this is an issue of the UI5 Tooling, you might report it using the following URL: https://github.com/SAP/ui5-tooling/issues/new/choose
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! openui5-sample-app@0.2.0 start: `ui5 serve "--verbose"`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the openui5-sample-app@0.2.0 start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
RandomByte commented 3 years ago

This is likely caused by this regex: https://github.com/SAP/ui5-builder/blob/037b0a690b3044418d069c0489002d39d33bca7c/lib/types/library/LibraryFormatter.js#L280

I guess a simple startsWith would do the trick just as good.

This can also be reproduced on any platform by building a library which contains regular expression metacharacters in their path. For example the OpenUI5 library sap.ui.core after renaming its directory to sap.ui.(core). I'll prepare a fix

RandomByte commented 3 years ago

https://github.com/SAP/ui5-builder/pull/622 fixes the issue at hand.

However I noticed that we dynamically create regular expressions from project namespaces on several occasions: https://github.com/SAP/ui5-builder/blob/037b0a690b3044418d069c0489002d39d33bca7c/lib/builder/builder.js#L386 https://github.com/SAP/ui5-builder/blob/037b0a690b3044418d069c0489002d39d33bca7c/lib/processors/bundlers/manifestBundler.js#L153 https://github.com/SAP/ui5-builder/blob/037b0a690b3044418d069c0489002d39d33bca7c/lib/tasks/generateCachebusterInfo.js#L47

This would cause similar issues in cases where the project namespace contains regular expression metacharacters. While I think this is unlikely to happen (and easily fixable), we should probably implement a validation of the namespace. Ideally we implement it now and log a warning, so we can switch it to throwing an error with UI5 Tooling 3.0.

RandomByte commented 3 years ago

I resolved all mentioned cases of creating a RegExp from a project's namespace in the same PR.