[cds-plugin-ui5] running mbt build with @sap/cds-dk@8 gives an error: Invalid priority value for UI5Plugin

jrramsaransing commented 2 months ago

Describe the bug We are using cds-plugin-ui5 and we where running mbt build and where greeted with an error today (15-07-2024). It was working before, the last time we tried it was 11-07-2024 and it worked (locally and via the azure pipeline).

After a lot of trying we found it's because of this command in our mta.yaml: - npx -p @sap/cds-dk cds build --production when we change it to - npx -p @sap/cds-dk@7 cds build --production, notice the @7 after cds-dk package it builds with success.

We got the same error in our azure devops pipeline, luckily we could recreate locally.

To Reproduce Steps to reproduce the behavior:

We have an MTA application, it consist of cds module and a fiori module with an mta.yaml file

In the mta.yaml we have the following:

...
build-parameters:
before-all:
- builder: custom
commands:
- npx -p @sap/cds-dk cds build --production

Via the terminal, we used mbt build command and we got an error.
See error

[2024-07-15 17:06:39]  INFO Cloud MTA Build Tool version 1.2.30
[2024-07-15 17:06:39]  INFO generating the "Makefile_20240715170639.mta" file...
[2024-07-15 17:06:39]  INFO done
[2024-07-15 17:06:39]  INFO executing the "make -f Makefile_20240715170639.mta p=cf mtar= strict=true mode=" command...
[2024-07-15 17:06:39]  INFO validating the MTA project
[2024-07-15 17:06:39]  INFO running the "before-all" build...
[2024-07-15 17:06:39]  INFO executing the "npx -p @sap/cds-dk cds build --production" command...
building project [/home/user/projects/SAP-BTP-GZ-PROJECTS], clean [true]
cds-dk [8.0.2], cds [7.9.3], compiler [4.9.4], home [/home/user/projects/SAP-BTP-GZ-PROJECTS/node_modules/@sap/cds]

{
  build: {
    target: 'gen',
    tasks: [
      { for: 'hana', src: 'db', options: { model: ['db', 'srv', 'app'] }},
      { for: 'nodejs', src: 'srv', options: { model: ['db', 'srv', 'app'] }},
      { for: 'ui5', src: 'srv', options: { model: ['db', 'srv', 'app'] }}
    ]
  }
}

Error: Invalid priority value for UI5Plugin.
The valid priority value range for custom plugins is -1024..-1 and 512..1024.
Range 0..512 is blocked for internal plugins.
The higher the value the earlier the plugin is run.
    at BuildTaskEngine._createPlugin (/home/user/.npm/_npx/b993340195f82cd5/node_modules/@sap/cds-dk/lib/build/buildTaskEngine.js:212:19)
    at /home/user/.npm/_npx/b993340195f82cd5/node_modules/@sap/cds-dk/lib/build/buildTaskEngine.js:49:37
    at Array.forEach (<anonymous>)
    at BuildTaskEngine.processTasks (/home/user/.npm/_npx/b993340195f82cd5/node_modules/@sap/cds-dk/lib/build/buildTaskEngine.js:47:15)
    at async build (/home/user/.npm/_npx/b993340195f82cd5/node_modules/@sap/cds-dk/lib/build/index.js:19:16)
    at async Object.build (/home/user/.npm/_npx/b993340195f82cd5/node_modules/@sap/cds-dk/bin/build.js:171:5)
    at async Object.exec (/home/user/.npm/_npx/b993340195f82cd5/node_modules/@sap/cds-dk/bin/cds.js:81:16)
[2024-07-15 17:06:40] ERROR the "before-all"" build failed: could not execute the "npx -p @sap/cds-dk cds build --production" command: exit status 1
make: *** [Makefile_20240715170639.mta:28: pre_build] Error 1
[2024-07-15 17:06:40] ERROR could not build the MTA project: could not execute the "make -f Makefile_20240715170639.mta p=cf mtar= strict=true mode=" command: exit status 2
Error: could not build the MTA project: could not execute the "make -f Makefile_20240715170639.mta p=cf mtar= strict=true mode=" command: exit status 2

Expected behavior A successful mbt build with @sap/cds-dk@8.

Desktop (please complete the following information):

OS: A snippet from inxi

System:
Kernel: 6.6.34-1-MANJARO arch: x86_64 bits: 64 compiler: gcc v: 14.1.1
Desktop: KDE Plasma v: 6.0.5 Distro: Manjaro base: Arch Linux

Package.json: a snippet

...
"dependencies": {
    "@azure/storage-blob": "12.23.0",
    "@cap-js-community/odata-v2-adapter": "1.12.11",
    "@esri/arcgis-rest-feature-layer": "3.7.0",
    "@sap-cloud-sdk/http-client": "3.16.0",
    "@sap-cloud-sdk/util": "3.16.0",
    "@sap/cds": "7.9.3",
    "@sap/cds-hana": "2.0.0",
    "@sap/textbundle": "5.1.0",
    "@sap/xssec": "4.1.3",
    "base64-stream": "1.0.0",
    "decimal.js": "10.4.3",
    "email-templates": "11.1.1",
    "express": "4.19.2",
    "hdb": "0.19.8",
    "luxon": "3.4.4",
    "passport": "0.7.0",
    "pdfmake": "0.2.10",
    "sap-cf-mailer": "0.0.5"
},
"devDependencies": {
    "@cap-js/sqlite": "1.7.2",
    "cds-plugin-ui5": "0.9.3",
    "cds-swagger-ui-express": "0.8.0",
    "eslint": "9.6.0",
    "jest": "29.7.0",
    "jest-jasmine2": "29.7.0",
    "jest-junit": "16.0.0",
    "rimraf": "5.0.7",
    "supertest": "7.0.0"
},
...
"cds": {
    "cds-plugin-ui5": {
        "modules": {
            "fiorimodule": {
                "configFile": "ui5.yaml",
                "mountPath": "/fiorimodule/webapp"
            }
        }
    },
...

Additional context Is this the right place to report this issue or does this need to be reported via SAP for me and make a case of it? If you need more information, let me know!

sammok83 commented 2 months ago

Confirmed I have this issue as well

petermuessig commented 2 months ago

@chgeo - seems that exactly that happened what I explained what we discussed. You fixed the trick and now the build plugin doesn't work anymore!

petermuessig commented 2 months ago

FYI: I told the CAP colleagues about that trick to ask for a proper solution. We need to align here with an official support to hook in as the last plugin to fix the build. Without the support of the CAP colleagues I can't help here...

chgeo commented 2 months ago

@petermuessig That was certainly not done to block you :)

@LotharBender can you check if we get this scenario working short-term? See the item in cap/cds-tools#1323

LotharBender commented 2 months ago

@petermuessig - you can fix the issue by simply returning the priority value -1 - the return this._priority || 1; hack is no longer required. This ensures that your plugin gets executed after the internal plugins which have a priority value range from 0 - 512.

petermuessig commented 2 months ago

@LotharBender thx for letting me know. As the cds-plugin-ui5 works with older versions of CDS as well, I need to keep this running compatibility wise. Means with @sap/cds-dk@8 I need to use the new priority handling, right?

petermuessig commented 2 months ago

Change is in prep - I now made it even possible to continue to work with older releases and newer releases of the cds-dk.

petermuessig commented 2 months ago

New version is available - please update

sammok83 commented 1 month ago

@petermuessig I'm still getting the error even after updating:

petermuessig commented 1 month ago

@sammok83: what version of the @sap/cds-dk do you have installed? Somehow the version detection of the @sap/cds-dk doesn't work for you. Can you show me the result of cds --version after running it in your project directory?

And please show me this part of the log:

petermuessig commented 1 month ago

I release a new patch #1045 which now states the proper versions also for the cds-plugin-ui5 to get more insights for such scenarios. I assume that there is a version conflict somehwere - I also saw that in some cases, the cds-plugin loads 2 versions of cds and this can also cause such issues because internally in @sap/cds sometimes they load it with require and sometimes with require(require.resolve(..., paths)) which is slightly different - but I didn't trace this down so much. Would be also interesting if you are in a mono repo or so?

sammok83 commented 1 month ago

@petermuessig The project is in BAS

devinea commented 1 month ago

@petermuessig I am having similar problems with cds watch https://github.com/devinea/btp-lcnc-capex branch cds8_watch_broken In BAS the plugin cds-plugin-ui5 is not loaded when running npm run watch-project1. It is loaded fine using npm run start.

user: btp-lcnc-capex $ npm run watch-project1

> capex@1.0.0 watch-project1
> cds watch --open project1/index.html?sap-ui-xx-viewCache=false --livereload false

cds serve all --with-mocks --in-memory? 

        ___________________________

[cds] - loaded model from 5 file(s):

  app/services.cds
  app/project1/annotations.cds
  srv/service.cds
  db/schema.cds
  node_modules/@sap/cds/common.cds

[cds] - connect using bindings from: { registry: '~/.cds-services.json' }
[cds] - connect to db > sqlite { url: ':memory:' }
  > init from db/data/sap.ui.lcnc-CapexType.csv 
  > init from db/data/sap.ui.lcnc-Capex.csv 
  > init from db/data/sap.ui.lcnc-BusinessUnits.csv 
  > init from db/data/sap.common-Currencies_texts.csv 
  > init from db/data/sap.common-Currencies.csv 
  > init from db/data/sap.common-Countries.csv 
/> successfully deployed to in-memory database. 

[cds] - using auth strategy { kind: 'dummy', impl: 'node_modules/@sap/cds/lib/auth/dummy-auth' } 

[cds] - serving MainOnlineService { path: '/serviceonline' }

-----------------------------------------------------------------------
WARNING: Package '@sap/cds' was loaded from different installations: [
  '/home/user/projects/btp-lcnc-capex/node_modules/@sap/cds/lib/index.js',
  '/managed-content/globals/pnpm/5/.pnpm/@sap+cds@8.0.4_express@4.19.2/node_modules/@sap/cds/lib/index.js'
] Rather ensure a single install only to avoid hard-to-resolve errors.
-----------------------------------------------------------------------

[cds] - server listening on { url: 'http://localhost:4004' }
[cds] - launched at 7/22/2024, 10:57:51 AM, version: 7.9.4, in: 493.67ms
[cds] - [ terminate with ^C ]

Error opening 'open-browser http://localhost:4004/project1/index.html?sap-ui-xx-viewCache=false' : Command failed: open-browser http://localhost:4004/project1/index.html?sap-ui-xx-viewCache=false
Error: connect ENOENT /extbin/basctl-7e34db04-5bc9-4283-b3b8-a016315e5daf.sock
    at PipeConnectWrap.afterConnect [as oncomplete] (node:net:1605:16)

[odata] - HEAD /serviceonline/ 
[odata] - GET /serviceonline/$metadata?sap-language=EN

user: btp-lcnc-capex $ cds -v
@cap-js/asyncapi: 1.0.2
@cap-js/openapi: 1.0.4
@sap/cds: 7.9.4
@sap/cds-compiler: 4.9.6
@sap/cds-dk: 8.0.3
@sap/cds-dk (global): 8.0.3
@sap/cds-fiori: 1.2.7
@sap/cds-foss: 5.0.1
@sap/cds-mtxs: 2.0.3
@sap/eslint-plugin-cds: 3.0.4
Node.js: v20.12.0
capex: 1.0.0
home: /home/user/projects/btp-lcnc-capex/node_modules/@sap/cds

Switching the project to cds 8 does not help.

This error seems relevant as locally on macOS the project loads the ui5 plugin for npm run watch-project1 ok.

WARNING: Package '@sap/cds' was loaded from different installations: [
  '/home/user/projects/btp-lcnc-capex/node_modules/@sap/cds/lib/index.js',
  '/managed-content/globals/pnpm/5/.pnpm/@sap+cds@8.0.4_express@4.19.2/node_modules/@sap/cds/lib/index.js'
] Rather ensure a single install only to avoid hard-to-resolve errors.

Also in BAS node /managed-content/globals/pnpm/5/node_modules/@sap/cds-dk/bin/cds.js watch --open project1/index.html?sap-ui-xx-viewCache=false --livereload false loads the plugin fine and also '@sap/cds' was loaded from different installations is not shown. But npm run watch-project1 fails to load the plugin.

petermuessig commented 1 month ago

@sammok83 @devinea - I released a new patch of the plugin with some advanced logging for duplicate @sap/cds versions - can you please update and verify and send me the part with the warnings of the duplicate @sap/cds entries please? You should find that in the log. THX

sammok83 commented 1 month ago

@petermuessig Here you go:

petermuessig commented 1 month ago

@sammok83 thx - strange - no warning but what I can see it are two different @sap/cds-dk versions - I try to reproduce this locally - let's see where this issue comes from - really strange...

petermuessig commented 1 month ago

Hmmm, still wondering about the following check not working in your case - a bit strange :-( :

if (!satisfies(cdsdkVersion, ">=8")) {

devinea commented 1 month ago

@petermuessig Here you go

user: btp-lcnc-capex $ npm run watch-project1

> capex@1.0.0 watch-project1
> cds watch --open project1/index.html?sap-ui-xx-viewCache=false --livereload false

cds serve all --with-mocks --in-memory? 

        ___________________________

[cds-plugin-ui5] [info] Running cds-plugin-ui5@0.9.5 (@sap/cds-dk@8.0.3, @sap/cds@8.0.4)
[cds-plugin-ui5] [warn]   !! Multiple versions of @sap/cds loaded !!
[cds-plugin-ui5] [warn]     => /home/user/projects/btp-lcnc-capex/node_modules/@sap/cds/lib/index.js
[cds-plugin-ui5] [warn]     => /managed-content/globals/pnpm/5/.pnpm/@sap+cds@8.0.4_express@4.19.2/node_modules/@sap/cds/lib/index.js
[cds] - loaded model from 5 file(s):

  app/services.cds
  app/project1/annotations.cds
  srv/service.cds
  db/schema.cds
  node_modules/@sap/cds/common.cds

[cds] - connect using bindings from: { registry: '~/.cds-services.json' }
[cds] - connect to db > sqlite { url: ':memory:' }
  > init from db/data/sap.ui.lcnc-CapexType.csv 
  > init from db/data/sap.ui.lcnc-Capex.csv 
  > init from db/data/sap.ui.lcnc-BusinessUnits.csv 
  > init from db/data/sap.common-Currencies_texts.csv 
  > init from db/data/sap.common-Currencies.csv 
  > init from db/data/sap.common-Countries.csv 
/> successfully deployed to in-memory database. 

[cds] - using auth strategy { kind: 'dummy', impl: 'node_modules/@sap/cds/lib/auth/dummy-auth' } 

[cds] - serving MainOnlineService { path: '/serviceonline' }

-----------------------------------------------------------------------
WARNING: Package '@sap/cds' was loaded from different installations: [
  '/home/user/projects/btp-lcnc-capex/node_modules/@sap/cds/lib/index.js',
  '/managed-content/globals/pnpm/5/.pnpm/@sap+cds@8.0.4_express@4.19.2/node_modules/@sap/cds/lib/index.js'
] Rather ensure a single install only to avoid hard-to-resolve errors.
-----------------------------------------------------------------------

[cds] - server listening on { url: 'http://localhost:4004' }
[cds] - launched at 7/22/2024, 12:25:05 PM, version: 7.9.4, in: 510.318ms
[cds] - [ terminate with ^C ]

Error opening 'open-browser http://localhost:4004/project1/index.html?sap-ui-xx-viewCache=false' : Command failed: open-browser http://localhost:4004/project1/index.html?sap-ui-xx-viewCache=false
Error: connect ENOENT /extbin/basctl-7ee063e7-5516-46f5-b010-d2e99440bd99.sock
    at PipeConnectWrap.afterConnect [as oncomplete] (node:net:1605:16)

strange thing is the version at /home/user/projects/btp-lcnc-capex/node_modules/@sap/cds/package.json is 7.9.4 and not 8.0.4 as reported in the log.

petermuessig commented 1 month ago

AAAHHH, it is different whether you run it with

cds build
==> fails

or

npx cds build
==> works

GRML, let's see where this comes from...

petermuessig commented 1 month ago

This is a bit weird - if the global binary is used it uses the local installed cds version and if the npx command uses the local cds binary it finds the external one.

chgeo commented 1 month ago

Can confirm this is still broken even with cds-dk 8.0.3, but only in BAS.

This is because BAS uses PNPM to install cds-dk, which sets NODE_PATH in the binstubs:

in /managed-content/globals/pnpm/bin/cds:

...
if [ -z "$NODE_PATH" ]; then
  export NODE_PATH="/managed-content/globals/pnpm/5/.pnpm/@sap+cds-dk@8.0.3/node_modules/@sap/cds-dk/bin/node_modules:/managed-content/globals/pnpm/5/.pnpm/@sap+cds-dk@8.0.3/node_modules/@sap/cds-dk/node_modules:/managed-content/globals/pnpm/5/.pnpm/@sap+cds-dk@8.0.3/node_modules/@sap/node_modules:/managed-content/globals/pnpm/5/.pnpm/@sap+cds-dk@8.0.3/node_modules:/managed-content/globals/pnpm/5/.pnpm/node_modules"
else
  export NODE_PATH="/managed-content/globals/pnpm/5/.pnpm/@sap+cds-dk@8.0.3/node_modules/@sap/cds-dk/bin/node_modules:/managed-content/globals/pnpm/5/.pnpm/@sap+cds-dk@8.0.3/node_modules/@sap/cds-dk/node_modules:/managed-content/globals/pnpm/5/.pnpm/@sap+cds-dk@8.0.3/node_modules/@sap/node_modules:/managed-content/globals/pnpm/5/.pnpm/@sap+cds-dk@8.0.3/node_modules:/managed-content/globals/pnpm/5/.pnpm/node_modules:$NODE_PATH"
fi
...
exec node  "$basedir/../5/node_modules/@sap/cds-dk/bin/cds.js" "$@"

NODE_PATH in turn adds the global @sap/cds installation to the resolution path which finally leads to the warning and the server inconsistency.

Workarounds

npx cds watch (if @sap-cds-dk is installed as dev dependency)
npx -y -p @sap/cds-dk cds watch
npx cds serve

Basically anything that does not execute /managed-content/globals/pnpm/bin/cds

Way forward

Talk to the BAS colleagues to remove NODE_PATH from the binstub. @idoprz will ping you

Hope this helps.

@DanSchlachter FYI

petermuessig commented 1 month ago

I found a solution to detect the properly used @sap/cds-dk version - with this I now have all in my hand to control the behavior of the build plugin.

See #1047

petermuessig commented 1 month ago

The version 0.9.6 is available soon.

@sammok83 @devinea - please retry

sammok83 commented 1 month ago

@petermuessig Tried cds w --profile hybrid and got this error:

petermuessig commented 1 month ago

Ok - this should be the last thing to fix... THX for being my CAP testers - and sorry for all that trouble... 😉

petermuessig commented 1 month ago

@sammok83 I'm releasing 0.9.7 now - this should hopefully do the final trick now. Please let me know whether it now fully works. Learned a lot this time how to identify the running binary and how to resolve the version of it - since it is hard to identify the version of the @sap/cds-dk which is used to run the plugin.

petermuessig commented 1 month ago

@sammok83 @devinea - so, the latest version is available and should be bullet-proof: cds-plugin-ui5@0.9.7 is the one to test... 🤞

devinea commented 1 month ago

@petermuessig Thanks for all the hard work on this one.

Still not working unfortunately.

user: btp-lcnc-capex $ npm run watch-project1

> capex@1.0.0 watch-project1
> cds watch --open project1/index.html?sap-ui-xx-viewCache=false --livereload false

cds serve all --with-mocks --in-memory? 

        ___________________________

[cds-plugin-ui5] [info] Running cds-plugin-ui5@0.9.7 (@sap/cds-dk@8.0.3, @sap/cds@8.0.4)
[cds-plugin-ui5] [warn]   !! Multiple versions of @sap/cds loaded !!
[cds-plugin-ui5] [warn]     => /home/user/projects/btp-lcnc-capex/node_modules/@sap/cds/lib/index.js
[cds-plugin-ui5] [warn]     => /managed-content/globals/pnpm/5/.pnpm/@sap+cds@8.0.4_express@4.19.2/node_modules/@sap/cds/lib/index.js
[cds] - loaded model from 5 file(s):

  app/services.cds
  app/project1/annotations.cds
  srv/service.cds
  db/schema.cds
  node_modules/@sap/cds/common.cds

[cds] - connect using bindings from: { registry: '~/.cds-services.json' }
[cds] - connect to db > sqlite { url: ':memory:' }
  > init from db/data/sap.ui.lcnc-CapexType.csv 
  > init from db/data/sap.ui.lcnc-Capex.csv 
  > init from db/data/sap.ui.lcnc-BusinessUnits.csv 
  > init from db/data/sap.common-Currencies_texts.csv 
  > init from db/data/sap.common-Currencies.csv 
  > init from db/data/sap.common-Countries.csv 
/> successfully deployed to in-memory database. 

[cds] - using auth strategy { kind: 'dummy', impl: 'node_modules/@sap/cds/lib/auth/dummy-auth' } 

[cds] - serving MainOnlineService { path: '/serviceonline' }

-----------------------------------------------------------------------
WARNING: Package '@sap/cds' was loaded from different installations: [
  '/home/user/projects/btp-lcnc-capex/node_modules/@sap/cds/lib/index.js',
  '/managed-content/globals/pnpm/5/.pnpm/@sap+cds@8.0.4_express@4.19.2/node_modules/@sap/cds/lib/index.js'
] Rather ensure a single install only to avoid hard-to-resolve errors.
-----------------------------------------------------------------------

[cds] - server listening on { url: 'http://localhost:4004' }
[cds] - launched at 7/23/2024, 7:51:16 AM, version: 7.9.4, in: 496.154ms
[cds] - [ terminate with ^C ]

^C

[cds] - my watch has ended.  

user: btp-lcnc-capex $

petermuessig commented 1 month ago

@devinea - but this seems to be another issue related to @sap/cds-dk (Cc: @chgeo)

It doesn't report a failure by cds-plugin-ui5 it is more stopping because of cds watch isn't coming up properly...

-----------------------------------------------------------------------
WARNING: Package '@sap/cds' was loaded from different installations: [
  '/home/user/projects/btp-lcnc-capex/node_modules/@sap/cds/lib/index.js',
  '/managed-content/globals/pnpm/5/.pnpm/@sap+cds@8.0.4_express@4.19.2/node_modules/@sap/cds/lib/index.js'
] Rather ensure a single install only to avoid hard-to-resolve errors.
-----------------------------------------------------------------------

But here @chgeo and the CAP team need to help - this is out of responsibility of the cds-plugin-ui5

chgeo commented 1 month ago

@petermuessig Have you seen my analysis above?

petermuessig commented 1 month ago

@chgeo - missed that one - good to know - so @devinea this is then on BAS side to fix

In general, the build issue should be gone now with the latest release of the cds-plugin-ui5. Closing for now...

ui5-community / ui5-ecosystem-showcase

[cds-plugin-ui5] running mbt build with @sap/cds-dk@8 gives an error: Invalid priority value for UI5Plugin #1033

Workarounds

Way forward