StatCan / kubeflow-pipelines

Machine Learning Pipelines for Kubeflow
Apache License 2.0
0 stars 1 forks source link

KFP v2.0.0-alpha.3 - Failure to build post-rebase #46

Closed Souheil-Yazji closed 2 years ago

Souheil-Yazji commented 2 years ago

Issue

Step 11/23 : RUN npm ci && npm run postinstall
 ---> Running in 9ad1d59e0e5f
npm ERR! cipm can only install packages when your package.json and package-lock.json or npm-shrinkwrap.json are in sync. Please update your lock file with `npm install` before continuing.
npm ERR! 
npm ERR! 
npm ERR! Missing: i18next@^19.9.1
npm ERR! Missing: i18next-browser-languagedetector@^6.0.1
npm ERR! Missing: i18next-http-backend@^1.0.21
npm ERR! Missing: react-i18next@^11.18.3
npm ERR! 

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2022-09-20T18_25_32_453Z-debug.log
The command '/bin/sh -c npm ci && npm run postinstall' returned a non-zero code: 1

Cause

Build fails due to what appears to be a mismatch between package.json and package-log.json. This is most-likely due to the rebase/merge modifying the package-lock.json file, which should only be modified using npm install

Full logs here: https://github.com/StatCan/kubeflow-pipelines/actions/runs/3092502404/jobs/5003841845#step:4:99

Souheil-Yazji commented 2 years ago

The package-lock.json file causing the issue is in the frontend directory. Resolution attempts (EDIT: this is not the correct approach, see comment in closed PR):

Step 11/22 : RUN npm run build
 ---> Running in 722a620a9744

> pipelines-frontend@0.1.0 prebuild /src
> npm run build:tailwind

> pipelines-frontend@0.1.0 build:tailwind /src
> npx tailwindcss build -i src/tailwind.css -o src/build/tailwind.output.css

Done in 817ms.

> pipelines-frontend@0.1.0 build /src
> npm run lint && EXTEND_ESLINT=true react-scripts build

> pipelines-frontend@0.1.0 lint /src
> eslint --ext js,ts,tsx src

/src/src/pages/Compare.tsx
  28:10  warning  'TFunction' is defined but never used       @typescript-eslint/no-unused-vars
  29:10  warning  'useTranslation' is defined but never used  @typescript-eslint/no-unused-vars

/src/src/pages/NewRun.tsx
  101:11  warning  'NewRunProps' is defined but never used  @typescript-eslint/no-unused-vars

✖ 3 problems (0 errors, 3 warnings)

Creating an optimized production build...
(node:95) [DEP_WEBPACK_COMPILATION_NORMAL_MODULE_LOADER_HOOK] DeprecationWarning: Compilation.hooks.normalModuleLoader was moved to NormalModule.getCompilationHooks(compilation).loader
(Use `node --trace-deprecation ...` to show where the warning was created)
Failed to compile.

Attempted import error: 'GettingStarted' is not exported from '../pages/GettingStarted' (imported as 'GettingStarted').

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! pipelines-frontend@0.1.0 build: `npm run lint && EXTEND_ESLINT=true react-scripts build`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the pipelines-frontend@0.1.0 build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2022-09-26T12_01_49_193Z-debug.log
The command '/bin/sh -c npm run build' returned a non-zero code: 1

Attempted import error: 'GettingStarted' is not exported from '../pages/GettingStarted' (imported as 'GettingStarted').

npm ERR! code ELIFECYCLE npm ERR! errno 1 npm ERR! pipelines-frontend@0.1.0 build: npm run lint && EXTEND_ESLINT=true react-scripts build npm ERR! Exit status 1 npm ERR! npm ERR! Failed at the pipelines-frontend@0.1.0 build script. npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in: npm ERR! /root/.npm/_logs/2022-09-26T12_20_31_820Z-debug.log The command '/bin/sh -c npm run build' returned a non-zero code: 1


As a sanity check, building the upstream repo passes this point but fails on step 13:

Step 13/22 : RUN ./scripts/yarn-licenses.sh ---> Running in 8d89242fc6a6 /bin/sh: 1: ./scripts/yarn-licenses.sh: not found The command '/bin/sh -c ./scripts/yarn-licenses.sh' returned a non-zero code: 127



https://github.com/StatCan/kubeflow-pipelines/pull/47
Souheil-Yazji commented 2 years ago

Above issue resolved by changing the import of GettingStarted in the router.js file. A new issue is encountered after:

Failed to compile.                                                                                                                                                                    [47/190]

TS2339: Property 'isPlaceholder' does not exist on type '{ x: number; y: number; width: number; height: number; class?: string | undefined; label?: string | undefined; padding?: number | und
efined; paddingX?: number | undefined; paddingY?: number | undefined; rx?: number | undefined; ry?: number | undefined; shape?: string | undefined; }'.                                       
    212 |             // Placeholder nodes never need adjustment because they always have only a single                                                                                       
    213 |             // incoming edge.                                                                                                                                                       
  > 214 |             if (!destinationNode.isPlaceholder) {                                                                                                                                   
        |                                  ^^^^^^^^^^^^^                                                                                                                                      
    215 |               // Set the edge's final segment to terminate at the top or bottom of the destination                                                                                  
    216 |               // node.                                                                                                                                                              
    217 |               yEnd = downwardPointingSegment                                                                                                                                        

npm ERR! code ELIFECYCLE                                                                                                                                                                      
npm ERR! errno 1                                                                                                                                                                              
npm ERR! pipelines-frontend@0.1.0 build: `npm run lint && EXTEND_ESLINT=true react-scripts build`                                                                                             
npm ERR! Exit status 1                                                                                                                                                                        
npm ERR!                                                                                                                                                                                      
npm ERR! Failed at the pipelines-frontend@0.1.0 build script.                                                                                                                                 
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.                                                                                            

npm ERR! A complete log of this run can be found in:                                                                                                                                          
npm ERR!     /root/.npm/_logs/2022-09-26T13_17_10_304Z-debug.log                                                                                                                              
The command '/bin/sh -c npm run build' returned a non-zero code: 1

This issue is a subset of issues caused by mismatched package versions.

Souheil-Yazji commented 2 years ago

Steps to successfully locally build Master:

  1. pull kubeflow/pipelines master
  2. cd frontend
  3. nvm install 14
  4. npm install eslinter --save-dev
  5. npm run build
  6. npm start

As a last resort, force-push master to fork then re-apply i18n

Souheil-Yazji commented 2 years ago

Resolved build issues by using the upstream node package/package lock files then adding back in the required dependencies for i18n using the following command: npm install --save <package>. Changes should not be made to the package.json file manually since npm ci is used for node package installation, which uses the tree structure described in the package-lock.json file.

Next steps:

Once the application was building, previously resolved CVEs were re-introduced so those must be resolved. A possibility would be to cherrypick the previous cve resolution commits.

Changing this issue to an 8 pointer due to unexpected complications.

Souheil-Yazji commented 2 years ago

Mitigated the following CVEs via package upgrade:

 Scanning for vulnerabilties in image: k8scc01covidacr.azurecr.io/kubeflow-pipeline/kubeflow-pipeline:2d1b5313
╔══════════════════════╤═════════════════╤═════════════════╤════════════════════════════════════════════════════╤══════════════════════╗
║ VULNERABILITY ID     │ PACKAGE NAME    │ SEVERITY        │ DESCRIPTION                                        │ TARGET               ║
╟──────────────────────┼─────────────────┼─────────────────┼────────────────────────────────────────────────────┼──────────────────────╢
║ CVE-2022-37434       │ zlib            │ CRITICAL        │ zlib through 1.2.12 has a heap-based buffer        │ k8scc01covidacr.     ║
║                      │                 │                 │ over-read or buffer overflow in inflate in         │ azurecr.io/kubeflow- ║
║                      │                 │                 │ inflate.c via a large gzip header extra field.     │ pipeline/kubeflow-   ║
║                      │                 │                 │ NOTE: only applications that call inflateGetHeader │ pipeline:2d1b5313    ║
║                      │                 │                 │ are affected. Some common applications bundle the  │ (alpine 3.15.0)      ║
║                      │                 │                 │ affected zlib source code but may be unable to     │                      ║
║                      │                 │                 │ call inflateGetHeader (e.g., see the nodejs/node   │                      ║
║                      │                 │                 │ reference).                                        │                      ║
╟──────────────────────┼─────────────────┼─────────────────┼────────────────────────────────────────────────────┼──────────────────────╢
║ CVE-2021-3918        │ json-schema     │ CRITICAL        │ json-schema is vulnerable to Improperly Controlled │ Node.js              ║
║                      │                 │                 │ Modification of Object Prototype Attributes        │                      ║
║                      │                 │                 │ ('Prototype Pollution')                            │                      ║
╟──────────────────────┼─────────────────┼─────────────────┼────────────────────────────────────────────────────┼──────────────────────╢
║ CVE-2021-44906       │ minimist        │ CRITICAL        │ Minimist <=1.2.5 is vulnerable to Prototype        │ Node.js              ║
║                      │                 │                 │ Pollution via file index.js, function setKey()     │                      ║
║                      │                 │                 │ (lines 69-95).                                     │                      ║
╟──────────────────────┼─────────────────┼─────────────────┼────────────────────────────────────────────────────┼──────────────────────╢
║ CVE-2021-23358       │ underscore      │ CRITICAL        │ The package underscore from 1.13.0-0 and before    │ Node.js              ║
║                      │                 │                 │ 1.13.0-2, from 1.3.2 and before 1.12.1 are         │                      ║
║                      │                 │                 │ vulnerable to Arbitrary Code Injection via the     │                      ║
║                      │                 │                 │ template function, particularly when a variable    │                      ║
║                      │                 │                 │ property is passed as an argument as it is not     │                      ║
║                      │                 │                 │ sanitized.                                         │                      ║
╟──────────────────────┼─────────────────┼─────────────────┼────────────────────────────────────────────────────┼──────────────────────╢
║ CVE-2021-23358       │ underscore      │ CRITICAL        │ The package underscore from 1.13.0-0 and before    │ Node.js              ║
║                      │                 │                 │ 1.13.0-2, from 1.3.2 and before 1.12.1 are         │                      ║
║                      │                 │                 │ vulnerable to Arbitrary Code Injection via the     │                      ║
║                      │                 │                 │ template function, particularly when a variable    │                      ║
║                      │                 │                 │ property is passed as an argument as it is not     │                      ║
║                      │                 │                 │ sanitized.                                         │                      ║
╚══════════════════════╧═════════════════╧═════════════════╧════════════════════════════════════════════════════╧══════════════════════╝