search

forcedotcom / sfdx-scanner

MIT License
214 stars 49 forks source link

[BUG] Error installing SFDX Scanner on Github actions pipeline. #1522

Closed rosangelys-pfm closed 3 months ago

rosangelys-pfm commented 3 months ago

Have you tried to resolve this issue yourself first?

Yes

Bug Description

I have a CI/CD job running with github actions where it used to work fine but I am now getting the following error when trying to install the SFDX Scanner for code scanning:

Installing plugin @salesforce/sfdx-scanner... ⣻ [2/4] Fetching packages... error @typescript-eslint/parser@7.14.1: The engine "node" is incompatible with this module. Expected version "^18.18.0 || >=20.0.0". Got "18.15.0" error Found incompatible module. Installing plugin @salesforce/sfdx-scanner... ⢿ info Visit https://yarnpkg.com/en/docs/cli/add for documentation about this command. Installing plugin @salesforce/sfdx-scanner... failed Error: yarn add @salesforce/sfdx-scanner@latest --non-interactive --mutex=file:/home/runner/.local/share/sfdx/yarn.lock --preferred-cache-folder=/home/runner/.cache/sfdx/yarn --check-files exited with code 1 Error: Process completed with exit code 1.

I installed the node version in my github pipeline as follows:

Install nodejs in the VM, and specify version

        - name: 'Install NodeJS'
          uses: actions/setup-node@v4
          with:
              node-version: '18'

This is the installing version: 18.20.3

Output / Logs

Node version:

Node V

Error:

Error sfdx

Steps To Reproduce

Trigger Github workflow Installation of node step is done SFDX installation is made

Expected Behavior

The installations are made successfully and code is scanned.

Operating System

Github Workflow VM: Ubuntu 22.04.4 LTS

Salesforce CLI Version

sfdx-cli/7.209.6 linux-x64 node-v18.15.0

Code Analyzer Plugin (@salesforce/sfdx-scanner) Version

@salesforce/sfdx-scanner (latest)

Additional Context (Screenshots, Files, etc)

No response

Workaround

No response

Urgency

High

sandeepkumargetweave commented 3 months ago

We're also facing this issue since yesterday.

Bheru-gadri commented 3 months ago

It's fixed and working fine for me. I just added a new step after the 'Install Node JS' step.

 - name: 'Ignore engines from yarn'
        run: |
            echo "Ignore engines"
            yarn config set ignore-engines true
Sachhiin commented 3 months ago

Hi @Bheru-gadri, I'm facing the same issue. Can you provide the entire YML file for reference?

Thanks

Sachhiin commented 3 months ago

I'm using bitbucket pipeline for deployment and facing this issue

lucas-tesan commented 3 months ago

Thanks for reporting, we are facing the same issue

rosangelys-pfm commented 3 months ago

For me, the problem was the node version that the SFDX CLI installation was installing too as a bundle. Fixed it by doing the following:

stephen-carter-at-sf commented 3 months ago

We just updated @salesforce/sfdx-scanner@latest to version 4.0 which contains a number of updates including an upgrade of some eslint libraries which require your node version to either be 18.18.0+ or 20.0.0+. Additionally we now require the JAVA version to 11 or greater.

Note that if you have the latest version of the sf cli then you should have node version 20+ already available. Even though you updated your node environment, your old sf cli version of sfdx-cli/7.209.6 linux-x64 node-v18.15.0 shows that it still uses node v18.15.0.

So you have a few options:

May I also recommend that you use the Run Code Analyzer GitHub action that we offer.

For more information, see this month's release notes: https://developer.salesforce.com/docs/platform/salesforce-code-analyzer/guide/release-notes.html

sandeepkumargetweave commented 3 months ago

@rosangelys-pfm it has fixed the issue on our side as well. Thanks!

johnbelosf commented 3 months ago

Please see @stephen-carter-at-sf 's comment above to resolve this issue. Closing for now.

johnbelosf commented 3 months ago

Reopening for wider awareness for the time being. Will close in a few days.

derrick-ejan-rs commented 3 months ago

For me, the problem was the node version that the SFDX CLI installation was installing too as a bundle. Fixed it by doing the following:

  • Updating node version being installed at the beginning of YML:
# Install nodejs in the VM, and specify version 20
            - name: 'Install NodeJS'
              uses: actions/setup-node@v4
              with:
                  node-version: '20'
  • Installing the latest version of SF CLI (Previously using installation with URL to tar.xz file) PREVIOUS:
# Install Salesforce CLI
            - name: 'Install Salesforce CLI'
              run: |
                  wget https://developer.salesforce.com/media/salesforce-cli/sfdx/channels/stable/sfdx-linux-x64.tar.xz
                  mkdir ~/sfdx
                  tar xJf sfdx-linux-x64.tar.xz -C ~/sfdx --strip-components 1
                  echo "$HOME/sfdx/bin" >> $GITHUB_PATH
                  ~/sfdx/bin/sfdx version

CURRENT (fixed the problem for me):

# Install Salesforce CLI
            - name: 'Install Salesforce CLI'
              run: |
                  npm install sfdx-cli --global

thanks this was my same issue here.

Sachhiin commented 3 months ago

Hi @stephen-carter-at-sf I'm still facing issue/error for bitbucket pipeline. I have tried with node - 20 bullseye, 22-bullseye, 22.3.0

It worked with below command: sf plugins install @salesforce/sfdx-scanner@3.26.0 and node:current-bullseye

But why it is not a long term solution?

Also, Can you suggest any workaround?

stephen-carter-at-sf commented 3 months ago

@Sachhiin what do you get for sf --version?

If you are using an old version of sf then regardless of what node you install in your environment, the sf cli will use its own version of node that it has bundled. So you most likely just need to upgrade sf.

Using v3.26.0 of the sfdx-scanner plugin long term isn't recommended because it locks you into the old v3.x which we no longer are developing nor patching. We are now in v4.x which is what we will be maintaining each month (while we develop the next generation Salesforce Code Analyzer).

And what exactly is the error that you are seeing?

sandeepkumargetweave commented 3 months ago

@stephen-carter-at-sf we tried using Salesforce Code Analyser but seeing the below error while executing the workflow, looks like we are missing some configuration to store the output file.

image

sandeepkumargetweave commented 3 months ago

rosangelys-pfm The issue is not completely resolved for us. It worked intermittently but now it is not working at all. Could you share your complete YAML file w.r.t SFDC Scanner ?

stephen-carter-at-sf commented 3 months ago

sandeepkumargetweave Can you share your github workflow file and the entire log output of your github workflow job? Is it a public github repo that you can just point to?

And is this one of your workflow steps?

    - name: Install Salesforce CLI
      run: npm install -g @salesforce/cli@latest
rosangelys-pfm commented 3 months ago

rosangelys-pfm The issue is not completely resolved for us. It worked intermittently but now it is not working at all. Could you share your complete YAML file w.r.t SFDC Scanner ? @sandeepkumargetweave This runs when our GitHub Pull Requests are opened

# Jobs to be executed when the above conditions are met
jobs:
    validate-deployment-on-UAT-org:
        env:
            SF_SCANNER_OUTFILE: 'scanner-output/scannerResult.xml'
            SF_SCANNER_FORMAT: 'csv'
            SF_SCANNER_SEVERITY: 3
            STG_ORG_ALIAS: UAT-STAGING
            AB_ORG_ALIAS: AB-SDBX-STAGING
            SFDX_HIDE_RELEASE_NOTES_FOOTER: true
            SFDX_HIDE_RELEASE_NOTES: true
        runs-on: ubuntu-latest
        steps:
            # Install nodejs in the VM, and specify version 20
            - name: 'Install NodeJS'
              uses: actions/setup-node@v4
              with:
                  node-version: '20'

            # Checkout the source code
            # The fetch-depth:0  allows us to access previous commits that have been pushed to the repository.

            # We'll need this later when we try to figure out which metadata has
            # changed between commits, so that we can only deploy that metadata
            # to the destination org
            - name: 'Checkout source code'
              uses: actions/checkout@v3
              with:
                  fetch-depth: 0

            # Install Salesforce CLI
            - name: 'Install Salesforce CLI'
              run: |
                  npm install sfdx-cli --global

            # Install the SFDX-Git-Delta plugin - https://github.com/scolladon/sfdx-git-delta
            # This plugin allows us to extract a package.xml with the metadata that has changed between commits.
            - name: 'Installing sfdx git delta'
              run: |
                  echo y | sfdx plugins:install sfdx-git-delta
                  sfdx plugins

            # Install java as it is required for the next step
            - name: 'Installing java'
              run: |
                  sudo apt-get update
                  sudo apt install default-jdk

            # Install SFDX scanner
            - name: 'Installing SFDX scanner'
              run: sfdx plugins:install @salesforce/sfdx-scanner

            # https://developer.salesforce.com/docs/atlas.en-us.sfdx_cli_reference.meta/sfdx_cli_reference/cli_reference_auth_sfdxurl.htm
            # This URL can then be used with the sfdx auth:sfdxurl:store to
            # authenticate the sfdx project in the repositry, against the UAT Org.

            # The UAT Org auth URL is stored in the Github Secret named SFDX_STAGING_URL
            # We will store the URL into a text file
            - name: 'Populate auth file with SFDX_URL secret of UAT org'
              shell: bash
              run: |
                  echo ${{ secrets.SFDX_STAGING_URL}} > ./SFDX_STAGING_URL.txt
                  secretFileSize=$(wc -c "./SFDX_STAGING_URL.txt" | awk '{print $1}')
                  if [ $secretFileSize == 1 ]; then
                      echo "Missing SFDX_STAGING_URL secret. Is this workflow running on a fork?";
                      exit 1;
                  fi

            # Authenticate to org using the URL stored in the text file
            - name: 'Authenticate to UAT Org (Staging)'
              run: sfdx auth:sfdxurl:store -f ./SFDX_STAGING_URL.txt -s -a ${{ env.STG_ORG_ALIAS }}

            # We use SFDX Git Delta to create a directory with only the metadata that has changed.
            # this allows us to deploy only those changes, as opposed to deploying the entire branch.
            # This helps reducing deployment times.
            - name: 'Create delta packages for new, modified or deleted metadata'
              run: |
                  mkdir changed-sources
                  sfdx sgd:source:delta --to "HEAD" --from "HEAD^" --output changed-sources/ --generate-delta --source force-app/ --ignore .sgdignore
                  cat ./changed-sources/**/package.xml
                  cat ./changed-sources/**/destructiveChanges.xml

            # Now we can use the sfdx scanner to scan the code in the delta directory
            # The output of the scan is stored in a file called apexScanResults.csv

            # The .xml file can later be uploaded to github, so that we can see the
            # results of the scan directly from the PR.
            - name: 'Scan code'
              id: scan-results
              run: |
                  cd changed-sources
                  sfdx scanner:run --format csv --target './**/*.cls' --severity-threshold 3 --outfile 'apexScanResults.csv'  
                  cd ..

            # Read PMD csv file results
            - name: 'Read Results CSV'
              if: failure() && steps.scan-results.outcome == 'failure'
              id: csv
              uses: juliangruber/read-file-action@v1
              with:
                  path: ./changed-sources/apexScanResults.csv

            ## Create table markdown from csv results
            - name: 'Create Markdown table'
              if: failure() && steps.csv.outcome == 'success' && steps.scan-results.outcome == 'failure'
              uses: petems/csv-to-md-table-action@master
              id: csv-table-output
              with:
                  csvinput: ${{ steps.csv.outputs.content }}

            # Upload the .csv file as a table to PR comments
            - name: 'Upload PMD Results on PR'
              if: failure() && steps.csv-table-output.outcome == 'success' && steps.scan-results.outcome == 'failure'
              uses: mshick/add-pr-comment@v1
              with:
                  message: |
                      ${{steps.csv-table-output.outputs.markdown-table}}
                  repo-token: ${{ secrets.GITHUB_TOKEN }}
                  allow-repeats: true

            # Check-only deployment and run all tests
            - name: 'Check-Only deploy delta changes - run all tests'
              run: |
                  sfdx force:source:deploy --manifest "changed-sources/package/package.xml" --postdestructivechanges "changed-sources/destructiveChanges/destructiveChanges.xml" --checkonly --testlevel RunLocalTests  --json --ignorewarnings
rosangelys-pfm commented 3 months ago

@sandeepkumargetweave we tried using Salesforce Code Analyser but seeing the below error while executing the workflow, looks like we are missing some configuration to store the output file.

image

That is a different issue and has to do with Code analyzer features being turned on in the GitHub repository you are working on

stephen-carter-at-sf commented 3 months ago

Is there a reason why you are using the old sfdx instead of sf? npm install sfdx-cli --global is what I see in your workflow when you should be doing npm install -g @salesforce/cli@latest

rosangelys-pfm commented 3 months ago

Is there a reason why you are using the old sfdx instead of sf? npm install sfdx-cli --global is what I see in your workflow when you should be doing npm install -g @salesforce/cli@latest

@stephen-carter-at-sf

No specific reason, this workflow has been active for quite a bit already so I just need to gather some time to go over the file and update versions and commands in all of our workflows.

My workflow is working fine now. I was posting my YML based on @sandeepkumargetweave questions

Sachhiin commented 3 months ago

Hey, @stephen-carter-at-sf

I'm using bitbucket pipelines for CI CD and attaching the YML that we are currently using.

In this we have made 2 updates to resolve the issue

  1. The node version was 16-bullseye which we updated to node:current-bullseye

  2. Made updates in sfdx scanner command : a. Earlier line --> sfdx plugins:install @salesforce/sfdx-scanner b. New line --> sfdx plugins:install @salesforce/sfdx-scanner@3.26.0

    1. As suggested by you, we can use 3.26.0 for temporary purpose but it's not a long term solution.

    I had tried with node:current-bullseye with sfdx plugins:install @salesforce/sfdx-scanner@4.3.0 but this combination is not working and giving the same issue. However, node:current-bullseye with sfdx plugins:install @salesforce/sfdx-scanner@3.26.0 is working fine for us.

Please find YML file in attachment: pipeline.txt

`# This is an example Starter pipeline configuration

Use a skeleton to build, test and deploy using manual and parallel steps

-----

You can specify a custom docker image from Docker Hub as your build environment.

image: node:current-bullseye definitions: steps:

stephen-carter-at-sf commented 3 months ago

@Sachhiin As mentioned before, your issue is most likely with the version of the sf or sfdx cli you are installing.

I see you have this commented out:

# npm install --global sfdx-cli@${SALESFORCE_CLI_VERSION} --ignore-scripts && npm install --global @salesforce/cli@${SF_CLI_VERSION}

But anyway, you should be using a version of sf which bundles the correct node 20+ version. So I recommended

npm install --global @salesforce/cli@latest

If you still are having issues, then do sf --version or sfdx --version to see what version of node your sf or sfdx is using. Then you can use with the latest @salesforce/sfdx-scanner plugin version of 4.3.0

Also I see you have --pmdconfig pmd-ruleset.xml in which case if you are having issues then see the comments over at: https://github.com/forcedotcom/sfdx-scanner/issues/1523

stephen-carter-at-sf commented 3 months ago

I'm going to close this issue now. If anyone else is having trouble... the remedy is to make sure you are using a later version of sf that has the required node version for our updated plugin. Or if you are using custom pmd rules with --pmdconfig pmd-ruleset.xml then you might need to migrate your rules to be compatible with PMD 7. See Migration Guide for PMD 7 for more information.