thoughtworks / talisman

Using a pre-commit hook, Talisman validates the outgoing changeset for things that look suspicious — such as tokens, passwords, and private keys.
https://thoughtworks.github.io/talisman/
MIT License
1.91k stars 242 forks source link

Integration of talisman with jenkins #370

Open mydailypuff opened 2 years ago

mydailypuff commented 2 years ago
pipeline {
  agent {
    kubernetes {
      yamlFile 'build/pod.yaml'
    }
  }
  options {
    buildDiscarder(logRotator(numToKeepStr: '6'))
  }

  stages {
    stage('talisman check') {
      steps {
        script{
          container('talisman') {
          sh '''
                export TALISMAN_HOME=/root/.talisman/bin && alias talisman=$TALISMAN_HOME/talisman_linux_amd64
                export TALISMAN_INTERACTIVE=true
                pwd
                talisman --scan
                head -10 talisman_report/talisman_reports/data/report.json'''
          }
        }
      }
    }
  }
}

This is my jenkinsfile. I was integrating talisman with jenkins. But it is not scanning, I guess it is not considering the current repo as the git repo. Below is the error log.

export TALISMAN_HOME=/root/.talisman/bin
+ alias talisman=/root/.talisman/bin/talisman_linux_amd64
+ export TALISMAN_INTERACTIVE=true
+ pwd
/home/jenkins/agent/workspace/SOCII/talisman
+ /root/.talisman/bin/talisman_linux_amd64 --scan
2022/07/01 04:43:40 exit status 128

I am able to get the talisman scan report in my local, however in jenkins it is throwing the error. Could anyone please help me with this!

svishwanath-tw commented 2 years ago

hi @mydailypuff could you execute talisman --scan --debug and share the output ?

mydailypuff commented 2 years ago
+ /home/jenkins/agent/workspace/SOCII/talisman/.talisman/bin/talisman_linux_amd64 --scan --debug
time="2022-07-05T03:57:38Z" level=debug msg="Talisman execution environment" Checksum= Debug=true GitHook=pre-push IgnoreHistory=false LogLevel=error Pattern= ReportDirectory=talisman_report Scan=true ScanWithHtml=false ShouldProfile=false
time="2022-07-05T03:57:38Z" level=info msg="Running scanner"
2022/07/05 03:57:38 exit status 128
[Pipeline] }
[Pipeline] // container

this is the log after adding debug flag
svishwanath-tw commented 2 years ago

Hi @mydailypuff. Thanks for the update. Unfortunately the new output does not throw more light on the problem at hand.

Talisman in scan mode looks for a .git folder (indirectly by executing git commands using the git binary) . It assumes that the working directory is the root of a git repository.

My questions here would be :

  1. Is /home/jenkins/agent/workspace/SOCII/talisman the base path of repo being scanned, is there a .git folder there ?
  2. Is the git executable available in the path ?
  3. Does the user executing talisman have read/write permissions on the working directory ?
mydailypuff commented 2 years ago

so how can I give permissions for talisman on the working directory? Actually in my local I did not give any permissions but it worked these are the directories. I could see that .git is present drwxr-xr-x 4 1000 1000 42 Jul 5 08:45 .. drwxr-xr-x 2 1000 1000 57 Jul 5 08:45 build -rw-r--r-- 1 1000 1000 10 Jul 5 08:45 README.md drwxr-xr-x 8 1000 1000 162 Jul 5 08:45 .git drwxr-xr-x 6 1000 1000 86 Jul 5 08:45 .

mydailypuff commented 2 years ago

given read and write permissions to .talisman and .git-template as well as all the files in the working directory; but of no use

jmatias commented 2 years ago

Can you recreate the issue in a docker container? We need to be able to reproduce it on our end.

mydailypuff commented 2 years ago

I tried with one other way, with this talisman is getting installed and scanning but not fetching any reports stage('talisman check') { steps { script{ container('builder') { sh ''' mkdir $WORKSPACE/.talisman cd $WORKSPACE/.talisman curl -L -O https://github.com/thoughtworks/talisman/releases/download/v1.3.0/talisman_linux_386 chmod +x talisman_linux_386 $WORKSPACE/.talisman/talisman_linux_386 --scan cd .. mv .talisman/talisman_reports/data/report.json . ls -ltr ''' } } } } Have not installed talisman in the container

mydailypuff commented 2 years ago

getting this in the report which is wrong, as I have secrets in my repo {"summary":{"types":{"filecontent":0,"filesize":0,"filename":0,"warnings":0,"ignores":0}},"results":[]}

svishwanath-tw commented 2 years ago

@mydailypuff : In your latest comments, release v1.3.0 and a 386 build are being used ? The first comment has talisman_linux_amd64 with no version specified. This is confusing to me. I'd recommend using the latest build (v1.28.0) for your OS_ARCH combination.