default tekton pipeline for appsody stack should include test task

@csantanapr commented on Fri Oct 11 2019

Is your feature request related to a problem? Please describe. Problem is that I think the default pipeline which includes 2 tasks build and deploy should have a 3rd task that runs the same process as appsody test

Describe the solution you'd like

The task test-task should be first in the pipeline.
This task should use the container appsody stack (ie Dockerfile-stack), this is specified in the .appsody-config.yaml from the git source For example nodejs:
```
cat .appsody-config.yaml
stack: kabanero/nodejs-express:0.2
```
The working directory should be /project
The command should similar to $APPSODY_INSTALL and $APPSODY_TEST

Describe alternatives you've considered I have something working for nodejs-express but I hit a few problems.

apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: csantana-nodejs-express-build-task
spec:
  inputs:
    resources:
      - name: git-source
        type: git
  steps:
    - name: npm-test
      image: kabanero/nodejs-express:0.2
      workingDir: /project
      command:
        - /bin/bash
      args:
        - -c
        - |
          set -e
          cp -a ${inputs.resources.git-source.path}/* $APPSODY_WATCH_DIR
          # perform npm install
          $APPSODY_INSTALL
          # run npm test on project and user-app
          delimiter=" && "
          s=$APPSODY_TEST$delimiter
          array=();
          while [[ $s ]]; do
              array+=( "${s%%"$delimiter"*}" );
              s=${s#*"$delimiter"};
          done;
          for i in "${array[@]}"
          do
            echo executing $i
            $i
          done

Some of the problems I hit: . hardcoding the image name kabanero/nodejs-express:0.2 this should be from the config file .appsody-config.yaml

. something weird about running $APPSODY_TEST with the value is npm test && npm test -- prefix user-app mocha tries to parse && as argument, so the workaround I did was to parse the variable and execute it

[nodejs-express-test-build-task : npm-test] > nodejs-express-simple@0.1.0 test /project/user-app
[nodejs-express-test-build-task : npm-test] > mocha "&&" "npm" "test"
[nodejs-express-test-build-task : npm-test] 
[nodejs-express-test-build-task : npm-test] Warning: Cannot find any files matching pattern "&&"
[nodejs-express-test-build-task : npm-test] Warning: Cannot find any files matching pattern "npm"

. I really wanted to avoid to run appsody extract, this takes too much time when using a volume, so I tried to do a softlink to avoid the cp -a, with ln -s /workspace/git-source /project/user-app the files get map correctly, the problem is with node.js when it tries to resolve node dependencies the cd .. get's resolve into /workspace instead of /project I guess we can add the appsody extract at this stage, and remove it from the build-task

I'm going to look into spring-boot and microprofile appsody stacks to implement the unit test tasks.

Additional context Tekton should have a way to dynamically specify the image name for step in a task some how, this way we don't need to hardcode kabanero/nodejs-express:0.2

cc @seabaylea

@csantanapr commented on Fri Oct 11 2019

In case is useful to anyone doing tekton task development and debugging, I'm using the tekton tkn CLI to simulate the same pipelinerun that get's trigger thru git webhook

tkn pipeline start nodejs-express-test-build-deploy-pipeline \                         
        -r git-source=csantana-nodejs-git \
        -r docker-image=csantana-nodejs-image \
        -p image-tag=master \
        -p image-name=docker-registry.default.svc:5000/kabanero-samples/appsody-sample-nodejs-express \
        -p release-name=appsody-sample-nodejs-express \
        -p repository-name=appsody-sample-nodejs-express \
        -p target-namespace=kabanero \
        -p docker-registry=docker-registry.default.svc:5000/kabanero-samples \
        -s kabanero-operator \
        -n kabanero

resources:

---
apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: csantana-nodejs-git
spec:
  type: git
  params:
    - name: url
      value: https://github.com/csantanapr/appsody-sample-nodejs-express
---
apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: csantana-nodejs-image
spec:
  type: image
  params:
    - name: url
      value: docker-registry.default.svc:5000/csantana/appsody-sample-nodejs-express:tkn

@csantanapr commented on Fri Oct 11 2019

Here is the log output, as you can see if running the mocha tests from the /project directory and second run on the /project/user-app

tkn pipelinerun logs nodejs-express-test-build-deploy-pipeline-run-gd2bq -f -n kabanero

output

[test-task : git-source-csantana-nodejs-git-rtsn5] {"level":"warn","ts":1570849767.937413,"logger":"fallback-logger","caller":"logging/config.go:69","msg":"Fetch GitHub commit ID from kodata failed: \"KO_DATA_PATH\" does not exist or is empty"}
[test-task : git-source-csantana-nodejs-git-rtsn5] {"level":"info","ts":1570849768.3113258,"logger":"fallback-logger","caller":"git/git.go:102","msg":"Successfully cloned https://github.com/csantanapr/appsody-sample-nodejs-express @ master in path /workspace/git-source"}

[test-task : npm-test] added 170 packages from 578 contributors and audited 295 packages in 6.59s
[test-task : npm-test] found 0 vulnerabilities
[test-task : npm-test] 
[test-task : npm-test] 
[test-task : npm-test] > nodejs-express@0.2.6 test /project
[test-task : npm-test] > mocha
[test-task : npm-test] 
[test-task : npm-test] [Sat Oct 12 03:09:38 2019] com.ibm.diagnostics.healthcenter.loader INFO: Node Application Metrics 5.0.5.201909191743 (Agent Core 4.0.5)
[test-task : npm-test] [2019-10-12 03:09:38.674] [INFO] k8sutil.js - k8sutil K8sutil() Current namespace kabanero
[test-task : npm-test] [Sat Oct 12 03:09:39 2019] com.ibm.diagnostics.healthcenter.mqtt INFO: Connecting to broker localhost:1883
[test-task : npm-test] 
[test-task : npm-test] 
[test-task : npm-test] App started on PORT 3000
[test-task : npm-test]   Node.js Express stack
[test-task : npm-test]     /metrics endpoint
[test-task : npm-test]       ✓ status (42ms)
[test-task : npm-test]       ✓ contains os_cpu_used_ratio
[test-task : npm-test] [2019-10-12 03:09:39.307] [ERROR] k8sutil.js - k8sutil K8sutil() pods "nodejs-express-test-build-deploy-pipeline-run-gd2bq-test--4njjn" not found
[test-task : npm-test]       ✓ contains process_cpu_used_ratio
[test-task : npm-test]     /ready endpoint
[test-task : npm-test]       ✓ status
[test-task : npm-test]     /live endpoint
[test-task : npm-test]       ✓ status
[test-task : npm-test]     /health endpoint
[test-task : npm-test]       ✓ status
[test-task : npm-test]     /blah endpoint
[test-task : npm-test]       ✓ status
[test-task : npm-test] 
[test-task : npm-test] 
[test-task : npm-test]   7 passing (178ms)
[test-task : npm-test] 
[test-task : npm-test] 
[test-task : npm-test] > nodejs-express-simple@0.1.0 test /project/user-app
[test-task : npm-test] > mocha
[test-task : npm-test] 
[test-task : npm-test] [Sat Oct 12 03:09:40 2019] com.ibm.diagnostics.healthcenter.loader INFO: Node Application Metrics 5.0.5.201909191743 (Agent Core 4.0.5)
[test-task : npm-test] [2019-10-12 03:09:41.503] [INFO] k8sutil.js - k8sutil K8sutil() Current namespace kabanero
[test-task : npm-test] [Sat Oct 12 03:09:41 2019] com.ibm.diagnostics.healthcenter.mqtt INFO: Connecting to broker localhost:1883
[test-task : npm-test] 
[test-task : npm-test] 
[test-task : npm-test] App started on PORT 3000
[test-task : npm-test]   Node.js Express Simple template
[test-task : npm-test]     / endpoint
[test-task : npm-test]       ✓ status (105ms)
[test-task : npm-test] [2019-10-12 03:09:41.961] [ERROR] k8sutil.js - k8sutil K8sutil() pods "nodejs-express-test-build-deploy-pipeline-run-gd2bq-test--4njjn" not found
[test-task : npm-test] 
[test-task : npm-test] 
[test-task : npm-test]   1 passing (177ms)
[test-task : npm-test]

@csantanapr commented on Fri Oct 11 2019

Here how they look in tekton dashboard of ICP4Apps v3

One weird thing, they run in the correct order test(1), build(2), deploy(3) but the rows in the UI doesn't show them like this when the pipelinerun is completed.

@csantanapr commented on Fri Oct 11 2019

Here is how my test pipeline resource looks like

apiVersion: tekton.dev/v1alpha1
kind: Pipeline
metadata:
  name: nodejs-express-test-build-deploy-pipeline
spec:
  resources:
    - name: git-source
      type: git
    - name: docker-image
      type: image
  tasks:
    - name: test-task
      taskRef:
        name: csantana-nodejs-express-build-task
      resources:
        inputs:
          - name: git-source
            resource: git-source
    - name: build-task
      taskRef:
        name: nodejs-express-build-task
      runAfter: [test-task]
      resources:
        inputs:
          - name: git-source
            resource: git-source
        outputs:
          - name: docker-image
            resource: docker-image
    - name: deploy-task
      taskRef:
        name: nodejs-express-deploy-task
      runAfter: [build-task]
      resources:
        inputs:
          - name: git-source
            resource: git-source
          - name: docker-image
            resource: docker-image

kabanero-io / kabanero-pipelines

default tekton pipeline for appsody stack should include test task #87