search

halkyonio / operator

Kubernetes Operator simplifying the development of microservices on k8s !
Apache License 2.0
40 stars 14 forks source link

Issue. No contextPath is defined if BuildConfig.contextPath is empty #147

Open cmoulliard opened 5 years ago

cmoulliard commented 5 years ago

Issue

No contextPath is defined within the taskRun created by the operator 

spec:
  inputs:
    params:
    - name: baseImage
      value: quay.io/halkyonio/spring-boot-maven-s2i
    - name: moduleDirName
      value: fruit-client-sb
    - name: contextPath
      value: ""

if BuildConfig.contextPath is empty

spec:
  buildConfig:
    contextPath: ""
    moduleDirName: fruit-client-sb
    ref: master
    type: s2i
    url: https://github.com/rhte-eu/cmoullia.git

This is because the existing code don't calculate it and don't even use the Task default parameters if empty

{Name: "contextPath", Value: c.Spec.BuildConfig.ContextPath},
cmoulliard commented 5 years ago

There is perhaps an issue which is not at all related as the following component, taskrun created don't allow to start the java app as code source is pushed to the deployments dir is not the uber jar file generated.

The s2i maven don't take place --> see log of s2i step but instead just copy the resources

STEP 8: RUN /usr/local/s2i/assemble
INFO S2I source build with plain binaries detected
INFO Copying binaries from /tmp/src to /deployments ...

Component

apiVersion: halkyon.io/v1beta1
kind: Component
metadata:
  name: fruit-client-lab
spec:
  deploymentMode: build
  port: 8080
  buildConfig:
    type: "s2i"
    url: https://github.com/rhte-eu/cmoullia.git
    ref: master
    moduleDirName: "fruit-client-sb"
  runtime: spring-boot
  version: 2.1.6

TaskRun created

apiVersion: tekton.dev/v1alpha1
kind: TaskRun
metadata:
  creationTimestamp: 2019-09-11T11:05:11Z
  generation: 1
  labels:
    build: fruit-client-lab
    component_cr: fruit-client-lab
    tekton.dev/task: s2i-buildah-push
  name: fruit-client-lab
  namespace: rhtelab
  ownerReferences:
  - apiVersion: halkyon.io/v1beta1
    blockOwnerDeletion: true
    controller: true
    kind: Component
    name: fruit-client-lab
    uid: 065ead81-d484-11e9-a220-06aaef5cff46
  resourceVersion: "721781"
  selfLink: /apis/tekton.dev/v1alpha1/namespaces/rhtelab/taskruns/fruit-client-lab
  uid: 0664ca8f-d484-11e9-a220-06aaef5cff46
spec:
  inputs:
    params:
    - name: baseImage
      value: quay.io/halkyonio/spring-boot-maven-s2i
    - name: moduleDirName
      value: fruit-client-sb
    - name: contextPath
      value: .
    resources:
    - name: git
      resourceRef: {}
      resourceSpec:
        params:
        - name: revision
          value: master
        - name: url
          value: https://github.com/rhte-eu/cmoullia.git
        type: git
  outputs:
    resources:
    - name: image
      resourceRef: {}
      resourceSpec:
        params:
        - name: url
          value: image-registry.openshift-image-registry.svc:5000/rhtelab/fruit-client-lab
        type: image
  serviceAccount: build-bot
  taskRef:
    kind: Task
    name: s2i-buildah-push
  timeout: 1h0m0s

Log of s2i build step

Log stream ended.
|
220 lines
STEP 1: FROM quay.io/halkyonio/spring-boot-maven-s2i
Getting image source signatures
Copying blob sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
Copying blob sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
Copying blob sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
Copying blob sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
Copying blob sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
Copying blob sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
Copying blob sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
Copying blob sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
Copying blob sha256:50408d34b7db1a7cac449215c8bf82b020a4e61bd542f66a8766b4804f3882fe
Copying blob sha256:5c2598df456066d90ae3ed81592c54247800071c7ee8845985a93db7e95e936f
Copying blob sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
Copying blob sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
Copying blob sha256:dc4e07da33bca48edefbcba8dadefa8e7ffc6fe3e8ee4db140600a62862a16ac
Copying blob sha256:6e99ee96b35e4771c55d82d9f0b7abb9ebe519932e7ad6832ee5f868afae5073
Copying blob sha256:7d663282d8c2165a918ddad80f047d3fe48547f91857dfd8a80f90c7e96a4f0c
Copying blob sha256:a9ce2e2d1ec5c47edf0f0ac1466c63bb1cb78cef2a94151face1ed124754fc7c
Writing manifest to image destination
Storing signatures
STEP 2: LABEL "io.openshift.s2i.build.image"="quay.io/halkyonio/spring-boot-maven-s2i"       "io.openshift.s2i.build.source-location"="/workspace/git/."
19c2347a2eecadb31aae1241cc708640ddf280071dcfd0560af486bbfe785e6e
STEP 3: ENV MAVEN_ARGS_APPEND="-pl fruit-client-sb"     MAVEN_S2I_ARTIFACT_DIRS="fruit-client-sb/target"     S2I_SOURCE_DEPLOYMENTS_FILTER="*.jar"
afab640c4c7e8adc82415b728afcb6466499aa5870204f206d79117daeb6ae13
STEP 4: USER root
5543549146a7019f9d229c9de30a32b08129095c49528fc2b671b4cd19a0889b
STEP 5: COPY upload/src /tmp/src
17fd4da7e85b33a7f2e0ac9756bf19a994ca25bf0264b3cdf4a981d59d98d7cb
STEP 6: RUN chown -R 1001:0 /tmp/src
70f3f8fcc452f843553ca7198bbb16e37b43f0205f94f40e11f2a621219a6aaf
STEP 7: USER 1001
dd1745faa92985c8e811976d26b9eb8a7e9c84b9bee3a10ef45ffa6fec54fd51
STEP 8: RUN /usr/local/s2i/assemble
INFO S2I source build with plain binaries detected
INFO Copying binaries from /tmp/src to /deployments ...
.gitignore
.git/
.git/FETCH_HEAD
.git/HEAD
.git/config
.git/description
.git/index
.git/shallow
.git/branches/
.git/hooks/
.git/hooks/applypatch-msg.sample
.git/hooks/commit-msg.sample
.git/hooks/post-update.sample
.git/hooks/pre-applypatch.sample
.git/hooks/pre-commit.sample
.git/hooks/pre-push.sample
.git/hooks/pre-rebase.sample
.git/hooks/pre-receive.sample
.git/hooks/prepare-commit-msg.sample
.git/hooks/update.sample
.git/info/
.git/info/exclude
.git/logs/
.git/logs/HEAD
.git/logs/refs/
.git/logs/refs/heads/
.git/logs/refs/heads/master
.git/logs/refs/remotes/
.git/logs/refs/remotes/origin/
.git/logs/refs/remotes/origin/master
.git/objects/
.git/objects/03/
.git/objects/03/3bddfacfb30c22ef01e9cf9a0f11ed0fa33612
.git/objects/04/
.git/objects/04/11e614ffe6467192a59beeec5a4c779e614f21
.git/objects/0d/
.git/objects/0d/3c995dc302193e00050f89e80ed653ed6f4764
.git/objects/0f/
.git/objects/0f/77bac91be61b3447b381c4909adf6f0229be98
.git/objects/10/
.git/objects/10/ffb0a376950463fb6d56c50979bcf478bcffec
.git/objects/12/
.git/objects/12/e0b09be92038ab52540a79cfc39163c2cfd5b8
.git/objects/14/
.git/objects/14/a9c087f52ab814e83d63f163184b15c0845120
.git/objects/1a/
.git/objects/1a/79e222b1057eac2c9a2c291afc918088ee284e
.git/objects/1b/
.git/objects/1b/4365786834ae5a4f1601bfa50409c49f561ec5
.git/objects/1b/7b296a910ef9dfa946bbccd423ca15281eecce
.git/objects/3c/
.git/objects/3c/a548ab9255696de083fea1fd2d338bb78f3645
.git/objects/3d/
.git/objects/3d/9bb72dd39356e827ce02c6fca453fb31e478c6
.git/objects/42/
.git/objects/42/4978ffd5dac6fe524e766db000e864a2c1ab83
.git/objects/43/
.git/objects/43/b123f063a8cac1d81bae0a5e979ecf03902379
.git/objects/49/
.git/objects/49/65bcf1dc8dc08f0700cbe2e216ebf9321fc1c5
.git/objects/4d/
.git/objects/4d/54f21a3551121246f19e52016b79faf40cd4eb
.git/objects/51/
.git/objects/51/4bb990d034112979dc1fc0d65d55581cea6255
.git/objects/5a/
.git/objects/5a/d1b8eb110d178f23c0c7b1804414839f9fa018
.git/objects/5d/
.git/objects/5d/8a7e0b3c8829c9ac0847b7f36b18e501704aee
.git/objects/64/
.git/objects/64/3ed2e07d3d6a7befee855e3538bf22bd24a3e6
.git/objects/6b/
.git/objects/6b/41e40b65e8130e7707a7949c3d3897e92f5d2a
.git/objects/6e/
.git/objects/6e/182701733de49e725404d7c969ac76abf0aaba
.git/objects/6e/39d6959fd23cb4306cdf41d159811b219d3319
.git/objects/79/
.git/objects/79/d4b7c1d75191b0d3c9e6b475e0506e2ddd4a01
.git/objects/86/
.git/objects/86/14099ab29548fdb4a870c62fad2d899e4cecc0
.git/objects/8b/
.git/objects/8b/518a38e147a708a63a19baaf4be72684534b4f
.git/objects/99/
.git/objects/99/faef2d37479fc3eefd36902caaae5a30017a1e
.git/objects/9a/
.git/objects/9a/06787dd9b7a4421516974a939c566d90866e7c
.git/objects/9c/
.git/objects/9c/416087049c053eb6dfef8348552178a906b8df
.git/objects/a4/
.git/objects/a4/62eb3c783da57371a75dda48de453f61d4e339
.git/objects/a7/
.git/objects/a7/2b7f2e0d0812536628831117ef412029aea7e5
.git/objects/b3/
.git/objects/b3/2bccbbeca67bcfc6a156a9a17385621cdadcde
.git/objects/c1/
.git/objects/c1/a47b5440dc36baf9efa2ff1c82bdec1da53c04
.git/objects/c3/
.git/objects/c3/a3172231d7b8cd3c12d554afec1e502f8b4124
.git/objects/c8/
.git/objects/c8/383b53a90702ce951371ea9642da74e41477eb
.git/objects/c9/
.git/objects/c9/fdbea0aa72261480b79b669bc61eecc6eba6ed
.git/objects/cb/
.git/objects/cb/080307da40fdc7d3466a8fc156dfa56d1bc654
.git/objects/d1/
.git/objects/d1/e2fec3caec2501dcf769b6d9542949e80266b0
.git/objects/d4/
.git/objects/d4/62eec9929ee6c8f12700ff0a4d754793b8ac55
.git/objects/d4/abcf024176b18d18e696ffcda0ae000a5fcd13
.git/objects/d8/
.git/objects/d8/ee8ad67f3b5efbb8198cc002c3088ae79f77ca
.git/objects/e9/
.git/objects/e9/00e97940a8dca74b8896ac84e1107375a1300d
.git/objects/f3/
.git/objects/f3/cef054de67761c54f15cb64adf9da92cdd112d
.git/objects/f6/
.git/objects/f6/6da037c96b3ce0aa63f82c646cae83dd6f8385
.git/objects/f6/eb5377b109999fbd84929bb607c876a6ac9cbf
.git/objects/f9/
.git/objects/f9/c9afb0cf5f714e18745db772a5742c94e532d9
.git/objects/fa/
.git/objects/fa/357b27c9337fd3aa8146e2c260e1f4553b402a
.git/objects/fc/
.git/objects/fc/7ccc396d18dddb0058d34cf37c1d9f05f1e21b
.git/objects/ff/
.git/objects/ff/f22915d799b3c312f3b2ba30857215e4fbfe57
.git/objects/info/
.git/objects/pack/
.git/refs/
.git/refs/heads/
.git/refs/heads/master
.git/refs/remotes/
.git/refs/remotes/origin/
.git/refs/remotes/origin/master
.git/refs/tags/
fruit-backend-sb/
fruit-backend-sb/README.md
fruit-backend-sb/pom.xml
fruit-backend-sb/src/
fruit-backend-sb/src/main/
fruit-backend-sb/src/main/java/
fruit-backend-sb/src/main/java/me/
fruit-backend-sb/src/main/java/me/fruitsand/
fruit-backend-sb/src/main/java/me/fruitsand/demo/
fruit-backend-sb/src/main/java/me/fruitsand/demo/CrudApplication.java
fruit-backend-sb/src/main/java/me/fruitsand/demo/exception/
fruit-backend-sb/src/main/java/me/fruitsand/demo/exception/NotFoundException.java
fruit-backend-sb/src/main/java/me/fruitsand/demo/exception/UnprocessableEntityException.java
fruit-backend-sb/src/main/java/me/fruitsand/demo/exception/UnsupportedMediaTypeException.java
fruit-backend-sb/src/main/java/me/fruitsand/demo/service/
fruit-backend-sb/src/main/java/me/fruitsand/demo/service/Fruit.java
fruit-backend-sb/src/main/java/me/fruitsand/demo/service/FruitController.java
fruit-backend-sb/src/main/java/me/fruitsand/demo/service/FruitRepository.java
fruit-backend-sb/src/main/resources/
fruit-backend-sb/src/main/resources/application-kubernetes.properties
fruit-backend-sb/src/main/resources/application.properties
fruit-backend-sb/src/main/resources/banner.txt
fruit-backend-sb/src/main/resources/static/
fruit-backend-sb/src/main/resources/static/index.html
fruit-backend-sb/src/test/
fruit-backend-sb/src/test/java/
fruit-backend-sb/src/test/java/me/
fruit-backend-sb/src/test/java/me/fruitsand/
fruit-backend-sb/src/test/java/me/fruitsand/demo/
fruit-backend-sb/src/test/java/me/fruitsand/demo/BoosterApplicationTest.java
fruit-backend-sb/src/test/resources/
fruit-backend-sb/src/test/resources/logback-test.xml
fruit-client-sb/
fruit-client-sb/pom.xml
fruit-client-sb/src/
fruit-client-sb/src/main/
fruit-client-sb/src/main/java/
fruit-client-sb/src/main/java/me/
fruit-client-sb/src/main/java/me/fruitsand/
fruit-client-sb/src/main/java/me/fruitsand/demo/
fruit-client-sb/src/main/java/me/fruitsand/demo/Application.java
fruit-client-sb/src/main/java/me/fruitsand/demo/service/
fruit-client-sb/src/main/java/me/fruitsand/demo/service/ClientController.java
fruit-client-sb/src/main/java/me/fruitsand/demo/service/Fruit.java
fruit-client-sb/src/main/resources/
fruit-client-sb/src/main/resources/application.properties
fruit-client-sb/src/main/resources/banner.txt
dd5c551d4d06a8ad6657dc65b6cdb9901b580591c86c8b7e8ac42cdc33b3dae9
STEP 9: CMD /usr/local/s2i/run
STEP 10: COMMIT image-registry.openshift-image-registry.svc:5000/rhtelab/fruit-client-lab
d53841a356dfd0fa360dfb788f0a30edbfa82dda34073bb9a6728c1ae00c5947

Pod log

Starting the Java application using /opt/jboss/container/java/run/run-java.sh ...
ERROR Neither $JAVA_MAIN_CLASS nor $JAVA_APP_JAR is set and 0 JARs found in /deployments (1 expected)
INFO exec  java -javaagent:/opt/jboss/container/jolokia/jolokia.jar=config=/opt/jboss/container/jolokia/etc/jolokia.properties -XX:+UseParallelOldGC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -XX:MaxMetaspaceSize=100m -XX:+ExitOnOutOfMemoryError -cp "." -jar   
Error: -jar requires jar file specification
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
Usage: java [-options] class [args...]
           (to execute a class)
   or  java [-options] -jar jarfile [args...]
           (to execute a jar file)
where options include:
    -d32      use a 32-bit data model if available
    -d64      use a 64-bit data model if available
    -server   to select the "server" VM
                  The default VM is server,
                  because you are running on a server-class machine.
cmoulliard commented 5 years ago

I think that the problem that we are faced is due to the fact that we don't test yet this use case where the project contains several maven modules without a parent pom and then the s2i process dont perform a maven build but instead copy the files when assemble is called

Demo project : https://github.com/rhte-eu/cmoullia.git

WDYT @iocanel

cmoulliard commented 5 years ago

The problem that we must resolve is a maven issue because it will also fail locally if we use the parameter passed to the s2i assmble script -> -pl fruit-client-sb

pwd
/Users/dabou/Temp/rhtelab

ls -la
total 24
drwxr-xr-x  10 dabou  staff   320 Sep 11 14:40 .
drwx---r-x+ 84 dabou  staff  2688 Sep 10 17:32 ..
drwxr-xr-x  12 dabou  staff   384 Sep 11 14:45 .git
-rw-r--r--   1 dabou  staff    16 Sep 10 17:34 .gitignore
drwxr-xr-x  11 dabou  staff   352 Sep 11 14:39 .idea
-rw-r--r--   1 dabou  staff   293 Sep 11 14:40 cp-module.yml
-rw-r--r--   1 dabou  staff   265 Sep 11 12:46 cp.yml
drwxr-xr-x   6 dabou  staff   192 Sep 11 10:10 fruit-backend-sb
drwxr-xr-x   5 dabou  staff   160 Sep 11 10:10 fruit-client-sb
drwxr-xr-x   7 dabou  staff   224 Sep 11 14:09 ocp

mvn clean package -pl fruit-client-sb        
[INFO] Scanning for projects...
[ERROR] [ERROR] Could not find the selected project in the reactor: fruit-client-sb @ 
[ERROR] Could not find the selected project in the reactor: fruit-client-sb -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MavenExecutionException
geoand commented 5 years ago

Either that, or make fruit-client-sb and fruit-backend-sb "children" by adding a pom.xml (containing these two) at the root of https://github.com/rhte-eu/cmoullia

cmoulliard commented 5 years ago

by adding a pom.xml

hal scaffolding command don't generate a parent pom.xml. That could be anyway a good idea.

geoand commented 5 years ago

Yes, that's probably a good way to go

iocanel commented 5 years ago

I think that the problem that we are faced is due to the fact that we don't test yet this use case where the project contains several maven modules without a parent pom and then the s2i process dont perform a maven build but instead copy the files when assemble is called

Demo project : https://github.com/rhte-eu/cmoullia.git

WDYT @iocanel

It think that its possible you are right. Given, that we do have the git root and the maven module root, it should be trivial to fix the contextPath. I am on it.

cmoulliard commented 5 years ago

If maven build is executed by the assemble script, here is what you will then see within the log -> mvn -e -Popenshift -DskipTests -Dcom.redhat.xpaas.repo.redhatga -Dfabric8.skip=true --batch-mode -Djava.net.preferIPv4Stack=true -s /tmp/artifacts/configuration/settings.xml -Dmaven.repo.local=/tmp/artifacts/m2 -pl fruit-client-sb package'

iocanel commented 5 years ago

@cmoulliard can you provide the Task that is named s2i-buildah-push?

cmoulliard commented 5 years ago

can you provide the Task that is named

Of course: see - https://github.com/halkyonio/operator/blob/e992955f6d41f5f1b8f99977c835099ebc9d8125/pkg/controller/component/tasks.go#L63

@iocanel

cmoulliard commented 5 years ago

As the current S2I image checks if there is a pom,xml located under the S2I_SOURCE_DIR - see and that we can't change such ENV VAR as documented here, I suggest that we mention for the end user that a pom.xml file must exist at the root of the git/maven project and this is mandatory

Make sense @iocanel or do you see an alternative like changing S2I_DESTINATION_DIR to point to /tmp/src by default or /tmp/src/moduleDirName if a module exists ?

cmoulliard commented 5 years ago

Using S2I_SOURCE_DIR or S2I_DESTINATION_DIR will not at all help when no pom.xml is defined as this is the rule checked by s2i

  if [ -f "${S2I_SOURCE_DIR}/pom.xml" ]; then
    # maven build
    maven_s2i_maven_build
  else
    # binary build
    maven_s2i_binary_build
  fi