ISTIO artifact Enrichment

[pmbsa]

Description

Support ISTIO fragment enrichment or just ISTIO raw resources (raised in fabric8 as well https://github.com/fabric8io/fabric8-maven-plugin/issues/1820)

Info

Hi, I realise fabric8 is now deprecated, we are planning a migration but that could take some time in the organisation I am working with. I wanted to know if there is a way for me to manage ISTIO resources within a fabric8 project. Even if I cant enrich them during the build I would like to at least be able to apply them so I can keep the delivery mechanisms the same for all projects. I have tried putting the .yml files into a raw folder but I still couldn't get the project to build. Any advice would be much appreciated.

thanks Paul

• f-m-p version : 4.3.1

[manusa]

Hi @pmbsa Could you please provide some code, YAML samples, link to other repos, etc. and describe the current behavior vs. the expected behavior? It would really help us get this feature completed faster.

[pmbsa]

Ye, Sorry, I made the daft assumption that this was something people were already doing so the ticket was structured more as a query than a request.

Right, examples of the yml for ISTIO artifacts are

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: minipbin
spec:
  hosts:
  - "mini.example.com"
  gateways:
  - mygateway-https
  http:
  - match:
    - uri:
        prefix: /status
    - uri:
        prefix: /delay
    route:
    - destination:
        port:
          number: 8000
        host: minibin

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: mygateway-https
spec:
  selector:
    istio: ingressgateway # use istio default ingress gateway
  servers:
  - port:
      number: 443
      name: https
      protocol: HTTPS
    tls:
      mode: SIMPLE
      credentialName: minibin-credential # must be the same as secret
    hosts:
    - mini.example.com

There are of course going to be more than that but as you can see they have an istio api and new "kinds".

Current behavior (for fabric8 which is all I have access to right now)

its worth mentioning that here was no expectation of these working off the bat, I was really just trying to use "raw" to be able to pass this stuff on directly knowing there was no enricher for them. (I did try some of the available istio enrichers but they are all way out of date by now.) mvn clean install -U fabric8:resource

[ERROR] Failed to execute goal io.fabric8:fabric8-maven-plugin:4.3.1:resource (default-cli) on project ISTIOGateway: Execution default-cli of goal io.fabric8:fabric8-maven-plugin:4.3.1:resource failed: No resource type found for:networking.istio.io/v1alpha3#Gateway

I have tried without the apiversion as well (knowing the outcome would be the same of course)

 Failed to execute goal io.fabric8:fabric8-maven-plugin:4.3.1:resource (default-cli) on project ISTIOGateway: Execution default-cli of goal io.fabric8:fabric8-maven-plugin:4.3.1:resource failed: No resource type found for:v1#Gateway

Expected Behavior

I would like to be able to enrich these types of artifacts if it all possible. Failing that, it would be good to be able to at least pass them straight through untouched.

thanks Paul

[manusa]

Thx, no worries, we'll try to find the appropriate solution for your use-case.

These are custom resources and that's why they are not processed as the rest of fragments (they can't be deserialized into Java Objects).

We'll see if we can do something and provide a fix for 1.1.0 milestone.

[rohanKanojia]

Since Istio VirtualService/Gateway is a Custom Resource. Can we for now at least apply them onto Kubernetes using our Custom Resource support?

[manusa]

This is something we need to discuss to see what is the best approach (dedicated Enricher, support for some custom resources, etc.)

If adding CRD support, this means adding an additional dependency or at least the POJOs that would represent those CRDs (in this case for ISTIO's API). It will also mean registering custom kinds, in the plugin, etc. So in case we take this direction, maybe it would be better if this was done in some sort of abstract way so that resources of any kind could be deserialized and enriched.

[pmbsa]

Morning all. While the best approach to this is thought through, is there something we might do internally (with fabric8 not jkube) as a stop gap solution that would allow us to have the ISITO artifacts in the project fabric8 folder, enrich them from the pom and pass them through to the target folder. From there we would use kubectl apply to deploy them anyway. We just need to be able to do that. Do we need to write a custom enricher perhaps that enables the ISTIO api for fabric8? Any advice would be really useful.

[rohanKanojia]

We do have a Custom Enricher sample: https://github.com/fabric8io/fabric8-maven-plugin/tree/master/samples/custom-enricher

However, I haven't tried it myself. I think you should be able to to write an enricher for Istio based on Fabric8 Kubernetes Client's Istio extension: https://github.com/snowdrop/istio-java-api

[rohanKanojia]

I was planning to suggest you our CustomResource approach with which you should be at least able to apply istio manifests onto your cluster on k8s/fabric8:apply phase. It usually works like this:

Add following configuration to pom.xml:

        <plugin>
          <groupId>org.eclipse.jkube</groupId>
          <artifactId>kubernetes-maven-plugin</artifactId>
          <version>${jkube.version}</version>
          <configuration>
            <resources>
                <customResourceDefinitions>
                    <customResourceDefinition>virtualservices.networking.istio.io</customResourceDefinition>
                    <customResourceDefinition>gateways.networking.istio.io</customResourceDefinition>
                </customResourceDefinitions>
            </resources>
          </configuration>
        </plugin>

Note: This doesn't work on master(tested on my branch)

jkube-istio-fragment : $ kubectl get gateway
kNo resources found in default namespace.
jkube-istio-fragment : $ kubectl get virtualservice
No resources found in default namespace.
jkube-istio-fragment : $ mvn k8s:apply
[INFO] Scanning for projects...
[INFO] 
[INFO] --------------------------< com.samples:demo >--------------------------
[INFO] Building demo 1.0.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- kubernetes-maven-plugin:1.0.2-SNAPSHOT:apply (default-cli) @ demo ---
[INFO] k8s: Using Kubernetes at https://192.168.39.24:8443/ in namespace default with manifest /home/rohaan/work/repos/jkube-istio-fragment/target/classes/META-INF/jkube/kubernetes.yml 
[INFO] k8s: Using namespace: default
[INFO] k8s: Creating a Service from kubernetes.yml namespace default name demo
[INFO] k8s: Created Service: target/jkube/applyJson/default/service-demo-9.json
[INFO] k8s: Creating a Deployment from kubernetes.yml namespace default name demo
[INFO] k8s: Created Deployment: target/jkube/applyJson/default/deployment-demo-8.json
[INFO] k8s: Created Custom Resource: networking.istio.io/v1alpha3#VirtualService reviews-route
[INFO] k8s: Created Custom Resource: networking.istio.io/v1alpha3#Gateway mygateway-https
[INFO] k8s: HINT: Use the command `kubectl get pods -w` to watch your pods start up
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  7.694 s
[INFO] Finished at: 2020-10-28T01:03:24+05:30
[INFO] ------------------------------------------------------------------------
jkube-istio-fragment : $ kubectl get gateway
NAME              AGE
mygateway-https   8s
jkube-istio-fragment : $ kubectl get virtualservice
NAME            GATEWAYS   HOSTS                                AGE
reviews-route              ["reviews.prod.svc.cluster.local"]   18s
jkube-istio-fragment : $ 

However, when I was testing this in a reproducer https://github.com/r0haaaan/jkube-istio-fragment , I found out that there is a bug in how JKube resolves CustomResourceDefinitions of same group.

I will create a PR to fix this.

[pmbsa]

Hi Rohan, apologies, I am almost certainly being obtuse here. In an effort to get something moving I am implementing the secret enricher example and then trying to use it in one of my projects. The project seems unable to find the secret-enricher. My fabric8 plugin configuration is an exact match for the example pom and the sourcecode for the enricher is identical. We are using fabric8 4.3.1.

I can see the enricher "name" is set in the constructor

 public SecretEnricher(MavenEnricherContext buildContext) {
    super(buildContext, "secret-enricher");
  }

in the project that uses it I have this config in the pom

               <dependencies>
                    <dependency>
                        <groupId>com.test.kubernetes.istio</groupId>
                        <artifactId>fmp-istio-enricher</artifactId>
                        <version>0.1-SNAPSHOT</version>
                    </dependency>
                </dependencies>
                <configuration>
                   <enricher>
                        <includes>
                            <!-- Include to standard profile -->
                            <include>secret-enricher</include>
                        </includes>
                        <config>
                            <secret-enricher>
                                <!-- Name of the secret is referenced in deployment.yml-->
                                <name>my-secret</name>
                                <!-- Use an URL to load a file -->
                                <mySecretFileData>file:///${project.basedir}/src/main/secret_data.txt</mySecretFileData>
                                <!-- Or a non-URL to include value literally -->
                                <mySecretProperty>my secret property from pom.xml for ${project.artifactId}</mySecretProperty>
                            </secret-enricher>
                        </config>
                    </enricher>        
                </configuration>

something obvious I have missed? I appear to be circling a drain here and it feels like this should be really easy to do. thanks Paul

[pmbsa]

which is the enricher that just does the basics of getting pom values into the yml files btw? that is really all I need to be doing for these files for the time being.

[rohanKanojia]

Umm, Do you have it publicly available so that we can check? Did you add your Enricher Definition to META-INF folder like done here: https://github.com/fabric8io/fabric8-maven-plugin/blob/master/samples/custom-enricher/secret-enricher/src/main/resources/META-INF/fabric8/enricher

Anyway, I tried out changing secret enricher a bit (after some copy pasting) to act for Istio. For me, it worked without any problems(I'm just generating a dummy Gateway manifest). You can find it here: https://github.com/rohanKanojia/custom-istio-enricher .

[pmbsa]

you sir are a legend! I will have a look at that enricher... Yup, the META-INF file is in place just the same... its super odd... sadly I cant share anything I am doing. All internal.... I will take a look at what you have done, really appreciate it

[rohanKanojia]

@pmbsa : :laughing: np(I've done Copy Pasting only). I would really appreciate it if you could migrate to JKube in the future. It's the same old Fabric8 Maven Plugin but with a different name. Our team has put a lot of effort into refactoring FMP out of maven. All we're looking for is support from our old users and feedback.

Maybe after our 1.1.0 milestone, we can add support for Enriching Istio fragments and applying them without any configuration.

[pmbsa]

We will definitely migrate to JKube in the near future. Its not ideal this situation but my customer is a Bank and nothing in a bank ever moves fast.

[pmbsa]

thanks again @rohanKanojia, I repeat my previous statement about your impending sainthood! :D.. That is all we needed. Just including the Istio custom enricher allows for the ISTIO manifests to be enriched and passed down into the target kubernetes.yml. I cant think you enough. As soon as you guys release 1.1.0 of Jkube we will look to migrate for sure.

[irefaat]

@rohanKanojia Thanks a lot for the istio enricher. Please, we need to add the DestinationRule to istio enricher:

destinationrules.networking.istio.io

[rohanKanojia]

Hi @irefaat ,

We don't support generating Istio artifacts right now. If you want to do this you would need to implement a custom enricher yourself. Here is an example of how to do this[0]. In this quickstart we have this enricher IstioEnricher in which we are generating Gateway. Generating DestinationRules should also be similar(If it's supported by Fabric8 Istio Extension

[0] https://github.com/eclipse/jkube/tree/master/quickstarts/kit/custom-istio-enricher

[irefaat]

Hi @irefaat ,

We don't support generating Istio artifacts right now. If you want to do this you would need to implement a custom enricher yourself. Here is an example of how to do this[0]. In this quickstart we have this enricher IstioEnricher in which we are generating Gateway. Generating DestinationRules should also be similar(If it's supported by Fabric8 Istio Extension

[0] https://github.com/eclipse/jkube/tree/master/quickstarts/kit/custom-istio-enricher

@rohanKanojia Thanks a lot. so I will need to add the create one enricher like "https://github.com/eclipse/jkube/tree/master/quickstarts/kit/custom-istio-enricher/istio-enricher", then put it as module in my project? Also please need to know how to add the already existing available enrichers like "istio.gateway" to be enriched/generated for me in my project. Thanks a lot.

[rohanKanojia]

You would need to add it like done in that quickstart:

https://github.com/eclipse/jkube/blob/8a03b82364f9b4aeb771179ab62aa690b89bc078/quickstarts/kit/custom-istio-enricher/app/pom.xml#L54-L88

[irefaat]

@rohanKanojia Agree, but my pom.xml parent is : `

org.springframework.boot

    <artifactId>spring-boot-starter-parent</artifactId>
    <version>2.4.3</version>
    <relativePath />
</parent>

`

not as in the quickstart: https://github.com/eclipse/jkube/blob/8a03b82364f9b4aeb771179ab62aa690b89bc078/quickstarts/kit/custom-istio-enricher/app/pom.xml#L26-L31

[rohanKanojia]

Ohk, I see. If you create this custom enricher as separate project and publish it to your internal maven registry, you should be able to use it I think.