move to networking v1alpha3 api

[bgokden]

We would like to move to istio 0.8 and v1alpha3 of networking api.

[metacosm]

I was planning on working on this this week, awesome to see you did the work already! Looks good after a quick look, will take it for a spin ASAP. Thank you!

[metacosm]

Coud you add some tests for the new network classes as well, please?

[bgokden]

Yes I will.

[bgokden]

I saw somebody asked for 0.8 compatibility. I have made some changes and added tests for VirtualService and Gateway. We are still testing in an actual environment. Now Creating, Updating, Deleting works for VirtualService. We need to test for Gateway. So we will push when it actually works.

[metacosm]

👍

[bgokden]

I have made a change in the properties files, maybe it is not a good way of doing it. I renamed v1alphav2 structures. Maybe there is an easy or right of doing it. This one currently works for us. @avalcepina tested it on a real cluster on gke.

I also added a IstioResourceList, we are using it for some time and it works for us while listing. At first, we used for route rule lists, but now we were able to use it for listing other resources. So the entry or RouteRuleList may not be needed.

In tests I am using withApiVersion to set the api version to right one. I think it is ok.

[metacosm]

Hi @bgokden, let me know when you're done so that I can review and merge. Thanks again!

[bgokden]

Hi, we are testing virtualservice in our app and we found a problem when using PortSelector.

It is protobuf definition is this: // PortSelector specifies the number of a port to be used for // matching or selection for final routing. message PortSelector { oneof port { // Valid port number uint32 number = 1; // $hide_from_docs string name = 2; } }

But the builder can not understand "oneof" definition instead it thinks there should be a Port. Which ends up in a wrong definition. I fixed this by manually editing the istio-schema.json file but it is not a long term solution. Do you know how to fix this?

[metacosm]

@bgokden No, I don't currently have a good solution for this but I will investigate this.

[metacosm]

Regarding IstioResourceList, what do you use it for? Have you looked at: https://github.com/snowdrop/istio-java-api/blob/master/istio-client/src/main/java/me/snowdrop/istio/client/KubernetesAdapter.java#L29? We use KubernetesResourceList…

[bgokden]

We had problem with KubernetesResourceList when we wanted to use list, it didn't cast istio resources. I added another deserializer called IstioListDeserializer, only for the line 49:

IstioResource resource = p.getCodec().treeToValue(elementNode, IstioResource.class);

Maybe it can work with KubernetesResourceList, we can try again.

[metacosm]

When do you plan on being done with this PR?

[bgokden]

I am actually done. With its current status, it works for us.

[metacosm]

Regarding IstioResourceList do you have an example of code that wasn't working with KubernetesResourceList?

[bgokden]

We are re-trying using KubernetesResourceList, right now. This one did not work when we first starting using this library, but we didn't retry for a long time.

`public List listCustomResources(IstioResource resource) { if(resource != null) { final CustomResourceDefinition customResourceDefinition = getCustomResourceDefinition(resource);

        return client.customResources(customResourceDefinition, IstioResource.class, KubernetesResourceList.class,
                DoneableIstioResource.class)
                .inNamespace(client.getNamespace())
                .list();
    }

    return Collections.emptyList();
}`

[avalcepina]

We tried with the following code in Scala:

Option(client.customResources( getCustomResourceDefinition("destinationrules.networking.istio.io", client), classOf[IstioResource], classOf[KubernetesResourceList[IstioResource]], classOf[DoneableIstioResource]) .inNamespace("test-ns") .list()).map(_.getItems.asScala.toSet).getOrElse(Set()) .flatMap(route => Option(route.getMetadata.getName))

If results are retrieved we get the following error:

An error has occurred. io.fabric8.kubernetes.client.KubernetesClientException: An error has occurred. at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:62) at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:53) at io.fabric8.kubernetes.client.utils.Serialization.unmarshal(Serialization.java:237) at io.fabric8.kubernetes.client.utils.Serialization.unmarshal(Serialization.java:190) at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:381) at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:343) at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:327) at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:605) at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:70) at io.vamp.drivers.impl.kubernetes.IstioAlpha3ITSpec.$anonfun$new$2(IstioAlpha3ITSpec.scala:239) at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85) at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83) at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104) at org.scalatest.Transformer.apply(Transformer.scala:22) at org.scalatest.Transformer.apply(Transformer.scala:20) at org.scalatest.WordSpecLike$$anon$1.apply(WordSpecLike.scala:1078) at org.scalatest.TestSuite.withFixture(TestSuite.scala:196) at org.scalatest.TestSuite.withFixture$(TestSuite.scala:195) at io.vamp.drivers.impl.kubernetes.IstioAlpha3ITSpec.withFixture(IstioAlpha3ITSpec.scala:46) at org.scalatest.WordSpecLike.invokeWithFixture$1(WordSpecLike.scala:1076) at org.scalatest.WordSpecLike.$anonfun$runTest$1(WordSpecLike.scala:1088) at org.scalatest.SuperEngine.runTestImpl(Engine.scala:289) at org.scalatest.WordSpecLike.runTest(WordSpecLike.scala:1088) at org.scalatest.WordSpecLike.runTest$(WordSpecLike.scala:1070) at io.vamp.drivers.impl.kubernetes.IstioAlpha3ITSpec.runTest(IstioAlpha3ITSpec.scala:46) at org.scalatest.WordSpecLike.$anonfun$runTests$1(WordSpecLike.scala:1147) at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:396) at scala.collection.immutable.List.foreach(List.scala:389) at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:384) at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:373) at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:410) at scala.collection.immutable.List.foreach(List.scala:389) at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:384) at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:379) at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:461) at org.scalatest.WordSpecLike.runTests(WordSpecLike.scala:1147) at org.scalatest.WordSpecLike.runTests$(WordSpecLike.scala:1146) at io.vamp.drivers.impl.kubernetes.IstioAlpha3ITSpec.runTests(IstioAlpha3ITSpec.scala:46) at org.scalatest.Suite.run(Suite.scala:1147) at org.scalatest.Suite.run$(Suite.scala:1129) at io.vamp.drivers.impl.kubernetes.IstioAlpha3ITSpec.org$scalatest$WordSpecLike$$super$run(IstioAlpha3ITSpec.scala:46) at org.scalatest.WordSpecLike.$anonfun$run$1(WordSpecLike.scala:1192) at org.scalatest.SuperEngine.runImpl(Engine.scala:521) at org.scalatest.WordSpecLike.run(WordSpecLike.scala:1192) at org.scalatest.WordSpecLike.run$(WordSpecLike.scala:1190) at io.vamp.drivers.impl.kubernetes.IstioAlpha3ITSpec.org$scalatest$BeforeAndAfterAll$$super$run(IstioAlpha3ITSpec.scala:46) at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:213) at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210) at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208) at io.vamp.drivers.impl.kubernetes.IstioAlpha3ITSpec.run(IstioAlpha3ITSpec.scala:46) at org.scalatest.tools.SuiteRunner.run(SuiteRunner.scala:45) at org.scalatest.tools.Runner$.$anonfun$doRunRunRunDaDoRunRun$13(Runner.scala:1340) at org.scalatest.tools.Runner$.$anonfun$doRunRunRunDaDoRunRun$13$adapted(Runner.scala:1334) at scala.collection.immutable.List.foreach(List.scala:389) at org.scalatest.tools.Runner$.doRunRunRunDaDoRunRun(Runner.scala:1334) at org.scalatest.tools.Runner$.$anonfun$runOptionallyWithPassFailReporter$24(Runner.scala:1031) at org.scalatest.tools.Runner$.$anonfun$runOptionallyWithPassFailReporter$24$adapted(Runner.scala:1010) at org.scalatest.tools.Runner$.withClassLoaderAndDispatchReporter(Runner.scala:1500) at org.scalatest.tools.Runner$.runOptionallyWithPassFailReporter(Runner.scala:1010) at org.scalatest.tools.Runner$.run(Runner.scala:850) at org.scalatest.tools.Runner.run(Runner.scala) at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.runScalaTest2(ScalaTestRunner.java:131) at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.main(ScalaTestRunner.java:28) Caused by: com.fasterxml.jackson.databind.JsonMappingException: No resource type found for kind:DestinationRuleList at [Source: (BufferedInputStream); line: 1, column: 1548] at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:270) at com.fasterxml.jackson.databind.DeserializationContext.mappingException(DeserializationContext.java:1711) at io.fabric8.kubernetes.internal.KubernetesDeserializer.deserialize(KubernetesDeserializer.java:56) at io.fabric8.kubernetes.internal.KubernetesDeserializer.deserialize(KubernetesDeserializer.java:31) at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4001) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3065) at io.fabric8.kubernetes.client.utils.Serialization.unmarshal(Serialization.java:235) ... 60 more

[bgokden]

The exception is a bit hard to understand but I know the cause. the exceptions thrown at KubernetesDeserializer line 38:

public KubernetesResource deserialize(JsonParser jp, DeserializationContext ctxt) throws IOException, JsonProcessingException { ObjectNode node = (ObjectNode)jp.readValueAsTree(); JsonNode kind = node.get("kind"); if (kind != null) { String value = kind.textValue(); Class<? extends KubernetesResource> resourceType = getTypeForName(value); if (resourceType == null) { throw ctxt.mappingException("No resource type found for kind:" + value); } else { return (KubernetesResource)jp.getCodec().treeToValue(node, resourceType); } } else { return null; } }

The method private static Class getTypeForName(String name) returns null since it only checks in kubernetes packages.

[metacosm]

Indeed. We have an issue opened for this actually: https://github.com/fabric8io/kubernetes-client/issues/942 I didn't see that issue because I was working only with the CRDs, not the CRs.

[metacosm]

I will merge and make the appropriate changes afterwards since I think we don't need IstioResourceList anymore with a newer kubernetes-model version. Thank you again for your work, very much appreciated! 👍