search

apache / incubator-kie-kogito-runtimes

Kogito Runtimes - Kogito is a cloud-native business automation technology for building cloud-ready business applications.
http://kogito.kie.org
Apache License 2.0
536 stars 206 forks source link

Fix #3569 #3572

Closed fjtirado closed 2 months ago

fjtirado commented 3 months ago

Fixes https://github.com/apache/incubator-kie-kogito-runtimes/issues/3569

fjtirado commented 2 months ago

Hi @fjtirado Thanks for the PR. Anyway, beside other issue, I think it would be better to avoid usage of "String" for "paths". We had a lot of issues over the last years exactly for that, and I'm afraid that this modification could bring back them, sooner or later. Maybe, java.nio.file.Path (or similar high-level API) could be used, that automatically take care of the underlyng OS ?

Sure, Path were already being used in FileContentLoader. The change in this PR is for using String rather than URI in URIContentLoaderFactory. URIContentLoaderFactory then parse the String and create an HTTPContentLoader, FileContentLoader or ClasspathContentLoader instance. FileContentLoader, which deals with files, were already using Path (but now, rathern than obtaining the Path from the URI, it obtains it from the String) The failure in windows was happening in URI.create (the URI has to be created from String in most instance, thats why using String makes full sense). This PR avoid that issue but replacing String by URI (the reason it affects so many classes is because URI was used for resource loading and openapi config key generation)

fjtirado commented 2 months ago

Thanks @fjtirado I see very few minor changes that could be done. Anyway, the biggest issue is to understand reason of reproducible failures and fix it, otherwise that PR can't be merged

As proof by this PR https://github.com/apache/incubator-kie-kogito-runtimes/pull/3579, it seems that any change in openapi-common jar will cause the reproducible build to complain. Im afraid is because maven-invoker takes old snapshots (for indirect build only dependencies like this one). In my opinon, we can ignore the issue and merge, since once the jar is installed (as in happened in a local build) maven-invoker wont complain.

gitgabrio commented 2 months ago

@fjtirado Sorry, but I think we can't ignore reproducible failure and we have to fix it before merging this one, even if it is not strictly related. If this is the problem -> Im afraid is because maven-invoker takes old snapshots (for indirect build only dependencies like this one). (and I have the impression it is very reasanable) then we have to fix that in the failing module - we have same similar setup elsewhere, and in the past we already faced similar issues for integration-tests. @baldimir ^^

Last note: I'm afraid reproducible build is a strong requirement, and we spent lot of time to fix it, we can't simply break it again.

fjtirado commented 2 months ago

@gitgabrio Im not saying we do not have to fix reproducible for springboot it test. Which I do not think is acceptable is to block any PR changing the openapi-common jar (and other jars which are build dependencies and not runtimes) because reproducilbe is messing up for those jars.
We will not be breaking anything, as proof in local calls, once the JAR is installed in the repo, springboot test reproducible will work

fjtirado commented 2 months ago

In any case, Ill try to fix the issue on springboot it test using this PR https://github.com/apache/incubator-kie-kogito-runtimes/pull/3579 first.

fjtirado commented 2 months ago

In any case, Ill try to fix the issue on springboot it test using this PR #3579 first.

Both disabling comparison goal for SpringBoot IT test module (see commit) or adding the openapi-common dependency for SpringBoot IT module works, but since @gitgabrio do not like the former and I do not think the latter is acceptable (because openapi-common is not a dependency of springboot it test, it is only used by Quarkus) and considering that the issue is that compare goal is including in the comparison list modules that are not SpringBoot dependencies (in the oklist in the comparison report file there is a bunch of quarkus modules) Im now trying to tell the artifact plugin that it should not consider the modified file. Its an intermediate solution that should be acceptable for anyone. I upgraded the maven artifact version because skipModules property requires >3.5.0

gitgabrio commented 2 months ago

thanks @fjtirado

but since @gitgabrio do not like the former

it is not my personal choice, but how we, as community, agreed to be compliant to the apache requirements. @porcelli @baldimir : please correct me if I'm misunderstanding something.

I do not think the latter is acceptable (because openapi-common is not a dependency of springboot it test, it is only used by Quarkus)

it seems that is a transitive dependency, so, regardless, still a dependency. But, again, I would suggest to double check how the twin IT module test behave, how it is configured, and why it seems to not be affected by the same issue.

fjtirado commented 2 months ago

it seems that is a transitive dependency, so, r

@gitgabrio There is not transitive dependency, thats the key thing. If there was, the problem wont appear because the modified jar will be installed by maven invoker in the local repository and the jars will be equal. Compare is complaining because is comparing the modified jar with the latest one available in the central repo (since the jar changed they are different)
Therefore, There is a problen with the compare for the SpringBoot IT test module, that is including all jars in the lot (see the quarkus ones in the provided list). Also, the requirement is to be reproducilbe for jars that are delivered as part of the release. Integration test modules are not released (and if they are, they should not)

fjtirado commented 2 months ago

@gitgabrio When upgrading maven artifact version from 3.4.1 to 3.5.1 in order to be able to use skipModules property, there is now this log in the springboot IT. I think thats a clear message than sooner than later we will need to disable it for IT test. 

2024-07-22T11:51:06.7483964Z [INFO] --- artifact:3.5.1:compare (compare) @ kogito-spring-boot-integration-tests ---
2024-07-22T11:51:06.7494111Z [WARNING] SCM source tag in buildinfo source.scm.tag=HEAD does not permit rebuilders reproducible source checkout
2024-07-22T11:51:06.8367280Z [INFO] Saved aggregate info on build to /home/runner/work/incubator-kie-kogito-runtimes/incubator-kie-kogito-runtimes/apache_incubator-kie-kogito-runtimes/springboot/integration-tests/target/kogito-spring-boot-integration-tests-999-SNAPSHOT.buildinfo
2024-07-22T11:51:06.8373658Z [INFO] Aggregate buildinfo copied to /home/runner/work/incubator-kie-kogito-runtimes/incubator-kie-kogito-runtimes/apache_incubator-kie-kogito-runtimes/target/kogito-runtimes-999-SNAPSHOT.buildinfo
2024-07-22T11:51:06.8375828Z [INFO] Checking against reference build from central...
2024-07-22T11:51:06.8377734Z [INFO] Reference buildinfo file not found: it will be generated from downloaded reference artifacts
2024-07-22T11:51:06.9215653Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-messaging:pom:999-SNAPSHOT
2024-07-22T11:51:06.9300875Z [WARNING] Reference artifact not found org.drools:drools-quarkus-rules:pom:999-SNAPSHOT
2024-07-22T11:51:06.9304077Z [WARNING] Reference artifact not found org.drools:drools-quarkus-decisions:pom:999-SNAPSHOT
2024-07-22T11:51:06.9307515Z [WARNING] Reference artifact not found org.kie:kie-quarkus-predictions:pom:999-SNAPSHOT
2024-07-22T11:51:06.9329618Z [WARNING] Reference artifact not found org.drools:drools-quarkus-rules-deployment:pom:999-SNAPSHOT
2024-07-22T11:51:06.9331727Z [WARNING] Reference artifact not found org.drools:drools-quarkus-decisions-deployment:pom:999-SNAPSHOT
2024-07-22T11:51:06.9333479Z [WARNING] Reference artifact not found org.kie:kie-quarkus-predictions-deployment:pom:999-SNAPSHOT
2024-07-22T11:51:06.9335444Z [WARNING] Reference artifact not found org.kie:kogito-addons-quarkus-common-deployment:pom:999-SNAPSHOT
2024-07-22T11:51:06.9337569Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-messaging-deployment:pom:999-SNAPSHOT
2024-07-22T11:51:06.9354021Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-persistence-jdbc:pom:999-SNAPSHOT
2024-07-22T11:51:06.9360411Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-events-process:pom:999-SNAPSHOT
2024-07-22T11:51:06.9374171Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-process-management:pom:999-SNAPSHOT
2024-07-22T11:51:06.9391221Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-persistence-jdbc-deployment:pom:999-SNAPSHOT
2024-07-22T11:51:06.9393497Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-events-process-deployment:pom:999-SNAPSHOT
2024-07-22T11:51:06.9395564Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-process-management-deployment:pom:999-SNAPSHOT
2024-07-22T11:51:06.9420009Z [WARNING] Reference artifact not found org.drools:drools-quarkus-rules-integration-test:pom:999-SNAPSHOT
2024-07-22T11:51:06.9422055Z [WARNING] Reference artifact not found org.drools:drools-quarkus-rules-integration-test-hot-reload:pom:999-SNAPSHOT
2024-07-22T11:51:06.9424241Z [WARNING] Reference artifact not found org.drools:drools-quarkus-decisions-integration-test:pom:999-SNAPSHOT
2024-07-22T11:51:06.9426300Z [WARNING] Reference artifact not found org.drools:drools-quarkus-decisions-integration-test-hot-reload:pom:999-SNAPSHOT
2024-07-22T11:51:06.9428330Z [WARNING] Reference artifact not found org.kie:kie-quarkus-predictions-integration-test:pom:999-SNAPSHOT
2024-07-22T11:51:06.9470464Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-persistence-infinispan:pom:999-SNAPSHOT
2024-07-22T11:51:06.9472857Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-persistence-infinispan-deployment:pom:999-SNAPSHOT
2024-07-22T11:51:06.9487851Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-persistence-kafka:pom:999-SNAPSHOT
2024-07-22T11:51:06.9489816Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-persistence-kafka-deployment:pom:999-SNAPSHOT
2024-07-22T11:51:06.9504969Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-persistence-filesystem:pom:999-SNAPSHOT
2024-07-22T11:51:06.9507007Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-persistence-filesystem-deployment:pom:999-SNAPSHOT
2024-07-22T11:51:06.9524367Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-persistence-mongodb:pom:999-SNAPSHOT
2024-07-22T11:51:06.9526394Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-persistence-mongodb-deployment:pom:999-SNAPSHOT
2024-07-22T11:51:06.9540791Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-persistence-postgresql:pom:999-SNAPSHOT
2024-07-22T11:51:06.9542847Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-persistence-postgresql-deployment:pom:999-SNAPSHOT
2024-07-22T11:51:06.9561311Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-persistence-rocksdb:pom:999-SNAPSHOT
2024-07-22T11:51:06.9563289Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-persistence-rocksdb-deployment:pom:999-SNAPSHOT
2024-07-22T11:51:06.9565429Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-process-management-integration-tests:pom:999-SNAPSHOT
2024-07-22T11:51:06.9567299Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-source-files:pom:999-SNAPSHOT
2024-07-22T11:51:06.9568970Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-source-files-deployment:pom:999-SNAPSHOT
2024-07-22T11:51:06.9577203Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-knative-eventing:pom:999-SNAPSHOT
2024-07-22T11:51:06.9579049Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-knative-eventing-deployment:pom:999-SNAPSHOT
2024-07-22T11:51:06.9581226Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-knative-eventing-integration-tests:pom:999-SNAPSHOT
2024-07-22T11:51:06.9613220Z [WARNING] Reference artifact not found org.kie:kogito-addons-quarkus-fabric8-kubernetes-service-catalog:pom:999-SNAPSHOT
2024-07-22T11:51:06.9615909Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-kubernetes:pom:999-SNAPSHOT
2024-07-22T11:51:06.9617652Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-kubernetes-deployment:pom:999-SNAPSHOT
2024-07-22T11:51:06.9621457Z [WARNING] Reference artifact not found org.kie:kogito-addons-quarkus-knative-serving:pom:999-SNAPSHOT
2024-07-22T11:51:06.9624516Z [WARNING] Reference artifact not found org.kie:kogito-addons-quarkus-knative-serving-deployment:pom:999-SNAPSHOT
2024-07-22T11:51:06.9627629Z [WARNING] Reference artifact not found org.kie:kogito-addons-quarkus-fabric8-kubernetes-service-catalog-deployment:pom:999-SNAPSHOT
2024-07-22T11:51:06.9630390Z [WARNING] Reference artifact not found org.kie:kogito-addons-quarkus-knative-serving-integration-tests:pom:999-SNAPSHOT
2024-07-22T11:51:06.9633022Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-kubernetes-integration-tests:pom:999-SNAPSHOT
2024-07-22T11:51:06.9643954Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-events-decisions:pom:999-SNAPSHOT
2024-07-22T11:51:06.9647089Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-events-decisions-deployment:pom:999-SNAPSHOT
2024-07-22T11:51:06.9659140Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-events-predictions:pom:999-SNAPSHOT
2024-07-22T11:51:06.9662098Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-events-predictions-deployment:pom:999-SNAPSHOT
2024-07-22T11:51:06.9673898Z [WARNING] Reference artifact not found org.drools:drools-addons-quarkus-events-rules:pom:999-SNAPSHOT
2024-07-22T11:51:06.9677271Z [WARNING] Reference artifact not found org.drools:drools-addons-quarkus-events-rules-deployment:pom:999-SNAPSHOT
2024-07-22T11:51:06.9691600Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-events-mongodb:pom:999-SNAPSHOT
2024-07-22T11:51:06.9694433Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-events-mongodb-deployment:pom:999-SNAPSHOT
2024-07-22T11:51:06.9709275Z [WARNING] Reference artifact not found org.kie:kogito-addons-quarkus-microprofile-config-service-catalog:pom:999-SNAPSHOT
2024-07-22T11:51:06.9713624Z [WARNING] Reference artifact not found org.kie:kogito-addons-quarkus-microprofile-config-service-catalog-deployment:pom:999-SNAPSHOT
2024-07-22T11:51:06.9716267Z [WARNING] Reference artifact not found org.kie:kogito-addons-quarkus-microprofile-config-service-catalog-integration-tests:pom:999-SNAPSHOT
2024-07-22T11:51:06.9760305Z [WARNING] Reference artifact not found org.kie:kogito-addons-quarkus-jobs-management:pom:999-SNAPSHOT
2024-07-22T11:51:06.9762276Z [WARNING] Reference artifact not found org.kie:kogito-addons-quarkus-jobs-management-deployment:pom:999-SNAPSHOT
2024-07-22T11:51:06.9770563Z [WARNING] Reference artifact not found org.kie:kogito-addons-quarkus-jobs-messaging:pom:999-SNAPSHOT
2024-07-22T11:51:06.9772404Z [WARNING] Reference artifact not found org.kie:kogito-addons-quarkus-jobs-messaging-deployment:pom:999-SNAPSHOT
2024-07-22T11:51:06.9774362Z [WARNING] Reference artifact not found org.kie:kogito-addons-quarkus-jobs-knative-eventing:pom:999-SNAPSHOT
2024-07-22T11:51:06.9776314Z [WARNING] Reference artifact not found org.kie:kogito-addons-quarkus-jobs-knative-eventing-deployment:pom:999-SNAPSHOT
2024-07-22T11:51:06.9795146Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-explainability:pom:999-SNAPSHOT
2024-07-22T11:51:06.9797234Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-explainability-deployment:pom:999-SNAPSHOT
2024-07-22T11:51:06.9799116Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-explainability-it:pom:999-SNAPSHOT
2024-07-22T11:51:06.9856042Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-monitoring-prometheus:pom:999-SNAPSHOT
2024-07-22T11:51:06.9858148Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-monitoring-prometheus-deployment:pom:999-SNAPSHOT
2024-07-22T11:51:06.9871205Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-monitoring-elastic:pom:999-SNAPSHOT
2024-07-22T11:51:06.9873415Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-monitoring-elastic-deployment:pom:999-SNAPSHOT
2024-07-22T11:51:06.9905857Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-process-svg:pom:999-SNAPSHOT
2024-07-22T11:51:06.9907673Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-process-svg-deployment:pom:999-SNAPSHOT
2024-07-22T11:51:06.9948337Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-marshallers-avro:pom:999-SNAPSHOT
2024-07-22T11:51:06.9950273Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-marshallers-avro-deployment:pom:999-SNAPSHOT
2024-07-22T11:51:06.9974493Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-process-definitions:pom:999-SNAPSHOT
2024-07-22T11:51:06.9976507Z [WARNING] Reference artifact not found org.kie:kie-addons-quarkus-process-definitions-deployment:pom:999-SNAPSHOT
2024-07-22T11:51:07.0004011Z [WARNING] Reference artifact not found org.kie.kogito:integration-tests-quarkus-openapi-client:pom:999-SNAPSHOT
2024-07-22T11:51:07.0005964Z [WARNING] Reference artifact not found org.kie.kogito:integration-tests-quarkus-rules:pom:999-SNAPSHOT
2024-07-22T11:51:07.0007881Z [WARNING] Reference artifact not found org.kie.kogito:integration-tests-quarkus-legacy-rules:pom:999-SNAPSHOT
2024-07-22T11:51:07.0009811Z [WARNING] Reference artifact not found org.kie.kogito:integration-tests-quarkus-decisions:pom:999-SNAPSHOT
2024-07-22T11:51:07.0011755Z [WARNING] Reference artifact not found org.kie.kogito:integration-tests-quarkus-resteasy-classic:pom:999-SNAPSHOT
2024-07-22T11:51:07.0036174Z [WARNING] Reference artifact not found org.kie.kogito:integration-tests-quarkus-resteasy-reactive:pom:999-SNAPSHOT
2024-07-22T11:51:07.0038169Z [WARNING] Reference artifact not found org.kie.kogito:integration-tests-kogito-plugin:pom:999-SNAPSHOT
2024-07-22T11:51:07.0040007Z [WARNING] Reference artifact not found org.kie.kogito:integration-tests-quarkus-norest:pom:999-SNAPSHOT
2024-07-22T11:51:07.0041880Z [WARNING] Reference artifact not found org.kie.kogito:integration-tests-quarkus-processes:pom:999-SNAPSHOT
2024-07-22T11:51:07.0043866Z [WARNING] Reference artifact not found org.kie.kogito:integration-tests-quarkus-processes-reactive:pom:999-SNAPSHOT
2024-07-22T11:51:07.0045969Z [WARNING] Reference artifact not found org.kie.kogito:integration-tests-processes-persistence-common:pom:999-SNAPSHOT
2024-07-22T11:51:07.0048109Z [WARNING] Reference artifact not found org.kie.kogito:integration-tests-quarkus-processes-postgresql:pom:999-SNAPSHOT
2024-07-22T11:51:07.0050218Z [WARNING] Reference artifact not found org.kie.kogito:integration-tests-quarkus-processes-infinispan:pom:999-SNAPSHOT
2024-07-22T11:51:07.0052297Z [WARNING] Reference artifact not found org.kie.kogito:integration-tests-quarkus-processes-mongodb:pom:999-SNAPSHOT
2024-07-22T11:51:07.0054543Z [WARNING] Reference artifact not found org.kie.kogito:integration-tests-quarkus-processes-kafka-persistence:pom:999-SNAPSHOT
2024-07-22T11:51:07.0056681Z [WARNING] Reference artifact not found org.kie.kogito:integration-tests-quarkus-processes-jdbc:pom:999-SNAPSHOT
2024-07-22T11:51:07.0058757Z [WARNING] Reference artifact not found org.kie.kogito:integration-tests-quarkus-processes-filesystem:pom:999-SNAPSHOT
2024-07-22T11:51:07.0060778Z [WARNING] Reference artifact not found org.kie.kogito:integration-tests-quarkus-source-files:pom:999-SNAPSHOT
2024-07-22T11:51:07.0062837Z [WARNING] Reference artifact not found org.kie.kogito:integration-tests-quarkus-gradle:pom:999-SNAPSHOT
2024-07-22T11:51:07.0524468Z [INFO] Reference build java.version: 17 (from MANIFEST.MF Build-Jdk-Spec)
2024-07-22T11:51:07.0525869Z [INFO] Reference build os.name: Unix (from pom.properties newline)
2024-07-22T11:51:07.1297187Z [INFO] Minimal buildinfo generated from downloaded artifacts: /home/runner/work/incubator-kie-kogito-runtimes/incubator-kie-kogito-runtimes/apache_incubator-kie-kogito-runtimes/target/reference/kogito-spring-boot-integration-tests-999-SNAPSHOT.buildinfo
2024-07-22T11:51:07.2140644Z [ERROR] size mismatch kie-addons-quarkus-messaging-999-SNAPSHOT.pom: investigate with diffoscope target/reference/org.kie/kie-addons-quarkus-messaging-999-SNAPSHOT.pom quarkus/addons/messaging/runtime/pom.xml