search

jqassistant-tutorials / jakarta-ee-dependencies

This project provides a setup to scan Maven Central repository for Jakarta EE artifacts and provide reports about dependencies of the involved specifications.
GNU General Public License v3.0
0 stars 2 forks source link

Jakarta EE 10.0.0-RC1 can not be feched from Staging repo #1

Open JanWesterkamp-iJUG opened 2 years ago

JanWesterkamp-iJUG commented 2 years ago

When the platform.artifact=jakarta.platform:jakarta.jakartaee-api:jar:10.0.0-RC1 (Jakarta EE 10 RC1) from the repo.url=https://jakarta.oss.sonatype.org/content/repositories/staging/ (Staging) is configured an empty result will be generated for the reports.

The Maven Artifact is available in the repo (https://jakarta.oss.sonatype.org/content/repositories/staging/jakarta/platform/jakarta.jakartaee-api/), but not fetched at all according to the search in the Maven log.

Further analysis using version 9.1.0-RC1 verifies this is not caused because of a root RC[1] version. When not cleaning the data, multiple runs for different repos are possible to be aggregated - but this works only with the same Maven Artifact version (!) - when the version deviates, then additional data is not attached (this might be a separate issue).

There is a bug in the Nexus too, that might interfere here: When opening the page https://jakarta.oss.sonatype.org/content/repositories/staging/jakarta/platform/jakarta.jakartaee-api/ the Last Modified column contains wrong timestamps for some of the meta data files!

Example: Site (https://jakarta.oss.sonatype.org/content/repositories/staging/jakarta/platform/jakarta.jakartaee-api/) shows lines: ... Name | Last Modified | Size | Description ... maven-metadata.xml Mon Nov 02 17:03:41 UTC 2020 297 ...

File maven-matedata.xml (https://jakarta.oss.sonatype.org/content/repositories/staging/jakarta/platform/jakarta.jakartaee-api/maven-metadata.xml) contains element: ...

**20220209030723**

...

JanWesterkamp-iJUG commented 2 years ago

This issue was first reported by Scott in the mainling list here: https://www.eclipse.org/lists/jakartaee-platform-dev/msg03246.html

@DirkMahler do you have any ideas where the root cause is located, in jQA or Nexus? If Nexus has the problem, is there a possibility to have a (simple) workaround for this (i.e. parsing the meta data file element instead of the site) or do you have observations like this with the jQA Maven Plugin before?

DirkMahler commented 2 years ago

The issue most likely is related to an outdated index in the repository:

[INFO] Updating repository index, this may take a while...
...
[INFO] Updated Maven index timestamp: Tue Dec 21 20:44:22 CET 2021

So the artifact itself is present but cannot be found by the jQAssistant scanner.

JanWesterkamp-iJUG commented 2 years ago

Thanks, I created an issue at Sonatype, where the Jakarta Staging is hosted, to get this fixed there: https://issues.sonatype.org/browse/OSSRH-78084

starksm64 commented 2 years ago

It looks like the jakarta.oss.sonatype.org repo index has been updated, so I updated my clone with the latest changes and reran verify with 10.0.0-RC1 as the platform.version. I'm still seeing empty dependency diagrams produced:

[INFO] Using local repository '/Users/starksm/Dev/JBoss/Jakarta/EE10/jakarta-ee-dependencies/cache/jqassistant/m2repo/repo1.maven.org/maven2' for URL 'https://repo1.maven.org/maven2'
[INFO] Updating repository index, this may take a while...
[INFO] Received a full update.
[INFO] Updated Maven index timestamp: Tue Feb 15 13:15:26 CST 2022
[INFO] Executing query for artifacts that have been updated since Wed Dec 31 18:00:00 CST 1969.
[INFO] Artifact query returned 26683939 hits (total).
[INFO] Starting scan.
...
[INFO] Scanning model 'jakarta.authorization:jakarta.authorization-api:pom:2.1.0-RC1'.
[INFO] Processed 2000/26683939 artifacts (duration: PT10M34.338S).
[INFO] Processed 2500/26683939 artifacts (duration: PT10M34.62S).
[INFO] Processed 3000/26683939 artifacts (duration: PT10M34.896S).
[INFO] Finished scan: 3171 artifacts (duration: PT10M40.55S).
[INFO] --- jqassistant-maven-plugin:1.11.1:analyze (default-cli) @ jakarta-ee-dependencies ---
[INFO] Executing analysis for 'jakarta-ee-dependencies'.
[INFO] Will warn on violations starting from severity 'MINOR'
[INFO] Will fail on violations starting from severity 'MAJOR'.
[INFO] Reading rules from directory '/Users/starksm/Dev/JBoss/Jakarta/EE10/jakarta-ee-dependencies/jqassistant'.
[INFO] Executing group 'jakarta-ee-dependencies'
[INFO] Applying concept 'jakarta-ee-dependencies:ArtifactDependencies' with severity: 'MINOR'.
[INFO] Applying concept 'jakarta-ee-dependencies:ArtifactLastModifiedAsString' with severity: 'MINOR'.
[INFO] Applying concept 'jakarta-ee-dependencies:PlatformArtifact' with severity: 'MINOR'.
[INFO] Applying concept 'jakarta-ee-dependencies:PlatformDependencyDiagram' with severity: 'MINOR'.
[INFO] Rendering diagram '/Users/starksm/Dev/JBoss/Jakarta/EE10/jakarta-ee-dependencies/target/jqassistant/report/plantuml/jakarta-ee-dependencies_PlatformDependencyDiagram.svg' 
[INFO] Applying concept 'jakarta-ee-dependencies:PlatformDependencyVersionsByArtifact' with severity: 'MINOR'.
[INFO] Applying concept 'jakarta-ee-dependencies:Specification' with severity: 'MINOR'.
[INFO] Applying concept 'jakarta-ee-dependencies:SpecificationDependencyDiagram' with severity: 'MINOR'.
[INFO] Rendering diagram '/Users/starksm/Dev/JBoss/Jakarta/EE10/jakarta-ee-dependencies/target/jqassistant/report/plantuml/jakarta-ee-dependencies_SpecificationDependencyDiagram.svg' 
[WARNING] --[ Concept Application Failure ]----------------------------------
[WARNING] Concept: jakarta-ee-dependencies:PlatformArtifact
[WARNING] Severity: MINOR
[WARNING] Number of rows: 0
[WARNING] Creates reports about dependencies as GraphML file and PlantUML component diagram.
[WARNING] -------------------------------------------------------------------
[WARNING] 

[WARNING] --[ Concept Application Failure ]----------------------------------
[WARNING] Concept: jakarta-ee-dependencies:PlatformDependencyDiagram
[WARNING] Severity: MINOR
[WARNING] Number of rows: 0
[WARNING] Creates reports about dependencies as GraphML file and PlantUML component diagram.
[WARNING] -------------------------------------------------------------------
[WARNING] 

[WARNING] --[ Concept Application Failure ]----------------------------------
[WARNING] Concept: jakarta-ee-dependencies:PlatformDependencyVersionsByArtifact
[WARNING] Severity: MINOR
[WARNING] Number of rows: 0
[WARNING] Creates a CSV report about dependency versions.
[WARNING] -------------------------------------------------------------------
[WARNING] 

[WARNING] --[ Concept Application Failure ]----------------------------------
[WARNING] Concept: jakarta-ee-dependencies:SpecificationDependencyDiagram
[WARNING] Severity: MINOR
[WARNING] Number of rows: 0
[WARNING] Creates reports about dependencies between specifications as GraphML file and PlantUML component diagram.
[WARNING] -------------------------------------------------------------------
[WARNING] 

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  30:40 min
[INFO] Finished at: 2022-02-22T22:28:30-06:00
[INFO] ------------------------------------------------------------------------
[INFO] Closing connection to store 'file:/Users/starksm/Dev/JBoss/Jakarta/EE10/jakarta-ee-dependencies/target/jqassistant/store/'.
DirkMahler commented 2 years ago

Hi,

I just tried using the latest greatest from the original repo (https://github.com/jqassistant-demo/jakarta-ee-dependencies) and ran the following command:

mvn verify -Drepo.url=https://jakarta.oss.sonatype.org/content/repositories/staging/ -Dplatform.version=10.0.0-RC1

This seemed to work, see attached diagram.

Cheers

Dirk

------ Originalnachricht ------ Von: "Scott M Stark" @.> An: "jqassistant-demo/jakarta-ee-dependencies" @.> Cc: "Dirk Mahler" @.>; "Mention" @.> Gesendet: 23.02.2022 05:38:48 Betreff: Re: [jqassistant-demo/jakarta-ee-dependencies] Jakarta EE 10.0.0-RC1 can not be feched from Staging repo (Issue #1)

It looks like the jakarta.oss.sonatype.org repo index has been updated, so I updated my clone with the latest changes and reran verify with 10.0.0-RC1 as the platform.version. I'm still seeing empty dependency diagrams produced:

[INFO] Using local repository '/Users/starksm/Dev/JBoss/Jakarta/EE10/jakarta-ee-dependencies/cache/jqassistant/m2repo/repo1.maven.org/maven2' for URL 'https://repo1.maven.org/maven2' [INFO] Updating repository index, this may take a while... [INFO] Received a full update. [INFO] Updated Maven index timestamp: Tue Feb 15 13:15:26 CST 2022 [INFO] Executing query for artifacts that have been updated since Wed Dec 31 18:00:00 CST 1969. [INFO] Artifact query returned 26683939 hits (total). [INFO] Starting scan. ... [INFO] Scanning model 'jakarta.authorization:jakarta.authorization-api:pom:2.1.0-RC1'. [INFO] Processed 2000/26683939 artifacts (duration: PT10M34.338S). [INFO] Processed 2500/26683939 artifacts (duration: PT10M34.62S). [INFO] Processed 3000/26683939 artifacts (duration: PT10M34.896S). [INFO] Finished scan: 3171 artifacts (duration: PT10M40.55S). [INFO] --- jqassistant-maven-plugin:1.11.1:analyze (default-cli) @ jakarta-ee-dependencies --- [INFO] Executing analysis for 'jakarta-ee-dependencies'. [INFO] Will warn on violations starting from severity 'MINOR' [INFO] Will fail on violations starting from severity 'MAJOR'. [INFO] Reading rules from directory '/Users/starksm/Dev/JBoss/Jakarta/EE10/jakarta-ee-dependencies/jqassistant'. [INFO] Executing group 'jakarta-ee-dependencies' [INFO] Applying concept 'jakarta-ee-dependencies:ArtifactDependencies' with severity: 'MINOR'. [INFO] Applying concept 'jakarta-ee-dependencies:ArtifactLastModifiedAsString' with severity: 'MINOR'. [INFO] Applying concept 'jakarta-ee-dependencies:PlatformArtifact' with severity: 'MINOR'. [INFO] Applying concept 'jakarta-ee-dependencies:PlatformDependencyDiagram' with severity: 'MINOR'. [INFO] Rendering diagram '/Users/starksm/Dev/JBoss/Jakarta/EE10/jakarta-ee-dependencies/target/jqassistant/report/plantuml/jakarta-ee-dependencies_PlatformDependencyDiagram.svg' [INFO] Applying concept 'jakarta-ee-dependencies:PlatformDependencyVersionsByArtifact' with severity: 'MINOR'. [INFO] Applying concept 'jakarta-ee-dependencies:Specification' with severity: 'MINOR'. [INFO] Applying concept 'jakarta-ee-dependencies:SpecificationDependencyDiagram' with severity: 'MINOR'. [INFO] Rendering diagram '/Users/starksm/Dev/JBoss/Jakarta/EE10/jakarta-ee-dependencies/target/jqassistant/report/plantuml/jakarta-ee-dependencies_SpecificationDependencyDiagram.svg' [WARNING] --[ Concept Application Failure ]---------------------------------- [WARNING] Concept: jakarta-ee-dependencies:PlatformArtifact [WARNING] Severity: MINOR [WARNING] Number of rows: 0 [WARNING] Creates reports about dependencies as GraphML file and PlantUML component diagram. [WARNING] ------------------------------------------------------------------- [WARNING]

[WARNING] --[ Concept Application Failure ]---------------------------------- [WARNING] Concept: jakarta-ee-dependencies:PlatformDependencyDiagram [WARNING] Severity: MINOR [WARNING] Number of rows: 0 [WARNING] Creates reports about dependencies as GraphML file and PlantUML component diagram. [WARNING] ------------------------------------------------------------------- [WARNING]

[WARNING] --[ Concept Application Failure ]---------------------------------- [WARNING] Concept: jakarta-ee-dependencies:PlatformDependencyVersionsByArtifact [WARNING] Severity: MINOR [WARNING] Number of rows: 0 [WARNING] Creates a CSV report about dependency versions. [WARNING] ------------------------------------------------------------------- [WARNING]

[WARNING] --[ Concept Application Failure ]---------------------------------- [WARNING] Concept: jakarta-ee-dependencies:SpecificationDependencyDiagram [WARNING] Severity: MINOR [WARNING] Number of rows: 0 [WARNING] Creates reports about dependencies between specifications as GraphML file and PlantUML component diagram. [WARNING] ------------------------------------------------------------------- [WARNING]

[INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 30:40 min [INFO] Finished at: 2022-02-22T22:28:30-06:00 [INFO] ------------------------------------------------------------------------ [INFO] Closing connection to store 'file:/Users/starksm/Dev/JBoss/Jakarta/EE10/jakarta-ee-dependencies/target/jqassistant/store/'.

— Reply to this email directly, view it on GitHub https://github.com/jqassistant-demo/jakarta-ee-dependencies/issues/1#issuecomment-1048443726, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADK5JMOCW6X7TTVS6QRRRTU4RQFRANCNFSM5N5IHJMA. You are receiving this because you were mentioned.Message ID: @.***>

JanWesterkamp-iJUG commented 2 years ago

Hi Scott,

can you provide your POM properties configuration or check them, please?

The repo configuration might still point to Maven central ('https://repo1.maven.org/maven2') and then the report would be correct, because 10.0.0-RC1 can not be found there, when interpret your log right.

Best, Jan

Am 23.02.22 um 05:38 schrieb Scott M Stark:

It looks like the jakarta.oss.sonatype.org repo index has been updated, so I updated my clone with the latest changes and reran verify with 10.0.0-RC1 as the platform.version. I'm still seeing empty dependency diagrams produced:

[INFO] Using local repository '/Users/starksm/Dev/JBoss/Jakarta/EE10/jakarta-ee-dependencies/cache/jqassistant/m2repo/repo1.maven.org/maven2' for URL 'https://repo1.maven.org/maven2' [INFO] Updating repository index, this may take a while... [INFO] Received a full update. [INFO] Updated Maven index timestamp: Tue Feb 15 13:15:26 CST 2022 [INFO] Executing query for artifacts that have been updated since Wed Dec 31 18:00:00 CST 1969. [INFO] Artifact query returned 26683939 hits (total). [INFO] Starting scan. ... [INFO] Scanning model 'jakarta.authorization:jakarta.authorization-api:pom:2.1.0-RC1'. [INFO] Processed 2000/26683939 artifacts (duration: PT10M34.338S). [INFO] Processed 2500/26683939 artifacts (duration: PT10M34.62S). [INFO] Processed 3000/26683939 artifacts (duration: PT10M34.896S). [INFO] Finished scan: 3171 artifacts (duration: PT10M40.55S). [INFO] --- jqassistant-maven-plugin:1.11.1:analyze (default-cli) @ jakarta-ee-dependencies --- [INFO] Executing analysis for 'jakarta-ee-dependencies'. [INFO] Will warn on violations starting from severity 'MINOR' [INFO] Will fail on violations starting from severity 'MAJOR'. [INFO] Reading rules from directory '/Users/starksm/Dev/JBoss/Jakarta/EE10/jakarta-ee-dependencies/jqassistant'. [INFO] Executing group 'jakarta-ee-dependencies' [INFO] Applying concept 'jakarta-ee-dependencies:ArtifactDependencies' with severity: 'MINOR'. [INFO] Applying concept 'jakarta-ee-dependencies:ArtifactLastModifiedAsString' with severity: 'MINOR'. [INFO] Applying concept 'jakarta-ee-dependencies:PlatformArtifact' with severity: 'MINOR'. [INFO] Applying concept 'jakarta-ee-dependencies:PlatformDependencyDiagram' with severity: 'MINOR'. [INFO] Rendering diagram '/Users/starksm/Dev/JBoss/Jakarta/EE10/jakarta-ee-dependencies/target/jqassistant/report/plantuml/jakarta-ee-dependencies_PlatformDependencyDiagram.svg' [INFO] Applying concept 'jakarta-ee-dependencies:PlatformDependencyVersionsByArtifact' with severity: 'MINOR'. [INFO] Applying concept 'jakarta-ee-dependencies:Specification' with severity: 'MINOR'. [INFO] Applying concept 'jakarta-ee-dependencies:SpecificationDependencyDiagram' with severity: 'MINOR'. [INFO] Rendering diagram '/Users/starksm/Dev/JBoss/Jakarta/EE10/jakarta-ee-dependencies/target/jqassistant/report/plantuml/jakarta-ee-dependencies_SpecificationDependencyDiagram.svg' [WARNING] --[ Concept Application Failure ]---------------------------------- [WARNING] Concept: jakarta-ee-dependencies:PlatformArtifact [WARNING] Severity: MINOR [WARNING] Number of rows: 0 [WARNING] Creates reports about dependencies as GraphML file and PlantUML component diagram. [WARNING]

[WARNING] [WARNING] --[ Concept Application Failure ]---------------------------------- [WARNING] Concept: jakarta-ee-dependencies:PlatformDependencyDiagram [WARNING] Severity: MINOR [WARNING] Number of rows: 0 [WARNING] Creates reports about dependencies as GraphML file and PlantUML component diagram. [WARNING]

[WARNING] [WARNING] --[ Concept Application Failure ]---------------------------------- [WARNING] Concept: jakarta-ee-dependencies:PlatformDependencyVersionsByArtifact [WARNING] Severity: MINOR [WARNING] Number of rows: 0 [WARNING] Creates a CSV report about dependency versions. [WARNING]

[WARNING] [WARNING] --[ Concept Application Failure ]---------------------------------- [WARNING] Concept: jakarta-ee-dependencies:SpecificationDependencyDiagram [WARNING] Severity: MINOR [WARNING] Number of rows: 0 [WARNING] Creates reports about dependencies between specifications as GraphML file and PlantUML component diagram. [WARNING]

[WARNING] [INFO]

[INFO] BUILD SUCCESS [INFO]

[INFO] Total time: 30:40 min [INFO] Finished at: 2022-02-22T22:28:30-06:00 [INFO]

[INFO] Closing connection to store 'file:/Users/starksm/Dev/JBoss/Jakarta/EE10/jakarta-ee-dependencies/target/jqassistant/store/'. |

— Reply to this email directly, view it on GitHub https://github.com/jqassistant-demo/jakarta-ee-dependencies/issues/1#issuecomment-1048443726, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUM254SYPT4MD2RALRSTRXLU4RQFRANCNFSM5N5IHJMA. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

starksm64 commented 2 years ago

That was the issue, I had not uncommitted out the repo.url for the jakarta.oss.sonatype.org repo. I have generated the graph for EE10.

JanWesterkamp-iJUG commented 2 years ago

It happend again! Instead of May 4th, the index points to April 14th...

[INFO] Current Maven index timestamp: Thu Apr 14 17:06:36 CEST 2022 [INFO] Updating repository index, this may take a while... [INFO] Received an incremental update. [INFO] Updated Maven index timestamp: Thu Apr 14 17:06:36 CEST 2022

I will have to knock Sonartype to rebuild the index.

JanWesterkamp-iJUG commented 2 years ago

At the moment the 10.0.0 release will not be visible to jQA.

JanWesterkamp-iJUG commented 2 years ago

Sonatype triggered the index creation manually (I created a 2nd ticket there https://issues.sonatype.org/browse/OSSRH-80749)

JanWesterkamp-iJUG commented 2 years ago

I asked Sonatype to trigger the index creation manually. I created a new ticket there https://issues.sonatype.org/browse/OSSRH-80987 to speed up things.

JanWesterkamp-iJUG commented 2 years ago

@DirkMahler Sonatype did a manual refesh of the index. It tryed it again to generate the report for version 10.0.0 of the (new) Core Profile (jakarta.platform:jakarta.jakartaee-core-api:10.0.0) to see the latest changes and saw, that the index still points to 2022-05-11 (!) and the reports are outdated. Then I deleted my local cache with a rerun showing this:

... [INFO] Using local repository '/Users/jan/git/jakarta-ee-dependencies/cache/jqassistant/m2repo/jakarta.oss.sonatype.org/content/repositories/staging' for URL 'https://jakarta.oss.sonatype.org/content/repositories/staging/' [INFO] Updating repository index, this may take a while... [INFO] Received a full update. [INFO] Updated Maven index timestamp: Wed May 11 22:08:26 CEST 2022 [INFO] Executing query for artifacts that have been updated since Thu Jan 01 01:00:00 CET 1970. [INFO] Artifact query returned 87553 hits (total). [INFO] Starting scan. ... And now updated reports! I do not understand this, because the changes where done after the 11th of May for sure ;-)

Any ideas?

DirkMahler commented 2 years ago

Just checked, current behavior is as follows:

Seems that is not an appropriate solution, taking the last index timestamp looks like being a better criteria.