Extract versionScheme, scaladocUrl, developers from pom files

[adpi2]

Motivation

The artifact page contains useful information about the pom file of an artifact.

Currently we have some useful information:

• organization and name
• project (Github repository)
• language (Java, Scala 2.12, Scala 2.13...)
• platform (JVM, Scala.js, Scala Native, sbt)
• licence
• (resolver)
• dependencies (and inverse dependencies)

See for instance https://index.scala-lang.org/typelevel/cats/artifacts/cats-core/2.9.0

But we can get more from the pom file:

• the scaladoc url:

  <properties>
  <info.apiURL>http://endpoints4s.github.io/api</info.apiURL>
  <properties>

• the version scheme

  <properties>
  <info.versionScheme>early-semver</info.versionScheme>
  </properties>

• the list of developers:

  <developers>
  <developer>
  <id>julienrf</id>
  <name>Julien Richard-Foy</name>
  <url>http://julien.richard-foy.fr</url>
  </developer>
  </developers>

As a good example of pom file you can see: https://repo1.maven.org/maven2/org/endpoints4s/algebra_2.13/1.7.0/algebra_2.13-1.7.0.pom

Identified Obstacles

Manage missing information with optional values.

Implementation Guideline

1. Add new fields in ArtifactModel: scaladocUrl and versionScheme should be optional
2. Fill those fileds in the PomConvert.apply method
3. Add corresponding types and fields in the Artifact case class: for scaladocUrl we can use the already existing Url case class, for versionScheme we can use a String, and for developers we can introduce a new case class Developer.
4. Create a migration file in this folder to add new columns in the artifacts table
5. Update the ArtifactsTable object to declare those field and read/write them. For the developers field we need a serializer. I recommend to define it using circe in the Codecs class.
6. Add some of information in the artifacts of the Values class to test that we can read and write them to the database without trouble. Those tests already exist in SqlDatabaseTests.
7. Add a scheduled job to find the missing information from the existing artifacts.

Expectations

1. We should extract the information from poms that we receive from Sonatype.
2. Reading and writing those field into the database should be tested.

[muskan3006]

Hi I would start looking into this one, is that okay?

[adpi2]

@muskan3006 Thank you for volunteering yourself on this issue. Don't hesitate to ask any question here if you miss some pieces of information.

[muskan3006]

sure