Open milesmatthias opened 5 years ago
Relevant: https://github.com/bazelbuild/migration-tooling/issues/82#issuecomment-416423649
If I figure this out, I'll submit a PR to improve the docs.
This is a good question and the docs should be clearer.
Generally this tool is not focused on the migration problem. There is some support:
Running that tool pointing at a pom.xml should set you up to some degree. The problem is we don't migrate often, but we do update dependencies a lot, so the maintenance of the dependencies.yaml and running to resolve dependencies is much MUCH better exercised.
We should put the ability to run this tool in the main command.
That's helpful, thanks. So I took the pom.xml from the example repo I mentioned, and tried to run that tool:
→ bazel run //src/scala/com/github/johnynek/bazel_deps/maven:maven_tool /Users/mmatthias/java-server/pom.xml
INFO: Analysed target //src/scala/com/github/johnynek/bazel_deps/maven:maven_tool (0 packages loaded).
INFO: Found 1 target...
Target //src/scala/com/github/johnynek/bazel_deps/maven:maven_tool up-to-date:
bazel-bin/src/scala/com/github/johnynek/bazel_deps/maven/maven_tool
bazel-bin/src/scala/com/github/johnynek/bazel_deps/maven/maven_tool.jar
INFO: Elapsed time: 0.171s, Critical Path: 0.00s
INFO: 0 processes.
INFO: Build completed successfully, 1 total action
INFO: Build completed successfully, 1 total action
Exception in thread "main" java.lang.RuntimeException: missing node!
at scala.sys.package$.error(package.scala:27)
at com.github.johnynek.bazel_deps.maven.Tool$.singleNode(Tool.scala:15)
at com.github.johnynek.bazel_deps.maven.Tool$.singleText(Tool.scala:20)
at com.github.johnynek.bazel_deps.maven.Tool$.parse(Tool.scala:111)
at com.github.johnynek.bazel_deps.maven.Tool$.main(Tool.scala:339)
at com.github.johnynek.bazel_deps.maven.Tool.main(Tool.scala)
full contents of pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.springframework</groupId>
<artifactId>gs-rest-service</artifactId>
<version>0.1.0</version>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.0.5.RELEASE</version>
</parent>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.jayway.jsonpath</groupId>
<artifactId>json-path</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<properties>
<java.version>1.8</java.version>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
<repositories>
<repository>
<id>spring-releases</id>
<url>https://repo.spring.io/libs-release</url>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>spring-releases</id>
<url>https://repo.spring.io/libs-release</url>
</pluginRepository>
</pluginRepositories>
</project>
Looks like this error is thrown if any dependency is missing any of the keys listed here: https://github.com/johnynek/bazel-deps/blob/master/src/scala/com/github/johnynek/bazel_deps/maven/Tool.scala#L92
However, when adding in all those requires keys with dummy data, I'm running into https://github.com/johnynek/bazel-deps/issues/55
EDIT: I'm getting the warning listed in #55, but it still worked.
Steps I took to generate a dependencies.yaml
and (almost) get bazel-deps working:
bazel run bazel run //src/scala/com/github/johnynek/bazel_deps/maven:maven_tool {path-to-pom-xml}
. A dependencies.yaml will be written next to your pom.xml file.set()
was not defined)options
key with a child of resolverType: 'coursier'
. See example. See #224 for context.bazel run //:parse -- generate -r {path-to-your-project-where-dependencies.yaml-lives} -s 3rdparty/workspace.bzl -d dependencies.yaml
, which reads your depedencies.yaml file, not bazel-deps', and generates a 3rd party directory in your project.Note that when you run step 2, you need the keys listed in step 1 (opened #225), but when you run step 4, you'll want to remove the unnecessary key values from your dependencies.yaml and make sure there's a version number that matches in https://mvnrepository.com central.
Also, if you get any errors when running step 4 because of something like "${os.detected.classifier}", this tool doesn't resolve tokens like that (opened #224).
I promise I'll submit a PR to improve the README with my detailed instructions from above when I get this entire flow working.
@johnynek current issue when I run my project's bazel build and try to include the maven dependencies:
→ make create
ERROR: /Users/mmatthias/demo/java-server/BUILD.bazel:15:1: Traceback (most recent call last):
File "/Users/mmatthias/demo/java-server/BUILD.bazel", line 15
maven_dependencies()
File "/Users/mmatthias/demo/java-server/3rdparty/workspace.bzl", line 131, in maven_dependencies
callback(hash)
File "/Users/mmatthias/demo/java-server/3rdparty/workspace.bzl", line 58, in callback
jar_artifact(artifact = hash["artifact"], name ..."], <4 more arguments>)
jar_artifact must be in the WORKSPACE file (used by //java-server:ch_qos_logback_logback_classic)
ERROR: error loading package 'java-server': Package 'java-server' contains errors
INFO: Elapsed time: 0.109s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
FAILED: Build did NOT complete successfully (0 packages loaded)
make: *** [create] Error 1
any ideas?
Hello, I am running into the same problem, any update on this?
@3nigm4 The problem being that you don't know how to generate the dependencies.yaml
from your pom.xml
, or the problem that when run the conversion program it doesn't work?
@thundergolfer the former -> that I don't know how to generate the dependencies.yaml from my pom.xml
@3nigm4 Yeah it isn't particularly clear. It's 11PM where I am now and I need to get to bed, but I'll pop back in here in the morning to get you a step-by-step.
@3nigm4 did you read my steps above? That's what I did.
@3nigm4 Yeah I'd forgotten that Miles has a pretty clear step by step above. The only problem is that the program you run in step 2 is likely to produce a dependencies.yaml
file that is invalid and can't be processed by the bazel run //:parse ...
step.
You can then manually clean the dependencies.yaml
file like Miles described.
At our company we've written a Python script that does the work of step 2 without the bugs. Unfortunately this is not of much help to you ☹️.
@3nigm4 did you read my steps above? That's what I did.
@milesmatthias --> great step by step write up !!
I am trying this for a Micronaut project that I have, however when I run step 2 I get the following error. I do not see any dependencies.yaml
file getting generated as well.
bazel run //src/scala/com/github/johnynek/bazel_deps/maven:maven_tool {path_to_project_directory}/pom_dummy.xml
DEBUG: Rule 'io_bazel_rules_scala' modified arguments {"shallow_since": "1551725653 -1000"}
INFO: Analysed target //src/scala/com/github/johnynek/bazel_deps/maven:maven_tool (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //src/scala/com/github/johnynek/bazel_deps/maven:maven_tool up-to-date:
bazel-bin/src/scala/com/github/johnynek/bazel_deps/maven/maven_tool
bazel-bin/src/scala/com/github/johnynek/bazel_deps/maven/maven_tool.jar
INFO: Elapsed time: 0.150s, Critical Path: 0.00s
INFO: 0 processes.
INFO: Build completed successfully, 1 total action
INFO: Build completed successfully, 1 total action
Exception in thread "main" java.lang.RuntimeException: missing node!
at scala.sys.package$.error(package.scala:27)
at com.github.johnynek.bazel_deps.maven.Tool$.singleNode(Tool.scala:15)
at com.github.johnynek.bazel_deps.maven.Tool$.singleText(Tool.scala:20)
at com.github.johnynek.bazel_deps.maven.Tool$.parse(Tool.scala:111)
at com.github.johnynek.bazel_deps.maven.Tool$.main(Tool.scala:341)
at com.github.johnynek.bazel_deps.maven.Tool.main(Tool.scala)
Here is the copy of the POM file that I am trying to use to generate the Dependencies.yaml
file.
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>test.service</groupId>
<artifactId>test.service</artifactId>
<version>0.1</version>
<properties>
<micronaut.version>1.0.5</micronaut.version>
<jdk.version>11</jdk.version>
<lombok.version>1.18.6</lombok.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<exec.mainClass>test.service.Application</exec.mainClass>
</properties>
<repositories>
<repository>
<id>jcenter.bintray.com</id>
<url>https://jcenter.bintray.com</url>
</repository>
</repositories>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>io.micronaut</groupId>
<artifactId>micronaut-bom</artifactId>
<version>${micronaut.version}</version>
<type>pom</type>
<scope>import</scope>
<classifier>dummy</classifier>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>${lombok.version}</version>
<scope>provided</scope>
<type>dummy</type>
<classifier>dummy</classifier>
</dependency>
<dependency>
<groupId>com.auth0</groupId>
<artifactId>java-jwt</artifactId>
<version>3.8.0</version>
<scope>dummy</scope>
<type>dummy</type>
<classifier>dummy</classifier>
</dependency>
<dependency>
<groupId>io.micronaut</groupId>
<artifactId>micronaut-tracing</artifactId>
<version>${micronaut.version}</version>
<scope>compile</scope>
<type>dummy</type>
<classifier>dummy</classifier>
</dependency>
<dependency>
<groupId>javax.annotation</groupId>
<artifactId>javax.annotation-api</artifactId>
<version>${micronaut.version}</version>
<scope>compile</scope>
<type>dummy</type>
<classifier>dummy</classifier>
</dependency>
<dependency>
<groupId>io.swagger.core.v3</groupId>
<artifactId>swagger-annotations</artifactId>
<version>${micronaut.version}</version>
<scope>compile</scope>
<type>dummy</type>
<classifier>dummy</classifier>
</dependency>
<dependency>
<groupId>io.micronaut</groupId>
<artifactId>micronaut-http-client</artifactId>
<version>${micronaut.version}</version>
<scope>compile</scope>
<type>dummy</type>
<classifier>dummy</classifier>
</dependency>
<dependency>
<groupId>io.micronaut.configuration</groupId>
<artifactId>micronaut-micrometer-core</artifactId>
<version>${micronaut.version}</version>
<scope>compile</scope>
<type>dummy</type>
<classifier>dummy</classifier>
</dependency>
<dependency>
<groupId>io.micronaut.configuration</groupId>
<artifactId>micronaut-micrometer-registry-prometheus</artifactId>
<version>${micronaut.version}</version>
<scope>compile</scope>
<type>dummy</type>
<classifier>dummy</classifier>
</dependency>
<dependency>
<groupId>io.micronaut</groupId>
<artifactId>micronaut-inject</artifactId>
<version>${micronaut.version}</version>
<scope>compile</scope>
<type>dummy</type>
<classifier>dummy</classifier>
</dependency>
<dependency>
<groupId>io.micronaut</groupId>
<artifactId>micronaut-validation</artifactId>
<version>${micronaut.version}</version>
<scope>compile</scope>
<type>dummy</type>
<classifier>dummy</classifier>
</dependency>
<dependency>
<groupId>io.micronaut</groupId>
<artifactId>micronaut-runtime</artifactId>
<version>${micronaut.version}</version>
<scope>compile</scope>
<type>dummy</type>
<classifier>dummy</classifier>
</dependency>
<dependency>
<groupId>io.micronaut</groupId>
<artifactId>micronaut-http-server-netty</artifactId>
<version>${micronaut.version}</version>
<scope>compile</scope>
<type>dummy</type>
<classifier>dummy</classifier>
</dependency>
<dependency>
<groupId>io.micronaut</groupId>
<artifactId>micronaut-management</artifactId>
<version>${micronaut.version}</version>
<scope>compile</scope>
<type>dummy</type>
<classifier>dummy</classifier>
</dependency>
<dependency>
<groupId>io.micronaut</groupId>
<artifactId>micronaut-inject-java</artifactId>
<version>${micronaut.version}</version>
<scope>compile</scope>
<type>dummy</type>
<classifier>dummy</classifier>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<version>1.2.3</version>
<scope>runtime</scope>
<type>dummy</type>
<classifier>dummy</classifier>
</dependency>
<dependency>
<groupId>io.jaegertracing</groupId>
<artifactId>jaeger-thrift</artifactId>
<version>${micronaut.version}</version>
<scope>runtime</scope>
<type>dummy</type>
<classifier>dummy</classifier>
</dependency>
<dependency>
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy</artifactId>
<version>2.4.12</version>
<scope>dummy</scope>
<type>dummy</type>
<classifier>dummy</classifier>
</dependency>
<dependency>
<groupId>org.spockframework</groupId>
<artifactId>spock-core</artifactId>
<version>${micronaut.version}</version>
<scope>test</scope>
<type>dummy</type>
<classifier>dummy</classifier>
<exclusions>
<exclusion>
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy-all</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>io.micronaut</groupId>
<artifactId>micronaut-inject-groovy</artifactId>
<version>${micronaut.version}</version>
<scope>test</scope>
<type>dummy</type>
<classifier>dummy</classifier>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
<type>dummy</type>
<classifier>dummy</classifier>
</dependency>
<dependency>
<groupId>org.hamcrest</groupId>
<artifactId>hamcrest-all</artifactId>
<version>1.3</version>
<scope>test</scope>
<type>dummy</type>
<classifier>dummy</classifier>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.1.0</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<transformers>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>${exec.mainClass}</mainClass>
</transformer>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.6.0</version>
<configuration>
<executable>java</executable>
<arguments>
<argument>-noverify</argument>
<argument>-XX:TieredStopAtLevel=1</argument>
<argument>-classpath</argument>
<classpath/>
<argument>${exec.mainClass}</argument>
</arguments>
</configuration>
</plugin>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.0</version>
<configuration>
<annotationProcessorPaths>
<path>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>${lombok.version}</version>
</path>
<path>
<groupId>io.micronaut.configuration</groupId>
<artifactId>micronaut-openapi</artifactId>
<version>${micronaut.version}</version>
</path>
<path>
<groupId>io.micronaut</groupId>
<artifactId>micronaut-inject-java</artifactId>
<version>${micronaut.version}</version>
</path>
<path>
<groupId>io.micronaut</groupId>
<artifactId>micronaut-validation</artifactId>
<version>${micronaut.version}</version>
</path>
</annotationProcessorPaths>
</configuration>
<executions>
<execution>
<id>test-compile</id>
<phase>process-test-sources</phase>
<goals>
<goal>testCompile</goal>
</goals>
<configuration>
<annotationProcessorPaths>
<path>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>${lombok.version}</version>
</path>
<path>
<groupId>io.micronaut.configuration</groupId>
<artifactId>micronaut-openapi</artifactId>
<version>${micronaut.version}</version>
</path>
<path>
<groupId>io.micronaut</groupId>
<artifactId>micronaut-inject-java</artifactId>
<version>${micronaut.version}</version>
</path>
<path>
<groupId>io.micronaut</groupId>
<artifactId>micronaut-validation</artifactId>
<version>${micronaut.version}</version>
</path>
</annotationProcessorPaths>
</configuration>
</execution>
</executions>
<dependencies>
<!-- lombok.launch.Agent above is from this plugin -->
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok-maven-plugin</artifactId>
<version>1.18.6.0</version>
</dependency>
<dependency>
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy-eclipse-compiler</artifactId>
<version>3.3.0-01</version>
</dependency>
<dependency>
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy-eclipse-batch</artifactId>
<version>2.5.6-02</version>
</dependency>
</dependencies>
</plugin>
<plugin>
<groupId>com.google.cloud.tools</groupId>
<artifactId>jib-maven-plugin</artifactId>
<version>1.0.2</version>
<configuration>
<to>
<image>myTestImage</image>
</to>
</configuration>
</plugin>
<plugin>
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy-eclipse-compiler</artifactId>
<version>3.3.0-01</version>
<extensions>true</extensions>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.1</version>
<configuration>
<useFile>false</useFile>
<includes>
<include>**/*Test.java</include>
<include>**/*Spec.java</include>
</includes>
</configuration>
</plugin>
<!-- <plugin>
<groupId>org.codehaus.gmavenplus</groupId>
<artifactId>gmavenplus-plugin</artifactId>
<version>1.6.3</version>
<executions>
<execution>
<goals>
<goal>addTestSources</goal>
<goal>addSources</goal>
<goal>compileTests</goal>
<goal>compile</goal>
</goals>
</execution>
</executions>
</plugin> -->
</plugins>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.0</version>
<configuration>
<source>${jdk.version}</source>
<target>${jdk.version}</target>
<encoding>UTF-8</encoding>
<compilerArgs>
<arg>-parameters</arg>
</compilerArgs>
<annotationProcessorPaths>
<path>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>${lombok.version}</version>
</path>
<path>
<groupId>io.micronaut.configuration</groupId>
<artifactId>micronaut-openapi</artifactId>
<version>${micronaut.version}</version>
</path>
<path>
<groupId>io.micronaut</groupId>
<artifactId>micronaut-inject-java</artifactId>
<version>${micronaut.version}</version>
</path>
<path>
<groupId>io.micronaut</groupId>
<artifactId>micronaut-validation</artifactId>
<version>${micronaut.version}</version>
</path>
</annotationProcessorPaths>
</configuration>
<executions>
<execution>
<id>test-compile</id>
<goals>
<goal>testCompile</goal>
</goals>
<configuration>
<compilerArgs>
<arg>-parameters</arg>
</compilerArgs>
<annotationProcessorPaths>
<path>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>${lombok.version}</version>
</path>
<path>
<groupId>io.micronaut.configuration</groupId>
<artifactId>micronaut-openapi</artifactId>
<version>${micronaut.version}</version>
</path>
<path>
<groupId>io.micronaut</groupId>
<artifactId>micronaut-inject-java</artifactId>
<version>${micronaut.version}</version>
</path>
<path>
<groupId>io.micronaut</groupId>
<artifactId>micronaut-validation</artifactId>
<version>${micronaut.version}</version>
</path>
</annotationProcessorPaths>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.codehaus.gmavenplus</groupId>
<artifactId>gmavenplus-plugin</artifactId>
<version>1.6.2</version>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>compileTests</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</pluginManagement>
</build>
<pluginRepositories>
<pluginRepository>
<id>bintray</id>
<name>Groovy Bintray</name>
<url>https://dl.bintray.com/groovy/maven</url>
<releases>
<updatePolicy>never</updatePolicy>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</pluginRepository>
</pluginRepositories>
</project>
Any ideas as to what could be going wrong?
@venkyvb You need to check this out -> https://github.com/johnynek/bazel-deps/issues/223#issuecomment-439068888
Hey guys, thanks for your support and let me change topic a little bit.,,
When testing new technology a typical workflow for me is to go to start.spring.io create a new project and see how the new technology integrates with your usual tools, leaving the happy path just a little.
If the readme contained a simple example of how to get going with a very basic project, this would have really helped me (and maybe others) figuring out the rest on their own...
Don't you love it aswell when maintainers put something like that in the readme?
As I don't fully understand the technology yet this may be too much to ask, I know this is rapidly changing tech, bazel supports multiple languages and there may be better use of your valuable time...
So please just take it as feedback on what would help me as a new user. And some basic instructions of how to get going really would!
This is definitely a major n00b question, but if I have a very basic Java Spring app that has a maven pom.xml (for example, see super basic https://github.com/spring-guides/gs-rest-service/tree/master/complete), how do I use bazel-deps to generate the bazel rules to analyze the dependency (spring boot in the example case)?
The usage on the readme talks about a dependencies.yaml. Is this file generated from the command? Or do I need to use something like https://github.com/takari/polyglot-maven to translate the pom.xml to dependencies.yaml?
Thanks!