Gradle build

[Arcnor]

Is this considered/going to be considered?

I think it's the fastest way of getting LWJGL3 to be used by other projects without the current LWJGL2 hassle of generating POMs separated from the main build. It can be used to compile without ant at all, or it can be mixed.

I started adding support and got the kotlin plugin to compile all the sources, but got confused about the Generator.kt

If you're not planning on adding this, please close as wont fix so I don't spend any more time on it :D

[big-guy]

@dhild @Arcnor I'm one of the native support maintainers for Gradle, would you mind if I pitch in a bit?

[Arcnor]

@big-guy even if you were not somebody that useful (that's awesome btw :smile:) no need to ask for permission, all the help we can get is welcome!

[big-guy]

Is https://github.com/dhild/lwjgl3/blob/gradle_project/build.gradle the latest?

[Arcnor]

Yes. Main problem was passing the parameters for OSX to run on the first thread, but fix what you think is wrong :P. Also, not sure if it works after the modules work @Spasi did, but @dhild knows more.

[Spasi]

I didn't want to waste any more time, so did the 3.0.0a release yesterday. Today I started working on the Gradle build, using @dhild's script as reference, but writing everything from scratch. I'd like to take the time to understand every little detail, to avoid wasting time later when I'll have to maintain it. Using Gradle 2.4-rc-1 for the parallel native compilation, LWJGL has tons of native files and I personally can't work without parallel builds.

From the reading I've done, it looks like the best approach would be to refactor the existing modules (templates, generated, core, util) as Gradle (sub)projects. Would that be a good idea?

Also, I'd like to keep the project root clean. All modules, except generated, are currently in /src. Would it be a problem if I made a /modules folder and put all sub-projects under there? Or do they really have to be directly under the root project?

[Arcnor]

That's awesome, congrats for the release.

As for your question, you can really put things wherever you want, so putting everything under modules will work perfectly fine. You will just need to reference it by doing modules/myModule and that's it. You can also have multiple build.gradle, so having one inside the modules directory will also be a good idea to keep everything more clean.

You can check any LibGDX project for a simple example if you want

[dhild]

@Spasi Very cool, release done!

The modules subdirectory sounds like a great idea.

There was one other idea I'd like to at least throw out for you to consider: splitting the modules along functionality lines, rather than by core, native, util. I tried this out, got it mostly working (just needed some more native build attention): https://github.com/dhild/lwjgl3/tree/source_reorganization

[Spasi]

I have just pushed the gradle branch. For now, the only changes are the new folder structure and the updated Ant scripts & IDEA project. Also clarified terminology a bit; the old "binding modules" are now just the "bindings" and "modules" are the compilation modules (as understood in Gradle or IDEA).

I would really appreciate any comments or suggestions on the new folder structure. It's a bit painful to refactor, so I'd like to make sure I get it right before pushing to master. If anyone has time to review the changes, please do let me know if you can think of any improvements.

I'll try to add the actual gradle scripts soon.

[huhlig]

If you are going to stick with Ant, why not just use ivy.

http://stackoverflow.com/questions/5111831/how-to-publish-3rdparty-artifacts-with-ivy-and-nexus

[upsonp]

I know this has been dead for quite some time other than tangentially mentioned from other topics, but...

Has anyone considered using Kotlin to create the Gradle build script. I was brushing up on my technologies because I'd like to make more use of LWJGL but there are a lot of things I don't understand and I get a lot of errors, which I don't know if they're me, my set up or the LWJGL itself. One of my biggest issues at the moment is just getting through the Ant build on my Windows machine. I had no issues on my personal Linux machine and had a few minutes today so I thought I'd check out my eclipse .classpath update on my work machine #373. Complete failure on the compile-templates over a Java Stack Heap issue.

I was doing a lot of reading into Gradle, which is what some people in my shop are using and I stumbled on Gradle with Kotlin. I know Kotlin is being used in LWJGL pretty heavily at the moment and at first I was like, "Oh great yet another language", but after spending some time watching conferences and tutorials on it, I'm quite excited. One of the conferences I watched was on how Gradle will be supporting Kotlin (that was in 2016, seems to be a done deal now). I pointed the Devs in my shop doing Gradle builds to it and they seem pretty interested in it as well.

Not promising I'm going to do it, mainly because I'm new to Kotlin and only vaguely acquainted with Gradle, but I might consider doing more research into and playing around more with it if I/it might be of use somewhere.

[Spasi]

Complete failure on the compile-templates over a Java Stack Heap issue.

I've only seen kotlinc failing when there is not enough memory available. Could you post more details about the error you're seeing?

Has anyone considered using Kotlin to create the Gradle build script.

I've been waiting for two features: a) incremental Kotlin compilation in Gradle builds and b) mature build.gradle.kts support. The former has been available for some time, the latter is getting there.

This remains a low priority issue though. The project won't gain anything from moving to Gradle and there are more pressing matters (e.g. ARM/Android support).

[upsonp]

From Running Ant Init

Buildfile: E:\_JBuilder\Personal\lwjgl3\build.xml
bindings:
init:
check-dependencies:
  [kotlinc] E:\_JBuilder\Personal\lwjgl3\bin\libs\kotlinc\build.txt doesn't exist
update-dependencies:
    [mkdir] Created dir: E:\_JBuilder\Personal\lwjgl3\bin\libs
    [mkdir] Created dir: E:\_JBuilder\Personal\lwjgl3\bin\libs\java
-lib-download:
-kotlinc-download:
  [kotlinc] Getting: https://github.com/JetBrains/kotlin/releases/download/v1.2.20/kotlin-compiler-1.2.20.zip
  [kotlinc] To: E:\_JBuilder\Personal\lwjgl3\bin\libs\kotlin-compiler-1.2.20.zip
  [kotlinc] https://github.com/JetBrains/kotlin/releases/download/v1.2.20/kotlin-compiler-1.2.20.zip moved to https://github-production-release-asset-2e65be.s3.amazonaws.com/3432266/83bc653a-fb8b-11e7-86d0-cb2dc1dc4b30?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20180215%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20180215T193417Z&X-Amz-Expires=300&X-Amz-Signature=3ca28f5affa86bd389a68657c87c1765b1adf410907e5277d30f1addddbcdf36&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dkotlin-compiler-1.2.20.zip&response-content-type=application%2Foctet-stream
  [kotlinc] ....................................................
.
.
.
  [kotlinc] ..........................
  [kotlinc] Expanding: E:\_JBuilder\Personal\lwjgl3\bin\libs\kotlin-compiler-1.2.20.zip into E:\_JBuilder\Personal\lwjgl3\bin\libs
  [kotlinc] Deleting: E:\_JBuilder\Personal\lwjgl3\bin\libs\kotlin-compiler-1.2.20.zip
  [kotlinc] The Kotlin compiler was updated to build: 1.2.20
BUILD SUCCESSFUL
Total time: 1 minute 25 seconds

From running compile-templates

Buildfile: E:\_JBuilder\Personal\lwjgl3\build.xml
bindings:
init:
check-dependencies:
-compile-generator:
[Generator] Compiling Kotlin generator...
  [kotlinc] Compiling [E:\_JBuilder\Personal\lwjgl3\modules\generator\src\main\kotlin] => [E:\_JBuilder\Personal\lwjgl3\bin\classes\generator]
  [kotlinc] info: kotlinc-jvm 1.2.20 (JRE 1.8.0_111-b14)
  [kotlinc] info: PERF: INIT: Compiler initialized in 1145 ms
  [kotlinc] info: PERF: ANALYZE: 18 files (8561 lines) in 8258 ms - 1036.692 loc/s
  [kotlinc] info: PERF: GENERATE: 18 files (8561 lines) in 8143 ms - 1051.332 loc/s
  [kotlinc] info: PERF: GC time for Copy is 592 ms
  [kotlinc] info: PERF: GC time for MarkSweepCompact is 218 ms
  [kotlinc] info: PERF: JIT time is 2102 ms
  [kotlinc] info: PERF: Find Java class performed 17 times, total time 3 ms
  [kotlinc] info: PERF: Type info performed 28505 times, total time 5967 ms
  [kotlinc] info: PERF: Call resolve performed 16207 times, total time 4556 ms
  [kotlinc] info: PERF: Binary class from Kotlin file performed 263 times, total time 171 ms
    [touch] Creating E:\_JBuilder\Personal\lwjgl3\bin\classes\generator\touch.txt
[javac: Generator Tools & Doclets] Compiling 7 source files to E:\_JBuilder\Personal\lwjgl3\bin\classes\generator
compile-templates:
[Templates] Compiling Kotlin templates. This will take 1-2 minutes...
    [mkdir] Created dir: E:\_JBuilder\Personal\lwjgl3\bin\classes\templates
  [kotlinc] Compiling [E:\_JBuilder\Personal\lwjgl3\modules\lwjgl\core\src\main\kotlin\core\dyncall, ...] => [E:\_JBuilder\Personal\lwjgl3\bin\classes\templates]
  [kotlinc] info: kotlinc-jvm 1.2.20 (JRE 1.8.0_111-b14)
  [kotlinc] ERROR: Exception while analyzing expression at (371,11) in E:/_JBuilder/Personal/lwjgl3/modules/lwjgl/opengl/src/main/kotlin/opengl/templates/GL45.kt
.
.
.
  [kotlinc] Caused by: java.lang.OutOfMemoryError: Java heap space
.
.
.
  [kotlinc] Caused by: java.lang.OutOfMemoryError: Java heap space
.
.
.

BUILD FAILED
E:\_JBuilder\Personal\lwjgl3\build.xml:255: Compile failed; see the compiler error output for details.

Total time: 2 minutes 1 second

Edit: cut out the stack trace

[Spasi]

From running compile-templates ... Caused by: java.lang.OutOfMemoryError: Java heap space

Try set ANT_OPTS=-Xmx1G before running ant. Or use a 64bit JVM.

[upsonp]

I gave up trying to run Ant from the command line.

1. I checked out the project fresh and started by copying the eclipse .classpath file to the root.
2. Added lwjgl3 as a project.
3. Then I created a "Run External Tool" under the "External Tools Configuration" dialog From there it was a piece of cake.
4. On the Targets tab I selected (Clean, init, compile-templates, compile-tests) and made sure that was the order they'd execute in.
5. On the JRE tab I added '-Xmx1G', at @Spasi suggestion.
6. Clicked Run and got a full build, no issues.
7. I updated the Class path for the tools.jar file
8. Clicked on the project and hit F5 for a refresh. (Then waited for a workspace rebuild)

It looks like it completed, but I'm missing the lwjgl.dll file though. I assume that was suppose to go in the bin/libs folder the same as the Linux *.so files.

Edit: Should also mention I did set ANT_OPTS and switched my default JDK to the 64-bit 1.8.0_161 install before trying the eclipse ant interface.

I got a bunch of errors like these

Buildfile: E:\_JBuilder\Personal\lwjgl3\build.xml

bindings:

init:

check-dependencies:

-compile-generator:
[javac: Generator Tools & Doclets] Compiling 7 source files to E:\_JBuilder\Personal\lwjgl3\bin\classes\generator
[javac: Generator Tools & Doclets] E:\_JBuilder\Personal\lwjgl3\modules\generator\src\main\java\org\lwjgl\system\ExcludeDoclet.java:7: error: package com.sun.javadoc does not exist
[javac: Generator Tools & Doclets] import com.sun.javadoc.*;
[javac: Generator Tools & Doclets] ^

Edit 2: I started looking at issue #349, just because, and I now realize the errors being generated on my Windows machine from the command line may be related to the deprecation of com.sun.javadoc.* Which if I'd actually read the error I might have made the connection sooner. No idea why I'd get them from the command line and not the Eclipse Ant plug-in. I made sure both were using the same JVM and Ant version.

[Spasi]

I got a bunch of errors like these

Is the JAVA_HOME environment variable set to the JDK 8 folder?

but I'm missing the lwjgl.dll file

Native compilation happens with ant compile-native or (demo or tests that depend on it). You'll need Visual Studio Community and a properly configured toolchain (usually running vcvarsall amd64_x86 is all you need).

[upsonp]

JAVA_HOME is set, I think it might be pointing to an Oracle DB or ArcGIS or [something else] install of the JDK though.

My work machine is sometimes locked down in terms of what I can install (depends on what my group policy permissions has decided is ok that day) and I don't have access to another windows box to develop on. So I'll have to see if I have permissions to install Visual Studio.

I appreciate the help, sometimes it takes me awhile to get familiarized with project and it's tools, but if there's some way I can give back, I'd like to help out.

[Spasi]

Gradle 5 has been released and the Kotlin DSL is now considered production-ready. The LWJGL codebase structure has also stabilized, so now is a good time to attempt a migration to Gradle.

[TheMrMilchmann]

I have already experimented with porting the build process to Gradle in the past and had a look at it again quite recently. I might publish my progress later this week to give you an idea of what it could look like.

However, one issue that's stopping me from cleanly transitioning to Gradle remains: There are circular dependencies between templates of some modules (e.g. OpenGL and OpenCL). While it is possible to compile all templates together this eliminates some of Gradle's benefits and I'd rather see those circular dependencies removed (if this is reasonable without too much duplication).

[Spasi]

There are circular dependencies between templates of some modules (e.g. OpenGL and OpenCL)

The cycle is now broken with 7657b20bd19e9e2cf0ecccc571baec37c9c22786. There are still one-way dependencies between modules (e.g. GLFW depends on the EGL, GL and Vulkan modules), but that shouldn't be a problem, right?

[elect86]

So, I picked up this (again) a couple of weeks ago and ported the most important ant task to gradle tasks. I didn't use Leon's work because of outdate (also Gradle gets developed super fast), but I took inspiration nonetheless.

The current work is here and right now only Linux is considered somehow complete. I'm working on Windows at the moment, but for MacOS I'd need someone helps..

Features:

• Tasks are fully integrated in the Gradle model, with outputs and inputs
• Is there additionally a way to extend this optimization (skipping on specific conditions) even further but has to be polished
• mutable states are shared via Build Service
• DSL for compiling and interacting with java, javac and kotlinc and the native build process

There are still tons of stuff to polish and tune (move to multithreading/parallel execution/coroutines), but I'd like to start gathering whatever feedback/hint/ideas I can, since there is a lot of time ahead yet

Ps: I guess the best would have been something more "Gradlesque" way, by moving all the templates in a module, the generator as well and having Gradle build them while setting up all the proper dependencies, but Spasi is used to have all the relevant code in one close space (c, java and generated) and so he would like to continue. I'm not sure if that design would have worked with the Spasi wish, but that's something to play with for the next time :)

[elect86]

I pushed another commit, the whole linux workflow can be considered complete now