Maybe someone could edit the tutorial / incorporate the below somehow to help others trying to get the project to work.
This is a very useful tutorial and I was grateful to come across it. However, it took me a long time to get it working. Even after I worked through gradle dependency issues, I kept running into the errors 'JDK 17.0.6 isn't compatible with Gradle 7.6.1. Please fix JAVA_HOME environment variable' or - roughly - 'The consumer was configured to find a runtime of a library compatible with Java 12, packaged as a jar, and its dependencies declared externally, as well as attribute 'org.gradle.plugin.api-version' with value '7.6.1' but:' (and the error messages ended like that with nothing after the 'but:'! :-) )
Under Windows 11 (IntelliJ IDEA 2021.2.4 (Community edition), the tutorial does not build / will not work as shown. Maybe it's a little out of date or for a different version of IDEA or only works on Linux or whatever. In any case, I think it could do with a bit more information and clearer instruction.
The problems for me were different file syntax, the unclear / misleading instructions and java / SDK version differences; I think it makes a lot of assumptions about prior knowledge or system setup but does not seem to state these assumptions. In case it's helpful to anyone else, I got it working using the following. I had Zookeeper and Kafka already installed and working so I didn't bother with that part of the tutorial and cannot say whether it's accurate or not. I also knew automatically how to adjust the various Zookeeper and Kafka command line tools for Windows vs Linux and the location of my installations so I'm not covering them here either.
If you do all the other stuff right, you WON'T NEED TO TYPE THESE DEPENDENCIES IN ANYWHERE BECUASE THEY WILL BE DONE FOR YOU WHEN YOU GENERATE THE SPRINGBOOT PROJECT. The tutorial doesn't actually say to add the dependencies to the build.gradle file. But I feel this is implied since it gives you a config XML code snippet under the 'add dependencies' instruction. At that time in the tutorial though, I found myself confused because there was nowhere on the relevant screen to type in the snippet (see later for that). For now, only FYI, the actual build.gradle dependencies (using IntelliJ IDEA and windows 11) are not formatted as documented in the tutorial. Instead of looking as shown in the tutorial, they look like this:
BUT YOU DON'T CARE ABOUT THAT AT THIS STAGE. Read the rest of this first and hopefully you won't have to come back here.
JAVA versions: As JAVA 20 JDK is now out and gradle doesn't - as of April 2023 when I wrote this - run under it, it's important to ensure you are using a supported version of JAVA which is the same across your operating system, IntelliJ IDEA and for gradle within IDEA. But the Springboot project generation page (see later) offers JAVA 20 as an option. So remember NOT to choose it (but like I said, see later for that). I got it working with JAVA / JDK 17.0.6. My %JAVA_HOME% environment variable points to C:\Program Files\Java\jdk-17. Before running IntelliJ IDEA, I also added %JAVA_HOME%\bin at the front of my windows %PATH% environment variable so the first java.exe and javac.exe executables in my %PATH% are the correct versions (I know, I know, it's rookie stuff but it makes a difference if the right JDK isn't being used both in the OS environment when you crank up IDEA and inside IDEA itself). Setting IDEA and gradle to point at the correct JDK alone, are not enough. In fact, I found having IDEA fire up under the wrong JAVA version but with the right JAVA version in its settings, lead to counter-intuitive error messages whose causes did not point to the root cause.
After pointing %JAVA_HOME% and %PATH% to the right places, I also ensured IDEA and gradle were pointing at the same JDK as windows inside IDEA itself. This involves setting the SDK in two places, the 'Project structure' and the 'Gradle JVM'.
2.1. Project Structure: Under my installation of IDEA, this meant navigating to File->Project Structure->Project Setting->Project and choosing JDK 17.0.6 from the 'detected SDKs' dropdown. I also chose language level 16 (randomly) but others may work (I didn't experiment).
2.2. 'Gradle JVM': Gradle also needs to be pointed at the correct JDK. For my installation of Intellij IDEA (Windows 11), this meant navigating to File->Settings->Tools->Build, Execution, Deployment->Gradle then in the Gradle JVM dropdown, ensuring JAVA_HOME (for me with version 17.0.6 beside it) was selected.
Unclear instructions: If you're a JAVA programmer like me, you'll be down with file system structure and package naming (the kind of way, they're strictly the same and all that jazz). You can deduce the package names and .java file names from the code snippets provided by the author (as I did) but it's not as clear as it could be in the tutorial. Also, if you create the .java (source code) files in the order shown, you'll at first see a lot of compile warnings in your code editor because you create code that depends (for example) on the AppConstants.java class before creating the AppConstants.java class itself. It does resolve as you work through but.., well, it's a confusing order to do it in. To make the process less confusing and to not upset you with too many scary red compile errors at coding time, I recommend doing the following in the following order BEFORE following the coding part of the tutorial through in the order shown (and bear in mind this is all AFTER the below part about going to the Springboot page and generating the project and downloading it and all that stuff in my step 4 and its subordinate steps, sorry my notes are out of order but at least I'm telling you that):
3.1. First create the following two packages and the net.javaguides.springbootkafka.utils.AppConstant.java (as coded in the tutorial) in your IDEA Gradle project:
net.javaguides.springbootkafka.kafka
net.javaguides.springbootkafka.utils
(Then create the net.javaguides.springbootkafka.utils.AppConstants.java as coded in the tutorial)
Then go back and follow the tutorial step by step (amending for what you've already done above - I'm going to assume you can handle that yourself), taking note of each case-sensitive class declaration (to know what to call each .java file) and each 'package' statement the author has placed at the top of each class, to know where to put each .java file in your project / package structure. I'm going to assume you already know how to name .java class files and where to put them and why.
Now, back-tracking a bit (sorry, I'm trying to knock this out fast - but I did tell you...), I think the generation of the Springboot Kafka project in step 2 of the tutorial could also be clarified - maybe this is because their project generation page has changed, I don't know but in any case, here's what I did.
4.1. The beginning of Step 2 of the tutorial instructs you to create a Springboot project on the start.springboot.io page (that's where the link redirects to anyway) but this instruction is sparse, misleading and missing some steps. I had a few goes at it before I got it right. Here's a list of steps / things to check that helped me:
On the Springboot page, I chose the following options:
Project: Gradle - Groovy
Language: JAVA
Spring Boot: 3.0.5 (careful, things might have changed by the time you get to this, this is as of April 2023)
Group: net.javaguides
Artifact and name I left the same: (springbootkafka)
Description:
package: net.javaguides.springbootkafka (this will be generated / filled out for you as you type the others in)
Packaging: Jar (left as default, War may also work but I didn't try)
Java: 17 (Remember gradle doesn't work under JAVA 20 and I had errors mentioning JAVA 12 as I worked through the tutorial so I stuck with 17 (because 17 >=12 && < 20 && 12 isn't an option even though I had JDK 12 installed) and it worked. Lower versions may work, up to you if you want to try, but 20 definitely won't work because gradle doesn't like it).
4.2. Dependencies: Ok! Here it is! The thing I've already banged on about once or twice at last cleared up! The tutorial gives you a code snippet of some XML dependencies which, if you're a gradle type of person, you might sort of latch on to the idea that they go in your build.gradle file. But they don't. Or, not when or how they're shown anyway. There's nowhere you can add these on the Springboot project generation page and they're not in the right format for build.gradle as I know it (mentioned above) anyway. Instead of typing these in anywhere as the tutorial instructs, you need to do the following: Still on the Springboot page, you need to click the 'ADD DEPENDENCIES' button. The tutorial is not clear on which ones you need to choose; the information is sort of there in the dependencies code snippet (where I deduced it from) but you have to work out the likely 'free text' names of the dependencies from that. The ones I chose (and which got the tutorial working) were:
4.2.1. "Spring for Apache Kafka" (description: "Publish, subscribe, store and process streams of records")
4.2.2. "Spring Web" (description: "Build Web, including RESTful, application using Spring MVC. Using Apache Tomcat as the default embedded container")
NOTE: There are other likely looking candidates. Didn't try 'em, dunno if they work or not but I suspect not.
Once you've clicked on each dependency, it will appear listed under the 'ADD DEPENDNCIES' button and the misleading code snippet will be redundant because the dependencies will automatically be added into the build.gradle file (as I showed above) included in the project you are now about to download.
4.3. But now what do you do? The tutorial instructs you to import the project into IDEA. But what do you import into IDEA? And how do you get it on to your computer to import it in the first place? It was obvious enough to me that you had to download something and import it since all these-here IDEs nowadays are just about importing folder structures all the time. But on the Springboot page itself, I had to use my imagination a bit to work out how this might be done. Here are the steps I took:
4.3.1. Once all the settings are correct, click the slightly misleadingly labelled GENERATE button.
4.3.2. A 'save' dialogue with a file (probably) called springbootkafka.zip will appear. Click 'save' (or whatever your OS equivalent).
4.3.3. The ZIP file you download will contain one folder which is likely to be called springbootkafka (like the ZIP file itself but, you know, without the .zip extension). Unzip the ZIP file wherever it got saved on your local filesystem (In my case, my Windows Downloads folder).
4.3.4. Optionally move the unzipped folder (which will likely be called springbootkafka) into the place where you keep all your IDEA projects. Or you can just work on it in the place you unzipped it, it's up to you. I only want you to be happy.
4.3.5. NOW the 'import the project into IDEA' step (still under Step 2 of the tutorial) comes in. Open IDEA and use the 'Open'' button - on the opening screen on IDEA - to open the folder (wherever you ended up putting it on your local storage). There are other ways to open the project in IDEA depending on what was going on when you last closed IDEA. But I'm not going to go into all that, you'll just have to figure it out on your own. But once you do get it open, now your build.gradle file will be ready to rock and roll with the dependencies already added and the relevant JAR files already downloaded.
NOW, you can start following the tutorial in more or less the order shown.
I tried to add this all to the comments of the the tutorial at:
https://www.javaguides.net/2022/06/spring-boot-apache-kafka-tutorial.html
But it said the content was too long.
Maybe someone could edit the tutorial / incorporate the below somehow to help others trying to get the project to work.
This is a very useful tutorial and I was grateful to come across it. However, it took me a long time to get it working. Even after I worked through gradle dependency issues, I kept running into the errors 'JDK 17.0.6 isn't compatible with Gradle 7.6.1. Please fix JAVA_HOME environment variable' or - roughly - 'The consumer was configured to find a runtime of a library compatible with Java 12, packaged as a jar, and its dependencies declared externally, as well as attribute 'org.gradle.plugin.api-version' with value '7.6.1' but:' (and the error messages ended like that with nothing after the 'but:'! :-) )
Under Windows 11 (IntelliJ IDEA 2021.2.4 (Community edition), the tutorial does not build / will not work as shown. Maybe it's a little out of date or for a different version of IDEA or only works on Linux or whatever. In any case, I think it could do with a bit more information and clearer instruction.
The problems for me were different file syntax, the unclear / misleading instructions and java / SDK version differences; I think it makes a lot of assumptions about prior knowledge or system setup but does not seem to state these assumptions. In case it's helpful to anyone else, I got it working using the following. I had Zookeeper and Kafka already installed and working so I didn't bother with that part of the tutorial and cannot say whether it's accurate or not. I also knew automatically how to adjust the various Zookeeper and Kafka command line tools for Windows vs Linux and the location of my installations so I'm not covering them here either.
dependencies { implementation 'org.springframework.boot:spring-boot-starter-web' implementation 'org.springframework.kafka:spring-kafka' testImplementation 'org.springframework.boot:spring-boot-starter-test' testImplementation 'org.springframework.kafka:spring-kafka-test' }
BUT YOU DON'T CARE ABOUT THAT AT THIS STAGE. Read the rest of this first and hopefully you won't have to come back here.
JAVA versions: As JAVA 20 JDK is now out and gradle doesn't - as of April 2023 when I wrote this - run under it, it's important to ensure you are using a supported version of JAVA which is the same across your operating system, IntelliJ IDEA and for gradle within IDEA. But the Springboot project generation page (see later) offers JAVA 20 as an option. So remember NOT to choose it (but like I said, see later for that). I got it working with JAVA / JDK 17.0.6. My %JAVA_HOME% environment variable points to C:\Program Files\Java\jdk-17. Before running IntelliJ IDEA, I also added %JAVA_HOME%\bin at the front of my windows %PATH% environment variable so the first java.exe and javac.exe executables in my %PATH% are the correct versions (I know, I know, it's rookie stuff but it makes a difference if the right JDK isn't being used both in the OS environment when you crank up IDEA and inside IDEA itself). Setting IDEA and gradle to point at the correct JDK alone, are not enough. In fact, I found having IDEA fire up under the wrong JAVA version but with the right JAVA version in its settings, lead to counter-intuitive error messages whose causes did not point to the root cause.
After pointing %JAVA_HOME% and %PATH% to the right places, I also ensured IDEA and gradle were pointing at the same JDK as windows inside IDEA itself. This involves setting the SDK in two places, the 'Project structure' and the 'Gradle JVM'.
2.1. Project Structure: Under my installation of IDEA, this meant navigating to File->Project Structure->Project Setting->Project and choosing JDK 17.0.6 from the 'detected SDKs' dropdown. I also chose language level 16 (randomly) but others may work (I didn't experiment).
2.2. 'Gradle JVM': Gradle also needs to be pointed at the correct JDK. For my installation of Intellij IDEA (Windows 11), this meant navigating to File->Settings->Tools->Build, Execution, Deployment->Gradle then in the Gradle JVM dropdown, ensuring JAVA_HOME (for me with version 17.0.6 beside it) was selected.
3.1. First create the following two packages and the net.javaguides.springbootkafka.utils.AppConstant.java (as coded in the tutorial) in your IDEA Gradle project:
net.javaguides.springbootkafka.kafka net.javaguides.springbootkafka.utils (Then create the net.javaguides.springbootkafka.utils.AppConstants.java as coded in the tutorial)
Then go back and follow the tutorial step by step (amending for what you've already done above - I'm going to assume you can handle that yourself), taking note of each case-sensitive class declaration (to know what to call each .java file) and each 'package' statement the author has placed at the top of each class, to know where to put each .java file in your project / package structure. I'm going to assume you already know how to name .java class files and where to put them and why.
4.1. The beginning of Step 2 of the tutorial instructs you to create a Springboot project on the start.springboot.io page (that's where the link redirects to anyway) but this instruction is sparse, misleading and missing some steps. I had a few goes at it before I got it right. Here's a list of steps / things to check that helped me: On the Springboot page, I chose the following options: Project: Gradle - Groovy Language: JAVA Spring Boot: 3.0.5 (careful, things might have changed by the time you get to this, this is as of April 2023) Group: net.javaguides Artifact and name I left the same: (springbootkafka) Description: package: net.javaguides.springbootkafka (this will be generated / filled out for you as you type the others in) Packaging: Jar (left as default, War may also work but I didn't try) Java: 17 (Remember gradle doesn't work under JAVA 20 and I had errors mentioning JAVA 12 as I worked through the tutorial so I stuck with 17 (because 17 >=12 && < 20 && 12 isn't an option even though I had JDK 12 installed) and it worked. Lower versions may work, up to you if you want to try, but 20 definitely won't work because gradle doesn't like it).
4.2. Dependencies: Ok! Here it is! The thing I've already banged on about once or twice at last cleared up! The tutorial gives you a code snippet of some XML dependencies which, if you're a gradle type of person, you might sort of latch on to the idea that they go in your build.gradle file. But they don't. Or, not when or how they're shown anyway. There's nowhere you can add these on the Springboot project generation page and they're not in the right format for build.gradle as I know it (mentioned above) anyway. Instead of typing these in anywhere as the tutorial instructs, you need to do the following: Still on the Springboot page, you need to click the 'ADD DEPENDENCIES' button. The tutorial is not clear on which ones you need to choose; the information is sort of there in the dependencies code snippet (where I deduced it from) but you have to work out the likely 'free text' names of the dependencies from that. The ones I chose (and which got the tutorial working) were: 4.2.1. "Spring for Apache Kafka" (description: "Publish, subscribe, store and process streams of records") 4.2.2. "Spring Web" (description: "Build Web, including RESTful, application using Spring MVC. Using Apache Tomcat as the default embedded container") NOTE: There are other likely looking candidates. Didn't try 'em, dunno if they work or not but I suspect not.
Once you've clicked on each dependency, it will appear listed under the 'ADD DEPENDNCIES' button and the misleading code snippet will be redundant because the dependencies will automatically be added into the build.gradle file (as I showed above) included in the project you are now about to download.
4.3. But now what do you do? The tutorial instructs you to import the project into IDEA. But what do you import into IDEA? And how do you get it on to your computer to import it in the first place? It was obvious enough to me that you had to download something and import it since all these-here IDEs nowadays are just about importing folder structures all the time. But on the Springboot page itself, I had to use my imagination a bit to work out how this might be done. Here are the steps I took:
4.3.1. Once all the settings are correct, click the slightly misleadingly labelled GENERATE button. 4.3.2. A 'save' dialogue with a file (probably) called springbootkafka.zip will appear. Click 'save' (or whatever your OS equivalent). 4.3.3. The ZIP file you download will contain one folder which is likely to be called springbootkafka (like the ZIP file itself but, you know, without the .zip extension). Unzip the ZIP file wherever it got saved on your local filesystem (In my case, my Windows Downloads folder). 4.3.4. Optionally move the unzipped folder (which will likely be called springbootkafka) into the place where you keep all your IDEA projects. Or you can just work on it in the place you unzipped it, it's up to you. I only want you to be happy. 4.3.5. NOW the 'import the project into IDEA' step (still under Step 2 of the tutorial) comes in. Open IDEA and use the 'Open'' button - on the opening screen on IDEA - to open the folder (wherever you ended up putting it on your local storage). There are other ways to open the project in IDEA depending on what was going on when you last closed IDEA. But I'm not going to go into all that, you'll just have to figure it out on your own. But once you do get it open, now your build.gradle file will be ready to rock and roll with the dependencies already added and the relevant JAR files already downloaded.