If you would like to write custom YARN application, i.e. via Spring Boot, using this manual (https://spring.io/guides/gs/yarn-basic/) you will face couple of errors:
One issue about guide.
Make sure you have corresponding versions of "spring-yarn-boot", "spring-boot-starter-parent" and "spring-boot-maven-plugin" in pom-files. Simpliest solution go to maven central and use latest versions.
Otherwise you may face "cannot instantiate 'yarnClient' bean" error, which tell you almost nothing why it failed
Another about this docker image.
Make sure you set limitless virtual memory for YARN.
For example, you may set in yarn-site.xml this properties:
<property>
<name>yarn.nodemanager.vmem-check-enabled</name>
<value>false</value>
<description>Whether virtual memory limits will be enforced for containers</description>
</property>
<property>
<name>yarn.nodemanager.vmem-pmem-ratio</name>
<value>4</value>
<description>Ratio between virtual memory to physical memory when setting memory limits for containers</description>
</property>
Cause you use it in Docker, you have to change it somehow after hadoop cluster initiation, or before, mapping volumes with "-v" option
Another solution is to use fork of this docker image, which has this settings by default, i.e. - https://github.com/sfedyakov/hadoop-271-cluster
If you would like to write custom YARN application, i.e. via Spring Boot, using this manual (https://spring.io/guides/gs/yarn-basic/) you will face couple of errors:
Cause you use it in Docker, you have to change it somehow after hadoop cluster initiation, or before, mapping volumes with "-v" option Another solution is to use fork of this docker image, which has this settings by default, i.e. - https://github.com/sfedyakov/hadoop-271-cluster