Dockerize Spark NLP

[tschaffter]

Tasks

• [x] Install Java
• [x] Install Apache Spark
• [x] Install Spark NLP
• [x] Install Spark NLP for Healthcare
• [ ] Update PHI identifier to user Spark NLP model

[tschaffter]

@clemEssien I needed to install man (mkdir -p /usr/share/man/man1) before installing Java. I now have Java installed:

$ docker exec -it phi-annotator bash
root@3003013fd504:/# java -version
openjdk version "11.0.11" 2021-04-20
OpenJDK Runtime Environment (build 11.0.11+9-post-Debian-1deb10u1)
OpenJDK 64-Bit Server VM (build 11.0.11+9-post-Debian-1deb10u1, mixed mode, sharing)

[tschaffter]

Apache Spark is running by default on port 8080. I now run uWSGI on port 8081 to prevent conflict.

After running apt install iproute2 -y

root@8254dd7f5469:/opt/spark# ss -tunelp | grep 8081
tcp     LISTEN   0        100              0.0.0.0:8081           0.0.0.0:*      uid:33 ino:135958 sk:3 <->
root@8254dd7f5469:/opt/spark# ss -tunelp | grep 8080
tcp     LISTEN   0        1                0.0.0.0:8080           0.0.0.0:*      users:(("java",pid=395,fd=336)) ino:140877 sk:5 <->

[tschaffter]

Notes

• [x] I can't start the Apache Spark server (start-master.sh) in the Dockerfile (obviously :)). I'll start the server in docker-entrypoint.sh.
• [x] Even when started manually from the container, the page http://localhost:8080 does not show the spark page illustrated in this article. The port 8080 is open and uWSGI is running and accessible on port 8081.

EDIT: I could not access port 8080 because I had another service running on my computer on that port. After stopping it, I can now access the Apache Spark landing page.

[tschaffter]

I can successfully load Spark NLP from the terminal using the instruction listed here.

[tschaffter]

Replaced by https://github.com/nlpsandbox/phi-annotator-spark-nlp/pull/3