clemEssien / spark-nlp-phi-annotator

Apache License 2.0
1 stars 0 forks source link

Issue installing Java 8 in dockerfile #5

Open clemEssien opened 3 years ago

clemEssien commented 3 years ago

Problem description

 Java 8 is not installed when the command to install is supplied in the Dockerfile. Java 8 is a requirement for the Spark 
 Session to be initialized and the JAVA_HOME environment variable should be set
 Unable to locate package openjdk-8-jdk

''' ERROR [ 3/17] RUN apt-get install openjdk-8-jdk 1.1s


[ 3/17] RUN apt-get install openjdk-8-jdk:

6 0.301 + apt-get install openjdk-8-jdk

6 0.331 Reading package lists...

6 0.888 Building dependency tree...

6 0.996 Reading state information...

6 1.086 E: Unable to locate package openjdk-8-jdk

'''

Steps to reproduce the issue

Add the following lines into the Dockerfile

  1. RUN apt-get update
  2. RUN apt-get install openjdk-8-jdk
  3. Then run docker-compose up

Expected behavior

     Java 8 is expected to be installed while creating the docker image

Attempted fixes

    I tried to install from official repositories instead of the manual installation by running the following commands:
tschaffter commented 3 years ago

@clemEssien Try installing the system library default-jre, see example.

clemEssien commented 3 years ago

The default-jre installs Java 11. The installation specifies Java 8. However, I tried it a few times, the latest error here is a Java gateway exception i.e. ''' Exception in thread "main" java.io.FileNotFoundException: /var/www/.ivy2/cache/resolved-org.apache.spark-spark-submit-parent-99cf1eb8-395b-40b9-a777-9618361c666f-1.0.xml (No such file or directory) phi-annotator | at java.base/java.io.FileOutputStream.open0(Native Method) phi-annotator | at java.base/java.io.FileOutputStream.open(FileOutputStream.java:298) phi-annotator | at java.base/java.io.FileOutputStream.(FileOutputStream.java:237) phi-annotator | at java.base/java.io.FileOutputStream.(FileOutputStream.java:187) phi-annotator | at org.apache.ivy.plugins.parser.xml.XmlModuleDescriptorWriter.write(XmlModuleDescriptorWriter.java:70) phi-annotator | at org.apache.ivy.plugins.parser.xml.XmlModuleDescriptorWriter.write(XmlModuleDescriptorWriter.java:62) phi-annotator | at org.apache.ivy.core.module.descriptor.DefaultModuleDescriptor.toIvyFile(DefaultModuleDescriptor.java:563) phi-annotator | at org.apache.ivy.core.cache.DefaultResolutionCacheManager.saveResolvedModuleDescriptor(DefaultResolutionCacheManager.java:176) phi-annotator | at org.apache.ivy.core.resolve.ResolveEngine.resolve(ResolveEngine.java:245) phi-annotator | at org.apache.ivy.Ivy.resolve(Ivy.java:523) phi-annotator | at org.apache.spark.deploy.SparkSubmitUtils$.resolveMavenCoordinates(SparkSubmit.scala:1418) phi-annotator | at org.apache.spark.deploy.DependencyUtils$.resolveMavenDependencies(DependencyUtils.scala:54) phi-annotator | at org.apache.spark.deploy.SparkSubmit.prepareSubmitEnvironment(SparkSubmit.scala:308) phi-annotator | at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:894) phi-annotator | at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180) phi-annotator | at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203) phi-annotator | at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90) phi-annotator | at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1030) phi-annotator | at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1039) phi-annotator | at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) phi-annotator | Failed to add operation for POST /api/v1/textContactAnnotations phi-annotator | Traceback (most recent call last): phi-annotator | File "openapi_server/main.py", line 10, in phi-annotator | app.add_api('openapi.yaml', pythonic_params=True) phi-annotator | File "/usr/local/lib/python3.8/site-packages/connexion/apps/flask_app.py", line 57, in add_api phi-annotator | api = super(FlaskApp, self).add_api(specification, kwargs) phi-annotator | File "/usr/local/lib/python3.8/site-packages/connexion/apps/abstract.py", line 144, in add_api phi-annotator | api = self.api_cls(specification, phi-annotator | File "/usr/local/lib/python3.8/site-packages/connexion/apis/abstract.py", line 111, in init phi-annotator | self.add_paths() phi-annotator | File "/usr/local/lib/python3.8/site-packages/connexion/apis/abstract.py", line 219, in add_paths phi-annotator | self._handle_add_operation_error(path, method, sys.exc_info()) phi-annotator | File "/usr/local/lib/python3.8/site-packages/connexion/apis/abstract.py", line 231, in _handle_add_operation_error phi-annotator | raise value.with_traceback(traceback) phi-annotator | File "/usr/local/lib/python3.8/site-packages/connexion/apis/abstract.py", line 209, in add_paths phi-annotator | self.add_operation(path, method) phi-annotator | File "/usr/local/lib/python3.8/site-packages/connexion/apis/abstract.py", line 162, in add_operation phi-annotator | operation = make_operation( phi-annotator | File "/usr/local/lib/python3.8/site-packages/connexion/operations/init.py", line 8, in make_operation phi-annotator | return spec.operation_cls.from_spec(spec, *args, *kwargs) phi-annotator | File "/usr/local/lib/python3.8/site-packages/connexion/operations/openapi.py", line 128, in from_spec phi-annotator | return cls( phi-annotator | File "/usr/local/lib/python3.8/site-packages/connexion/operations/openapi.py", line 75, in init phi-annotator | super(OpenAPIOperation, self).init( phi-annotator | File "/usr/local/lib/python3.8/site-packages/connexion/operations/abstract.py", line 96, in init phi-annotator | self._resolution = resolver.resolve(self) phi-annotator | File "/usr/local/lib/python3.8/site-packages/connexion/resolver.py", line 40, in resolve phi-annotator | return Resolution(self.resolve_function_from_operation_id(operation_id), operation_id) phi-annotator | File "/usr/local/lib/python3.8/site-packages/connexion/resolver.py", line 61, in resolve_function_from_operation_id phi-annotator | return self.function_resolver(operation_id) phi-annotator | File "/usr/local/lib/python3.8/site-packages/connexion/utils.py", line 111, in get_function_from_name phi-annotator | module = importlib.import_module(module_name) phi-annotator | File "/usr/local/lib/python3.8/importlib/init.py", line 127, in import_module phi-annotator | return _bootstrap._gcd_import(name[level:], package, level) phi-annotator | File "", line 1014, in _gcd_import phi-annotator | File "", line 991, in _find_and_load phi-annotator | File "", line 975, in _find_and_load_unlocked phi-annotator | File "", line 671, in _load_unlocked phi-annotator | File "", line 783, in exec_module phi-annotator | File "", line 219, in _call_with_frames_removed phi-annotator | File "./openapi_server/controllers/text_contact_annotation_controller.py", line 4, in phi-annotator | from openapi_server import nlp_config as cf phi-annotator | File "./openapi_server/nlp_config.py", line 19, in phi-annotator | spark = Spark().spark phi-annotator | File "./openapi_server/nlp_config.py", line 16, in init phi-annotator | self.spark = sparknlp_jsl.start(os.environ['JSL_SECRET'], params=params) phi-annotator | File "/usr/local/lib/python3.8/site-packages/sparknlp_jsl/init.py", line 80, in start phi-annotator | return builder.getOrCreate() phi-annotator | File "/usr/local/lib/python3.8/site-packages/pyspark/sql/session.py", line 228, in getOrCreate phi-annotator | sc = SparkContext.getOrCreate(sparkConf) phi-annotator | File "/usr/local/lib/python3.8/site-packages/pyspark/context.py", line 384, in getOrCreate phi-annotator | SparkContext(conf=conf or SparkConf()) phi-annotator | File "/usr/local/lib/python3.8/site-packages/pyspark/context.py", line 144, in init phi-annotator | SparkContext._ensure_initialized(self, gateway=gateway, conf=conf) phi-annotator | File "/usr/local/lib/python3.8/site-packages/pyspark/context.py", line 331, in _ensure_initialized phi-annotator | SparkContext._gateway = gateway or launch_gateway(conf) phi-annotator | File "/usr/local/lib/python3.8/site-packages/pyspark/java_gateway.py", line 108, in launch_gateway phi-annotator | raise Exception("Java gateway process exited before sending its port number") phi-annotator | Exception: Java gateway process exited before sending its port number phi-annotator | unable to load app 0 (mountpoint='') (callable not found or import error) phi-annotator | no app loaded. going in full dynamic mode *** '''