When running this getting-started, observed the following error due to agent attempting to connect to spline before the latter was ready.
agent_1 | [java] 21/07/30 12:45:28 INFO HttpLineageDispatcher: Producer URL: http://172.17.0.1:8080/producer
agent_1 | [java] Exception in thread "main" java.lang.IllegalArgumentException: Error while instantiating 'org.apache.spark.sql.internal.SessionStateBuilder':
...
agent_1 | [java] Caused by: za.co.absa.spline.harvester.exception.SplineInitializationException: Spark Agent was not able to establish connection to Spline Gateway
...
agent_1 | [java] Caused by: java.net.SocketException: Unexpected end of file from server
...
This PR fixes by having agent wait for spline initialization in a similar manner to how spline waits for arangodb initialization.
${SPLINE_REST_PORT} is sufficient because this value is already present in .env.
$${} escaping for other values prevents Compose from interpolating a value (that would default to a blank string since not present in .env) and allows the environment variables present under environment:, passed in via command line, or present in the original Dockerfile to be interpreted. Reference: https://docs.docker.com/compose/compose-file/compose-file-v3/#variable-substitution.
When running this getting-started, observed the following error due to
agentattempting to connect tosplinebefore the latter was ready.This PR fixes by having
agentwait forsplineinitialization in a similar manner to howsplinewaits forarangodbinitialization.${SPLINE_REST_PORT}is sufficient because this value is already present in.env.$${}escaping for other values prevents Compose from interpolating a value (that would default to a blank string since not present in.env) and allows the environment variables present underenvironment:, passed in via command line, or present in the original Dockerfile to be interpreted. Reference: https://docs.docker.com/compose/compose-file/compose-file-v3/#variable-substitution.