Error building mleap-spring-boot

ronandoolan2 commented 1 year ago

running sbt compile inside the mleap-spring-boot folder gives the following errors, could you please advise

[info] welcome to sbt 1.9.1 (Ubuntu Java 11.0.19)
[info] loading project definition from mleapmleap-spring-boot/project
mleap/mleap-spring-boot/build.sbt:1: error: not found: object ml
import ml.combust.mleap.{Common, Dependencies, Protobuf}
       ^
mleap/mleap-spring-boot/build.sbt:3: error: not found: value DockerPlugin
enablePlugins(DockerPlugin, JavaAppPackaging)
              ^
mleap/mleap-spring-boot/build.sbt:3: error: not found: value JavaAppPackaging
enablePlugins(DockerPlugin, JavaAppPackaging)

Also shouldn't this folder contain a dockerfile?

jsleight commented 1 year ago

Think you need to run compile from the repo root. You're not getting the same build environment from inside the mleap-spring-boot directory. E.g., you have sbt 1.9.1 when it should be 1.4.9.

ronandoolan2 commented 1 year ago

I did build the root first.

I updated the project/build.properties inside mleap-spring-boot to use sbt 1.4.9. Not sure why it was using 1.9.1 there. But sbt 1.4.9 still gives the same errors

jsleight commented 1 year ago

I get the same error as you if I try to compile from the subdirectory -- to be honest I never have needed to do things in the subdirectories and always just run sbt commands from the root. Why do you want to build from the subdirectory?

ronandoolan2 commented 1 year ago

I was trying to get the binary for the mleap-spring-boot dockerfile. But now that I've digged into it I can see its a script calling the jar files. How do I generate this one for example? ml.combust.mleap.mleap-spring-boot-0.21.0-SNAPSHOT.jar

ronandoolan2 commented 1 year ago

#!/usr/bin/env bash

###  ------------------------------- ###
###  Helper methods for BASH scripts ###
###  ------------------------------- ###

die() {
  echo "$@" 1>&2
  exit 1
}

realpath () {
(
  TARGET_FILE="$1"
  CHECK_CYGWIN="$2"

  cd "$(dirname "$TARGET_FILE")"
  TARGET_FILE=$(basename "$TARGET_FILE")

  COUNT=0
  while [ -L "$TARGET_FILE" -a $COUNT -lt 100 ]
  do
      TARGET_FILE=$(readlink "$TARGET_FILE")
      cd "$(dirname "$TARGET_FILE")"
      TARGET_FILE=$(basename "$TARGET_FILE")
      COUNT=$(($COUNT + 1))
  done

  if [ "$TARGET_FILE" == "." -o "$TARGET_FILE" == ".." ]; then
    cd "$TARGET_FILE"
    TARGET_FILEPATH=
  else
    TARGET_FILEPATH=/$TARGET_FILE
  fi

  # make sure we grab the actual windows path, instead of cygwin's path.
  if [[ "x$CHECK_CYGWIN" == "x" ]]; then
    echo "$(pwd -P)/$TARGET_FILE"
  else
    echo $(cygwinpath "$(pwd -P)/$TARGET_FILE")
  fi
)
}

# TODO - Do we need to detect msys?

# Uses uname to detect if we're in the odd cygwin environment.
is_cygwin() {
  local os=$(uname -s)
  case "$os" in
    CYGWIN*) return 0 ;;
    *)  return 1 ;;
  esac
}

# This can fix cygwin style /cygdrive paths so we get the
# windows style paths.
cygwinpath() {
  local file="$1"
  if is_cygwin; then
    echo $(cygpath -w $file)
  else
    echo $file
  fi
}

# Make something URI friendly
make_url() {
  url="$1"
  local nospaces=${url// /%20}
  if is_cygwin; then
    echo "/${nospaces//\\//}"
  else
    echo "$nospaces"
  fi
}

# This crazy function reads in a vanilla "linux" classpath string (only : are separators, and all /),
# and returns a classpath with windows style paths, and ; separators.
fixCygwinClasspath() {
  OLDIFS=$IFS
  IFS=":"
  read -a classpath_members <<< "$1"
  declare -a fixed_members
  IFS=$OLDIFS
  for i in "${!classpath_members[@]}"
  do
    fixed_members[i]=$(realpath "${classpath_members[i]}" "fix")
  done
  IFS=";"
  echo "${fixed_members[*]}"
  IFS=$OLDIFS
}

# Fix the classpath we use for cygwin.
fix_classpath() {
  cp="$1"
  if is_cygwin; then
    echo "$(fixCygwinClasspath "$cp")"
  else
    echo "$cp"
  fi
}
# Detect if we should use JAVA_HOME or just try PATH.
get_java_cmd() {
  if [[ -n "$JAVA_HOME" ]] && [[ -x "$JAVA_HOME/bin/java" ]];  then
    echo "$JAVA_HOME/bin/java"
  else
    echo "java"
  fi
}

echoerr () {
  echo 1>&2 "$@"
}
vlog () {
  [[ $verbose || $debug ]] && echoerr "$@"
}
dlog () {
  [[ $debug ]] && echoerr "$@"
}
execRunner () {
  # print the arguments one to a line, quoting any containing spaces
  [[ $verbose || $debug ]] && echo "# Executing command line:" && {
    for arg; do
      if printf "%s\n" "$arg" | grep -q ' '; then
    printf "\"%s\"\n" "$arg"
      else
    printf "%s\n" "$arg"
      fi
    done
    echo ""
  }

  # we use "exec" here for our pids to be accurate.
  exec "$@"
}
addJava () {
  dlog "[addJava] arg = '$1'"
  java_args+=( "$1" )
}
addApp () {
  dlog "[addApp] arg = '$1'"
  app_commands+=( "$1" )
}
addResidual () {
  dlog "[residual] arg = '$1'"
  residual_args+=( "$1" )
}
addDebugger () {
  addJava "-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=$1"
}

require_arg () {
  local type="$1"
  local opt="$2"
  local arg="$3"
  if [[ -z "$arg" ]] || [[ "${arg:0:1}" == "-" ]]; then
    die "$opt requires <$type> argument"
  fi
}
is_function_defined() {
  declare -f "$1" > /dev/null
}

# Attempt to detect if the script is running via a GUI or not
# TODO - Determine where/how we use this generically
detect_terminal_for_ui() {
  [[ ! -t 0 ]] && [[ "${#residual_args}" == "0" ]] && {
    echo "true"
  }
  # SPECIAL TEST FOR MAC
  [[ "$(uname)" == "Darwin" ]] && [[ "$HOME" == "$PWD" ]] && [[ "${#residual_args}" == "0" ]] && {
    echo "true"
  }
}

# Processes incoming arguments and places them in appropriate global variables.  called by the run method.
process_args () {
  local no_more_snp_opts=0
  while [[ $# -gt 0 ]]; do
    case "$1" in
         --) shift && no_more_snp_opts=1 && break ;;
       -h|-help) usage; exit 1 ;;
    -v|-verbose) verbose=1 && shift ;;
      -d|-debug) debug=1 && shift ;;

    -no-version-check) no_version_check=1 && shift ;;

       -mem) echo "!! WARNING !! -mem option is ignored. Please use -J-Xmx and -J-Xms" && shift 2 ;;
     -jvm-debug) require_arg port "$1" "$2" && addDebugger $2 && shift 2 ;;

      -main) custom_mainclass="$2" && shift 2 ;;

     -java-home) require_arg path "$1" "$2" && jre=`eval echo $2` && java_cmd="$jre/bin/java" && shift 2 ;;

 -D*|-agentlib*) addJava "$1" && shift ;;
        -J*) addJava "${1:2}" && shift ;;
          *) addResidual "$1" && shift ;;
    esac
  done

  if [[ no_more_snp_opts ]]; then
    while [[ $# -gt 0 ]]; do
      addResidual "$1" && shift
    done
  fi

  is_function_defined process_my_args && {
    myargs=("${residual_args[@]}")
    residual_args=()
    process_my_args "${myargs[@]}"
  }
}

# Actually runs the script.
run() {
  # TODO - check for sane environment

  # process the combined args, then reset "$@" to the residuals
  process_args "$@"
  set -- "${residual_args[@]}"
  argumentCount=$#

  #check for jline terminal fixes on cygwin
  if is_cygwin; then
    stty -icanon min 1 -echo > /dev/null 2>&1
    addJava "-Djline.terminal=jline.UnixTerminal"
    addJava "-Dsbt.cygwin=true"
  fi

  # check java version
  if [[ ! $no_version_check ]]; then
    java_version_check
  fi

  if [ -n "$custom_mainclass" ]; then
    mainclass=("$custom_mainclass")
  else
    mainclass=("${app_mainclass[@]}")
  fi

  # Now we check to see if there are any java opts on the environment. These get listed first, with the script able to override them.
  if [[ "$JAVA_OPTS" != "" ]]; then
    java_opts="${JAVA_OPTS}"
  fi

  # run sbt
  execRunner "$java_cmd" \
    ${java_opts[@]} \
    "${java_args[@]}" \
    -cp "$(fix_classpath "$app_classpath")" \
    "${mainclass[@]}" \
    "${app_commands[@]}" \
    "${residual_args[@]}"

  local exit_code=$?
  if is_cygwin; then
    stty icanon echo > /dev/null 2>&1
  fi
  exit $exit_code
}

# Loads a configuration file full of default command line options for this script.
loadConfigFile() {
  cat "$1" | sed '/^\#/d'
}

# Now check to see if it's a good enough version
# TODO - Check to see if we have a configured default java version, otherwise use 1.6
java_version_check() {
  readonly java_version=$("$java_cmd" -version 2>&1 | awk -F '"' '/version/ {print $2}')
  if [[ "$java_version" == "" ]]; then
    echo
    echo No java installations was detected.
    echo Please go to http://www.java.com/getjava/ and download
    echo
    exit 1
  elif [[ ! "$java_version" > "1.6" ]]; then
    echo
    echo The java installation you have is not up to date
    echo $app_name requires at least version 1.6+, you have
    echo version $java_version
    echo
    echo Please go to http://www.java.com/getjava/ and download
    echo a valid Java Runtime and install before running $app_name.
    echo
    exit 1
  fi
}

###  ------------------------------- ###
###  Start of customized settings    ###
###  ------------------------------- ###
usage() {
 cat <<EOM
Usage: $script_name [options]

  -h | -help         print this message
  -v | -verbose      this runner is chattier
  -d | -debug        set sbt log level to debug
  -no-version-check  Don't run the java version check.
  -main <classname>  Define a custom main class
  -jvm-debug <port>  Turn on JVM debugging, open at the given port.

  # java version (default: java from PATH, currently $(java -version 2>&1 | grep version))
  -java-home <path>         alternate JAVA_HOME

  # jvm options and output control
  JAVA_OPTS          environment variable, if unset uses "$java_opts"
  -Dkey=val          pass -Dkey=val directly to the java runtime
  -J-X               pass option -X directly to the java runtime
             (-J is stripped)

  # special option
  --                 To stop parsing built-in commands from the rest of the command-line.
             e.g.) enabling debug and sending -d as app argument
             \$ ./start-script -d -- -d

In the case of duplicated or conflicting options, basically the order above
shows precedence: JAVA_OPTS lowest, command line options highest except "--".
EOM
}

###  ------------------------------- ###
###  Main script                     ###
###  ------------------------------- ###

declare -a residual_args
declare -a java_args
declare -a app_commands
declare -r real_script_path="$(realpath "$0")"
declare -r app_home="$(realpath "$(dirname "$real_script_path")")"
# TODO - Check whether this is ok in cygwin...
declare -r lib_dir="$(realpath "${app_home}/../lib")"
declare -a app_mainclass=("ml.combust.mleap.springboot.Starter")

declare -r script_conf_file="${app_home}/../conf/application.ini"
declare -r app_classpath="$lib_dir/ml.combust.mleap.mleap-spring-boot-0.21.0-SNAPSHOT.jar:$lib_dir/ml.combust.mleap.mleap-executor-0.21.0-SNAPSHOT.jar:$lib_dir/ml.combust.mleap.mleap-runtime-0.21.0-SNAPSHOT.jar:$lib_dir/ml.combust.mleap.mleap-core-0.21.0-SNAPSHOT.jar:$lib_dir/ml.combust.mleap.mleap-base-0.21.0-SNAPSHOT.jar:$lib_dir/ml.combust.mleap.mleap-tensor-0.21.0-SNAPSHOT.jar:$lib_dir/ml.combust.bundle.bundle-ml-0.21.0-SNAPSHOT.jar:$lib_dir/org.scala-lang.scala-library-2.12.13.jar:$lib_dir/io.spray.spray-json_2.12-1.3.2.jar:$lib_dir/org.apache.spark.spark-mllib-local_2.12-3.2.0.jar:$lib_dir/org.scalanlp.breeze_2.12-1.2.jar:$lib_dir/org.scalanlp.breeze-macros_2.12-1.2.jar:$lib_dir/org.scala-lang.scala-reflect-2.12.13.jar:$lib_dir/com.github.fommil.netlib.core-1.1.2.jar:$lib_dir/net.sourceforge.f2j.arpack_combined_all-0.1.jar:$lib_dir/net.sf.opencsv.opencsv-2.3.jar:$lib_dir/com.github.wendykierp.JTransforms-3.1.jar:$lib_dir/pl.edu.icm.JLargeArrays-1.5.jar:$lib_dir/com.chuusai.shapeless_2.12-2.3.3.jar:$lib_dir/org.typelevel.macro-compat_2.12-1.1.1.jar:$lib_dir/org.typelevel.spire_2.12-0.17.0.jar:$lib_dir/org.typelevel.spire-macros_2.12-0.17.0.jar:$lib_dir/org.scala-lang.modules.scala-collection-compat_2.12-2.1.1.jar:$lib_dir/org.typelevel.algebra_2.12-2.0.1.jar:$lib_dir/org.typelevel.cats-kernel_2.12-2.1.1.jar:$lib_dir/org.typelevel.spire-platform_2.12-0.17.0.jar:$lib_dir/org.typelevel.spire-util_2.12-0.17.0.jar:$lib_dir/org.apache.commons.commons-math3-3.4.1.jar:$lib_dir/org.apache.spark.spark-tags_2.12-3.2.0.jar:$lib_dir/org.spark-project.spark.unused-1.0.0.jar:$lib_dir/dev.ludovic.netlib.blas-2.2.0.jar:$lib_dir/com.github.rwl.jtransforms-2.4.0.jar:$lib_dir/com.google.protobuf.protobuf-java-3.5.1.jar:$lib_dir/com.thesamet.scalapb.scalapb-runtime_2.12-0.7.1.jar:$lib_dir/com.thesamet.scalapb.lenses_2.12-0.7.0-test2.jar:$lib_dir/com.lihaoyi.fastparse_2.12-1.0.0.jar:$lib_dir/com.lihaoyi.fastparse-utils_2.12-1.0.0.jar:$lib_dir/com.lihaoyi.sourcecode_2.12-0.1.4.jar:$lib_dir/com.jsuereth.scala-arm_2.12-2.0.jar:$lib_dir/commons-io.commons-io-2.5.jar:$lib_dir/com.typesafe.akka.akka-stream_2.12-2.6.14.jar:$lib_dir/com.typesafe.akka.akka-actor_2.12-2.6.14.jar:$lib_dir/com.typesafe.config-1.4.0.jar:$lib_dir/org.scala-lang.modules.scala-java8-compat_2.12-0.8.0.jar:$lib_dir/com.typesafe.akka.akka-protobuf-v3_2.12-2.6.14.jar:$lib_dir/org.reactivestreams.reactive-streams-1.0.3.jar:$lib_dir/com.typesafe.ssl-config-core_2.12-0.4.2.jar:$lib_dir/org.scala-lang.modules.scala-parser-combinators_2.12-1.1.2.jar:$lib_dir/com.typesafe.scala-logging.scala-logging_2.12-3.9.0.jar:$lib_dir/org.springframework.boot.spring-boot-starter-web-2.6.2.jar:$lib_dir/org.springframework.boot.spring-boot-starter-2.6.2.jar:$lib_dir/org.springframework.boot.spring-boot-2.6.2.jar:$lib_dir/org.springframework.spring-core-5.3.14.jar:$lib_dir/org.springframework.spring-jcl-5.3.14.jar:$lib_dir/org.springframework.spring-context-5.3.14.jar:$lib_dir/org.springframework.spring-aop-5.3.14.jar:$lib_dir/org.springframework.spring-beans-5.3.14.jar:$lib_dir/org.springframework.spring-expression-5.3.14.jar:$lib_dir/org.springframework.boot.spring-boot-autoconfigure-2.6.2.jar:$lib_dir/org.springframework.boot.spring-boot-starter-logging-2.6.2.jar:$lib_dir/ch.qos.logback.logback-classic-1.2.9.jar:$lib_dir/ch.qos.logback.logback-core-1.2.9.jar:$lib_dir/org.slf4j.slf4j-api-1.7.32.jar:$lib_dir/org.apache.logging.log4j.log4j-to-slf4j-2.17.0.jar:$lib_dir/org.slf4j.jul-to-slf4j-1.7.32.jar:$lib_dir/jakarta.annotation.jakarta.annotation-api-1.3.5.jar:$lib_dir/org.yaml.snakeyaml-1.29.jar:$lib_dir/org.springframework.boot.spring-boot-starter-json-2.6.2.jar:$lib_dir/org.springframework.spring-web-5.3.14.jar:$lib_dir/com.fasterxml.jackson.core.jackson-databind-2.13.1.jar:$lib_dir/com.fasterxml.jackson.core.jackson-annotations-2.13.1.jar:$lib_dir/com.fasterxml.jackson.core.jackson-core-2.13.1.jar:$lib_dir/com.fasterxml.jackson.datatype.jackson-datatype-jdk8-2.13.1.jar:$lib_dir/com.fasterxml.jackson.datatype.jackson-datatype-jsr310-2.13.1.jar:$lib_dir/com.fasterxml.jackson.module.jackson-module-parameter-names-2.13.1.jar:$lib_dir/org.springframework.boot.spring-boot-starter-tomcat-2.6.2.jar:$lib_dir/org.apache.tomcat.embed.tomcat-embed-core-9.0.56.jar:$lib_dir/org.apache.tomcat.embed.tomcat-embed-el-9.0.56.jar:$lib_dir/org.apache.tomcat.embed.tomcat-embed-websocket-9.0.56.jar:$lib_dir/org.springframework.spring-webmvc-5.3.14.jar:$lib_dir/org.springframework.boot.spring-boot-starter-actuator-2.6.2.jar:$lib_dir/org.springframework.boot.spring-boot-actuator-autoconfigure-2.6.2.jar:$lib_dir/org.springframework.boot.spring-boot-actuator-2.6.2.jar:$lib_dir/io.micrometer.micrometer-core-1.8.1.jar:$lib_dir/org.hdrhistogram.HdrHistogram-2.1.12.jar:$lib_dir/org.latencyutils.LatencyUtils-2.0.3.jar:$lib_dir/org.apache.commons.commons-lang3-3.7.jar:$lib_dir/com.thesamet.scalapb.scalapb-runtime-grpc_2.12-0.7.1.jar:$lib_dir/io.grpc.grpc-stub-1.10.0.jar:$lib_dir/io.grpc.grpc-core-1.10.0.jar:$lib_dir/io.grpc.grpc-context-1.10.0.jar:$lib_dir/com.google.code.gson.gson-2.7.jar:$lib_dir/com.google.guava.guava-19.0.jar:$lib_dir/com.google.errorprone.error_prone_annotations-2.1.2.jar:$lib_dir/com.google.code.findbugs.jsr305-3.0.0.jar:$lib_dir/io.opencensus.opencensus-api-0.11.0.jar:$lib_dir/io.opencensus.opencensus-contrib-grpc-metrics-0.11.0.jar:$lib_dir/io.grpc.grpc-protobuf-1.10.0.jar:$lib_dir/com.google.protobuf.protobuf-java-util-3.5.1.jar:$lib_dir/com.google.api.grpc.proto-google-common-protos-1.0.0.jar:$lib_dir/io.grpc.grpc-protobuf-lite-1.10.0.jar:$lib_dir/com.thesamet.scalapb.scalapb-json4s_2.12-0.7.1.jar:$lib_dir/org.json4s.json4s-jackson_2.12-3.5.1.jar:$lib_dir/org.json4s.json4s-core_2.12-3.5.1.jar:$lib_dir/org.json4s.json4s-ast_2.12-3.5.1.jar:$lib_dir/org.json4s.json4s-scalap_2.12-3.5.1.jar:$lib_dir/com.thoughtworks.paranamer.paranamer-2.8.jar:$lib_dir/org.scala-lang.modules.scala-xml_2.12-1.0.6.jar"

# java_cmd is overrode in process_args when -java-home is used
declare java_cmd=$(get_java_cmd)

# if configuration files exist, prepend their contents to $@ so it can be processed by this runner
[[ -f "$script_conf_file" ]] && set -- $(loadConfigFile "$script_conf_file") "$@"

run "$@"

jsleight commented 1 year ago

How do I generate this one for example? ml.combust.mleap.mleap-spring-boot-0.21.0-SNAPSHOT.jar

If you run sbt package from the repo root then it will build all the jars. The one you're looking for would be located in mleap-spring-boot/target/scala-2.12/

ronandoolan2 commented 1 year ago

Ok great thanks

combust / mleap

Error building mleap-spring-boot #859