search

marklogic-community / roxy

Deployment tool for MarkLogic applications. Also provides optional unit test and XQuery MVC structure
Other
87 stars 66 forks source link

mlcpbatch failing after ML upgrade #816

Open ArunPrakashThangavelu opened 7 years ago

ArunPrakashThangavelu commented 7 years ago

I'm trying to run mlcpbatch after the ML upgrade but i'm getting the below error

build   15-Jun-2017 16:00:33    Started mlcpbatch
build   15-Jun-2017 16:00:39    Running mlcp with options file: axis.temporal.optionsjava -cp "/app/dvmdsgl/mlcp/mlcp-8.0-4/conf:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/commons-io-2.4.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/xml-apis-1.4.01.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/hadoop-common-2.6.0.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/xpp3-1.1.3.3.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/hadoop-yarn-client-2.6.0.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/mlcp-8.0-4.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/commons-logging-1.1.3.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/jackson-xc-1.9.13.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/jena-iri-0.9.5.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/htrace-core-3.0.4.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/commons-codec-1.4.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/log4j-1.2.17.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/hadoop-auth-2.6.0.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/guava-11.0.2.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/hadoop-yarn-common-2.6.0.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/jena-arq-2.10.0.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/slf4j-api-1.6.1.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/jena-core-2.10.0.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/xercesImpl-2.10.0.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/commons-modeler-2.0.1.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/hadoop-annotations-2.6.0.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/commons-lang-2.6.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/commons-cli-1.2.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/jackson-jaxrs-1.9.13.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/avro-1.7.4.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/jersey-core-1.9.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/jackson-core-asl-1.9.13.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/junit-4.11.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/commons-configuration-1.6.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/jackson-mapper-asl-1.9.13.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/jersey-client-1.9.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/marklogic-mapreduce2-2.1-4.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/hadoop-mapreduce-client-common-2.6.0.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/hadoop-mapreduce-client-core-2.6.0.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/commons-httpclient-3.1.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/marklogic-xcc-8.0-4.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/protobuf-java-2.5.0.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/commons-collections-3.2.1.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/hadoop-yarn-api-2.6.0.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/xstream-1.4.2.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/commons-csv-1.2.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/hadoop-mapreduce-client-jobclient-2.6.0.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/hadoop-hdfs-2.6.0.jar:/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/slf4j-log4j12-1.6.4.jar" "-DCONTENTPUMP_HOME=/app/dvmdsgl/mlcp/mlcp-8.0-4" -Dfile.encoding=UTF-8 -Dxcc.txn.compatible=true "-Djava.library.path=/app/dvmdsgl/mlcp/mlcp-8.0-4/lib/native" -Xmx512m  com.marklogic.contentpump.ContentPump -option_file /mlcp/axis.temporal.options  -username test1_nds_gl_si -password '####' -host itgsydapd654.macbank -port 8061
build   15-Jun-2017 16:00:39    
error   15-Jun-2017 16:00:40    17/06/15 16:00:40 ERROR contentpump.ContentPump: Error while expanding arguments
error   15-Jun-2017 16:00:40    java.lang.IllegalArgumentException: Unknown command: -option_file
error   15-Jun-2017 16:00:40        at com.marklogic.contentpump.Command.forName(Command.java:1373)
error   15-Jun-2017 16:00:40        at com.marklogic.contentpump.ContentPump.runCommand(ContentPump.java:83)
error   15-Jun-2017 16:00:40        at com.marklogic.contentpump.ContentPump.main(ContentPump.java:62)
error   15-Jun-2017 16:00:40    Unknown command: -option_file
error   15-Jun-2017 16:00:40    Try 'mlcp help' for usage.
build   15-Jun-2017 16:00:40

Also when i tried to run in my local then iI'm getting this error

bash-4.1$ /app/dvmdsgl/java/jdk-8/bin/java -jar roxy.jar int mlcpbatch -v
LoadError: no such file to load -- rubygems
  require at org/jruby/RubyKernel.java:1040
   (root) at file:/app/dvmdsgl/home/dvmdsglx/Bamboo_Agent/xml-data/build-dir/NDS-NACT132-JOB1/ml-pipeline/roxy/roxy.jar!/META-INF/init.rb:7
  require at org/jruby/RubyKernel.java:1040
   (root) at <script>:1
ERROR: org.jruby.embed.EvalFailedException: (LoadError) no such file to load -- rubygems

ANy idea what could cause such errors and how to resolve this?

Thanks, Arun

grtjn commented 7 years ago

I'm unfamiliar with the roxy command mlcpbatch, I only know ml {env} mlcp. I do know that -option_file is not a valid MLCP parameter, it should be -options_file. See also: http://docs.marklogic.com/guide/mlcp/import#id_23879

grtjn commented 7 years ago

Regarding the rubygem issue, I'm less familiar with JRuby, but according to https://stackoverflow.com/questions/2896485/no-such-file-to-load-rubygems-loaderror, and https://stackoverflow.com/questions/17448241/cant-run-unit-test-using-jruby-no-such-file-to-load-rubygems, it seems to come from the fact that the Ruby or JRuby you are running either cannot find gem and its libraries, or it is not included/installed properly.

Please look at above links to see if the suggested solutions work for you. Let us know if you found a solution, and we may be able to improve our roxy.jar using that.