Closed soumalya-hue closed 5 years ago
Please could you help me with this issue as soon as possible, Our production model deployment is stopped due to this is issue!!! The issue does not come when we set numRound =1 but as soon as we change to numRound =2,we get this issue
Did you try using latest version (0.90)?
Nope,can you tell me is it compatible with 2.3.2.3.1.0.0-78 and which jars should I use , if I am manually downloading the jars for xgboost?
Actually, 0.90 requires Spark 2.4. You should use 0.82 instead.
Can you provide me a guiding link to install 0.82 xgboost , so that I can run a similar code in Spark 2.3.2.3.1.0.0-78?
According to https://xgboost.readthedocs.io/en/latest/jvm/xgboost4j_spark_tutorial.html#refer-to-xgboost4j-spark-dependency, you can specify XGBoost4J-Spark as a dependency in pom.xml. In general, all recent versions of XGBoost4J-Spark is available on Maven Central.
Stack trace returned 10 entries: [bt] (0) /hadoop/yarn/local/usercache/spark/appcache/application_1571576194248_0016/container_e11_1571576194248_0016_02_000003/tmp/libxgboost4j2340608630152808150.so(dmlc::StackTrace()+0x19d) [0x7f00924af4dd] [bt] (1) /hadoop/yarn/local/usercache/spark/appcache/application_1571576194248_0016/container_e11_1571576194248_0016_02_000003/tmp/libxgboost4j2340608630152808150.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x18) [0x7f00924b0388] [bt] (2) /hadoop/yarn/local/usercache/spark/appcache/application_1571576194248_0016/container_e11_1571576194248_0016_02_000003/tmp/libxgboost4j2340608630152808150.so(xgboost::tree::CQHistMaker::HistEntry::Add(float, xgboost::detail::GradientPairInternal)+0x4ef) [0x7f0092610dcf]
[bt] (3) /hadoop/yarn/local/usercache/spark/appcache/application_1571576194248_0016/container_e11_1571576194248_0016_02_000003/tmp/libxgboost4j2340608630152808150.so(+0x1e829a) [0x7f009260829a]
[bt] (4) /hadoop/yarn/local/usercache/spark/appcache/application_1571576194248_0016/container_e11_1571576194248_0016_02_000003/tmp/libxgboost4j2340608630152808150.so(+0x1e8ac7) [0x7f0092608ac7]
[bt] (5) /hadoop/yarn/local/usercache/spark/appcache/application_1571576194248_0016/container_e11_1571576194248_0016_02_000003/tmp/libxgboost4j2340608630152808150.so(xgboost::tree::GlobalProposalHistMaker::CreateHist(std::vector<xgboost::detail::GradientPairInternal, std::allocator<xgboost::detail::GradientPairInternal > > const&, xgboost::DMatrix, std::vector<unsigned int, std::allocator > const&, xgboost::RegTree const&)+0x749) [0x7f0092616099]
[bt] (6) /hadoop/yarn/local/usercache/spark/appcache/application_1571576194248_0016/container_e11_1571576194248_0016_02_000003/tmp/libxgboost4j2340608630152808150.so(xgboost::tree::HistMaker::Update(std::vector<xgboost::detail::GradientPairInternal, std::allocator<xgboost::detail::GradientPairInternal > > const&, xgboost::DMatrix , xgboost::RegTree)+0x2d9) [0x7f00926183e9]
[bt] (7) /hadoop/yarn/local/usercache/spark/appcache/application_1571576194248_0016/container_e11_1571576194248_0016_02_000003/tmp/libxgboost4j2340608630152808150.so(xgboost::tree::HistMaker::Update(xgboost::HostDeviceVector<xgboost::detail::GradientPairInternal > , xgboost::DMatrix, std::vector<xgboost::RegTree, std::allocator<xgboost::RegTree> > const&)+0xa3) [0x7f009260a973]
[bt] (8) /hadoop/yarn/local/usercache/spark/appcache/application_1571576194248_0016/container_e11_1571576194248_0016_02_000003/tmp/libxgboost4j2340608630152808150.so(xgboost::gbm::GBTree::BoostNewTrees(xgboost::HostDeviceVector<xgboost::detail::GradientPairInternal > , xgboost::DMatrix, int, std::vector<std::unique_ptr<xgboost::RegTree, std::default_delete >, std::allocator<std::unique_ptr<xgboost::RegTree, std::default_delete > > > )+0x80b) [0x7f009253a4fb]
[bt] (9) /hadoop/yarn/local/usercache/spark/appcache/application_1571576194248_0016/container_e11_1571576194248_0016_02_000003/tmp/libxgboost4j2340608630152808150.so(xgboost::gbm::GBTree::DoBoost(xgboost::DMatrix, xgboost::HostDeviceVector<xgboost::detail::GradientPairInternal > , xgboost::ObjFunction*)+0x825) [0x7f009253af35]
####################################################################
The code is as follows: To get into spark shell I use : #################################################################### spark-shell --jars /home/ubuntu/xgboost4j-0.72.jar,/home/ubuntu/xgboost4j-spark-0.72.jar #################################################################### The current code that I am using is as follows: #################################################################### import java.util.Calendar import org.apache.log4j.{Level, Logger} import ml.dmlc.xgboost4j.scala.spark.XGBoost import org.apache.spark.ml.feature. import org.apache.spark.sql. import org.apache.spark.sql.functions.lit
val now=Calendar.getInstance() val date=java.time.LocalDate.now val currentHour = now.get(Calendar.HOUR_OF_DAY) val currentMinute = now.get(Calendar.MINUTE) val direct="./results/"+date+"-"+currentHour+"-"+currentMinute+"/" println(direct)
val dataset = spark.read.option("header", "true").option("inferSchema", true).csv("/pyspark_model_run/Existing_train_data_Prescoring_mod.csv") val datatest = spark.read.option("header", "true").option("inferSchema", true).csv("/pyspark_model_run/Existing_test_data_Prescoring_mod.csv")
dataset.cache() datatest.cache()
val df = dataset.na.fill(0).sample(true,0.7,10) val df_test = datatest.na.fill(0)
val header = df.columns.filter(!_.contains("train_dataMSISDN")).filter(!.contains("labels")) val assembler = new VectorAssembler().setInputCols(header).setOutputCol("features")
val train_DF0 = assembler.transform(df) val test_DF0 = assembler.transform(df_test)
println("VectorAssembler Done!")
val train = train_DF0.withColumn("label", df("labels").cast("double")).select("label", "features") // val test = test_DF0.withColumn("label", lit(1.0)).withColumnRenamed("Id","id").select("id", "label", "features") val test = test_DF0.withColumn("label", df_test("ts_label").cast("double")).withColumn("test_data_MSISDN", df_test("test_data_MSISDN").cast("double")).select("test_data_MSISDN","label","features")
// Split the data into training and test sets (30% held out for testing). val Array(trainingData, testData) = train.randomSplit(Array(0.7, 0.3), seed = 0)
println(trainingData.size)
// number of iterations val numRound = 2 val numWorkers = 1 // training parameters val paramMap = List( "eta" -> 0.01, "max_depth" -> 5, "min_child_weight" -> 1, "subsample" -> 1, "colsample_bytree" -> 1, "eval_metric" -> "auc", "objective" -> "binary:logistic").toMap println("Starting Xgboost ")
val xgBoostModelWithDF = XGBoost.trainWithDataFrame(trainingData, paramMap,round = numRound, nWorkers = numWorkers, useExternalMemory = true)
###################################################################