search

mc2-project / mc2

A Platform for Secure Analytics and Machine Learning
Apache License 2.0
297 stars 43 forks source link

Handle and log opaque sql error #170

Closed mc2-bot closed 3 years ago

mc2-bot commented 3 years ago

Handles any error thrown by Opaque SQL when submitting a query. Previously, even upon error, the client would exit and log success. This PR logs an error if one is thrown by Opaque SQL, and also prints out the stack trace on the compute side (which is returned by Opaque SQL).

The result looks something like

2021-07-20 23:59:30 - ERROR - Error executing Opaque SQL query:                                                                                                                                                       
OpaqueSQLError: org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory file:/tmp/opaque_sql_result/data already exists                                                                                 
  at org.apache.hadoop.mapred.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:131)                                                                                                                            
  at org.apache.spark.internal.io.HadoopMapRedWriteConfigUtil.assertConf(SparkHadoopWriter.scala:298)                                                                                                                 
  at org.apache.spark.internal.io.SparkHadoopWriter$.write(SparkHadoopWriter.scala:71)                                                                                                                                
  at org.apache.spark.rdd.PairRDDFunctions.$anonfun$saveAsHadoopDataset$1(PairRDDFunctions.scala:1090)

Co-authored-by: Chester Leung chestercleung@gmail.com