sql-machine-learning / sqlflow

Brings SQL and AI together.
https://sqlflow.org
Apache License 2.0
5.08k stars 697 forks source link

Enrich the log message for the investigation of bugs or job failures. #2889

Open brightcoder01 opened 4 years ago

brightcoder01 commented 4 years ago

Such as the bug reproduce step in issue #2888 , the error message for None value input for XGBoost is as follows:

Traceback (most recent call last):

  File "<stdin>", line 108, in <module>

  File "/opt/sqlflow/python/runtime/xgboost/train.py", line 170, in train

    for per_batch_dmatrix in dtrain:

  File "/opt/sqlflow/python/runtime/xgboost/dataset.py", line 89, in xgb_dataset

    raw_data_dir=raw_data_dir)

  File "/opt/sqlflow/python/runtime/xgboost/dataset.py", line 130, in dump_dmatrix

    feature_metas)

  File "/opt/sqlflow/python/runtime/db.py", line 350, in read_features_from_row

    feature_metas[name], name)

  File "/opt/sqlflow/python/runtime/db.py", line 188, in read_feature

    int_raw_val = INT64_TYPE(raw_val)

TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

It would be better that we can have more obvious log message such as telling which feature contains None Value can help the user or developer to resolve this failure.

PS: Welcome to add more cases in this issue.

brightcoder01 commented 4 years ago

Add one more case of TO RUN statement, no detailed logs for the runnable program:

2020/09/09 06:36:45 SQLFlow Step Execute:
SELECT * FROM iris.train
TO RUN sqlflow/runnable:v0.0.1
CMD "binning.py",
   "--dbname=iris",
   "--columns=sepal_length,sepal_width",
   "--bin_method=bucket,log_bucket",
   "--bin_num=10,5"
INTO train_binning_result;
message:<message:"runSQLProgram error: exit status 1" >
workflow step failed: runSQLProgram error: exit status 1