search

JuliaAI / MLJTestIntegration.jl

Utilities to test implementations of the MLJ model interface and provide integration tests for the MLJ ecosystem
MIT License
4 stars 1 forks source link

failures are not very legible #22

Open ExpandingMan opened 2 years ago

ExpandingMan commented 2 years ago

For example

XGBoostClassifier: Test Failed at /home/expandingman/.julia/dev/MLJXGBoostInterface/test/runtests.jl:235
  Expression: isempty(failures)
   Evaluated: isempty(NamedTuple{(:name, :package_name, :test, :exception), NTuple{4, Any}}[NamedTuple{(:name, :package_name, :test, :exception), NTuple{4, Any}}(("XGBoostClassifier", "XGBoost", "fitted_machine", ErrorException("Call to XGBoost C function XGBoosterUpdateOneIter failed: [18:48:00] /workspace/srcdir/xgboost/src/objective/regression_obj.cu:36: Check failed: info.labels.Size() == preds.Size() (200 vs. 400) : Invalid shape of labels.\nStack trace:\n  [bt] (0) /home/expandingman/.julia/artifacts/501324fbf7eec8ade7ee93955843c5cbdbd43fdc/lib/libxgboost.so(+0x32baa7) [0x7f9f5cfa9aa7]\n  [bt] (1) /home/expandingman/.julia/artifacts/501324fbf7eec8ade7ee93955843c5cbdbd43fdc/lib/libxgboost.so(+0x32bd53) [0x7f9f5cfa9d53]\n  [bt] (2) /home/expandingman/.julia/artifacts/501324fbf7eec8ade7ee93955843c5cbdbd43fdc/lib/libxgboost.so(xgboost::obj::RegLossObj<xgboost::obj::LogisticClassification>::GetGradient(xgboost::HostDeviceVector<float> const&, xgboost::MetaInfo const&, int, xgboost::HostDeviceVector<xgboost::detail::GradientPairInternal<float> >*)+0x29) [0x7f9f5cfbcf49]\n  [bt] (3) /home/expandingman/.julia/artifacts/501324fbf7eec8ade7ee93955843c5cbdbd43fdc/lib/libxgboost.so(xgboost::LearnerImpl::UpdateOneIter(int, std::shared_ptr<xgboost::DMatrix>)+0x295) [0x7f9f5cef1f15]\n  [bt] (4) /home/expandingman/.julia/artifacts/501324fbf7eec8ade7ee93955843c5cbdbd43fdc/lib/libxgboost.so(XGBoosterUpdateOneIter+0x42) [0x7f9f5cd99ba2]\n  [bt] (5) [0x7fa0047cb7a8]\n  [bt] (6) [0x7fa0047cc32b]\n  [bt] (7) [0x7fa0047cd11b]\n  [bt] (8) [0x7fa0047cd1ac]\n\n")))])

It doesn't help that the Test stdlib doesn't have very good printing which I believe is producing this output, however the situation is exacerbated with these integration tests because it's harder to track down the exact test that failed.

I suggest that some try catching to print errors more legibly may be appropriate.

ablaom commented 2 years ago

Have you tried using the throw=true option to force throwing of the error? I usually find this suffices in tracking down specific errors.