Illegal instruction (core dumped)

[SeibertronSS]

When I use the MACE algorithm in the tabular classification task, Illegal instruction (core dumped) often appears. Why?

[yangwenz]

How many features are considered in this problem?

[SeibertronSS]

There is a maximum of 54 features. I thought about the problem caused by too many features, but when I shrink the features to two, I still get the error. One phenomenon is that this error does not appear when my model is XgBoost or LightGBM. But this error occurs when my model is RandomForest or MLP.

[yangwenz]

It seems an evironment issue: https://stackoverflow.com/questions/65631801/illegal-instructioncore-dumped-error-on-jetson-nano. MACE can handle 54 features pretty well unless there are too many (> 10k) feature values for some feature.

[SeibertronSS]

By the way, I run the program in a docker container.

[SeibertronSS]

It seems an evironment issue: https://stackoverflow.com/questions/65631801/illegal-instructioncore-dumped-error-on-jetson-nano. MACE can handle 54 features pretty well unless there are too many (> 10k) feature values for some feature.

I wonder, will MACE use the training set? Or does MACE only use the test set. There are a lot of samples in my training set, about 70k.

[yangwenz]

This post may help to resolve the issue: https://github.com/ultralytics/yolov3/issues/1826, related to cpu avx.

[yangwenz]

70k samples are not large, MACE can handle it well. You can also reduce the number of training samples by calling omnixai.sampler.tabular.Sampler (https://github.com/salesforce/OmniXAI/blob/main/omnixai/sampler/tabular.py), e.g., pick 10k samples when creating the explainer.

[SeibertronSS]

@yangwenz Thank you for your help. Seems to be a problem caused by insufficient CPU resources

[yangwenz]

No problem, feel free to contact us if there are other issues related to OmniXAI.

[SeibertronSS]

What operations did OmniXAI perform before doing the MACE interpretation? I'm doing some log printing in MACEExplainer, but the error is sent before these logs are printed. It seems that there is a problem with these preprocessing

[yangwenz]

What kind of errors you encountered?

[SeibertronSS]

Illegal instruction (core dumped)

[SeibertronSS]

😂😂

[yangwenz]

In the initialization step, MACE first runs the predict function on the training data and then builds a KNN index based on the features and predicted labels.

[SeibertronSS]

I think I should try single step debugging

[yangwenz]

If there is no problem when xgboost is applied, probably the problem does not come from MACE because MACE only uses the blackbox prediction function.

[SeibertronSS]

如果应用xgboost时没有问题，那么问题可能不是来自MACE，因为MACE只使用了黑盒预测功能。

I agree with you.

[SeibertronSS]

I found out that the cause of the problem is that the hnswlib package does not match the CPU. The solution is to clone the code of hnswlib and then use pip install . to reinstall hnswlib instead of using pip install hnswlib

[yangwenz]

Thanks a lot for the effort. We will provide more options for KNN search besides hnswlib in the future.