search

dmlc / xgboost

Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library, for Python, R, Java, Scala, C++ and more. Runs on single machine, Hadoop, Spark, Dask, Flink and DataFlow
https://xgboost.readthedocs.io/en/stable/
Apache License 2.0
26.07k stars 8.7k forks source link

multioutput is not supported by current objective function #8201

Closed tomkr000 closed 2 years ago

tomkr000 commented 2 years ago

On Macbook Air M2, so that might be the problem, however I get the below error using:

Init classifier

xgb_cl0 = xgb.XGBClassifier( objective= 'multi:softmax', tree_method='hist', max_depth=10, n_estimators=500, learning_rate=0.01, num_class=5 )

XGBoostError Traceback (most recent call last) Input In [19], in <cell line: 22>() 12 xgb_cl0 = xgb.XGBClassifier( 13 objective= 'multi:softmax', 14 tree_method='hist', (...) 18 num_class=5 19 ) 21 # Fit ---> 22 xgb_cl0.fit(X_train, y_train) 24 # Predict 25 preds = xgb_cl0.predict(X_test)

File ~/miniforge3/envs/omg/lib/python3.8/site-packages/xgboost/core.py:575, in _deprecate_positional_args..inner_f(*args, kwargs) 573 for k, arg in zip(sig.parameters, args): 574 kwargs[k] = arg --> 575 return f(kwargs)

File ~/miniforge3/envs/omg/lib/python3.8/site-packages/xgboost/sklearn.py:1400, in XGBClassifier.fit(self, X, y, sample_weight, base_margin, eval_set, eval_metric, early_stopping_rounds, verbose, xgb_model, sample_weight_eval_set, base_margin_eval_set, feature_weights, callbacks) 1379 model, metric, params, early_stopping_rounds, callbacks = self._configure_fit( 1380 xgb_model, eval_metric, params, early_stopping_rounds, callbacks 1381 ) 1382 train_dmatrix, evals = _wrap_evaluation_matrices( 1383 missing=self.missing, 1384 X=X, (...) 1397 enable_categorical=self.enable_categorical, 1398 ) -> 1400 self._Booster = train( 1401 params, 1402 train_dmatrix, 1403 self.get_num_boosting_rounds(), 1404 evals=evals, 1405 early_stopping_rounds=early_stopping_rounds, 1406 evals_result=evals_result, 1407 obj=obj, 1408 custom_metric=metric, 1409 verbose_eval=verbose, 1410 xgb_model=model, 1411 callbacks=callbacks, 1412 ) 1414 if not callable(self.objective): 1415 self.objective = params["objective"]

File ~/miniforge3/envs/omg/lib/python3.8/site-packages/xgboost/core.py:575, in _deprecate_positional_args..inner_f(*args, kwargs) 573 for k, arg in zip(sig.parameters, args): 574 kwargs[k] = arg --> 575 return f(kwargs)

File ~/miniforge3/envs/omg/lib/python3.8/site-packages/xgboost/training.py:181, in train(params, dtrain, num_boost_round, evals, obj, feval, maximize, early_stopping_rounds, evals_result, verbose_eval, xgb_model, callbacks, custom_metric) 179 if cb_container.before_iteration(bst, i, dtrain, evals): 180 break --> 181 bst.update(dtrain, i, obj) 182 if cb_container.after_iteration(bst, i, dtrain, evals): 183 break

File ~/miniforge3/envs/omg/lib/python3.8/site-packages/xgboost/core.py:1778, in Booster.update(self, dtrain, iteration, fobj) 1775 self._validate_features(dtrain) 1777 if fobj is None: -> 1778 _check_call(_LIB.XGBoosterUpdateOneIter(self.handle, 1779 ctypes.c_int(iteration), 1780 dtrain.handle)) 1781 else: 1782 pred = self.predict(dtrain, output_margin=True, training=True)

File ~/miniforge3/envs/omg/lib/python3.8/site-packages/xgboost/core.py:246, in _check_call(ret) 235 """Check the return value of C API call 236 237 This function will raise exception when error occurs. (...) 243 return value from API calls 244 """ 245 if ret != 0: --> 246 raise XGBoostError(py_str(_LIB.XGBGetLastError()))

XGBoostError: [23:05:54] /Users/runner/work/xgboost/xgboost/python-package/build/temp.macosx-11.0-arm64-cpython-38/xgboost/include/xgboost/objective.h:86: multioutput is not supported by current objective function Stack trace: [bt] (0) 1 libxgboost.dylib 0x000000015bb447a8 dmlc::LogMessageFatal::~LogMessageFatal() + 124 [bt] (1) 2 libxgboost.dylib 0x000000015bc4998c xgboost::ObjFunction::Targets(xgboost::MetaInfo const&) const + 84 [bt] (2) 3 libxgboost.dylib 0x000000015bc0b304 xgboost::LearnerConfiguration::ConfigureTargets() + 160 [bt] (3) 4 libxgboost.dylib 0x000000015bc00820 xgboost::LearnerConfiguration::Configure() + 924 [bt] (4) 5 libxgboost.dylib 0x000000015bc00b9c xgboost::LearnerImpl::UpdateOneIter(int, std::__1::shared_ptr) + 128 [bt] (5) 6 libxgboost.dylib 0x000000015bb48524 XGBoosterUpdateOneIter + 140 [bt] (6) 7 libffi.8.dylib 0x000000010380804c ffi_call_SYSV + 76 [bt] (7) 8 libffi.8.dylib 0x000000010380574c ffi_call_int + 1208 [bt] (8) 9 _ctypes.cpython-38-darwin.so 0x0000000102fd451c _ctypes_callproc + 1196

`objective'= 'binary:logistic' works, but need multi-class as binary:logistic results in multiple classes selected.

pip freeze:

absl-py==1.2.0 aiosignal==1.2.0 appnope @ file:///home/conda/feedstock_root/build_artifacts/appnope_1649077682618/work argon2-cffi @ file:///home/conda/feedstock_root/build_artifacts/argon2-cffi_1640817743617/work argon2-cffi-bindings @ file:///Users/runner/miniforge3/conda-bld/argon2-cffi-bindings_1649500376817/work asttokens @ file:///home/conda/feedstock_root/build_artifacts/asttokens_1660605382950/work astunparse==1.6.3 attrs @ file:///home/conda/feedstock_root/build_artifacts/attrs_1659291887007/work backcall @ file:///home/conda/feedstock_root/build_artifacts/backcall_1592338393461/work backports.functools-lru-cache @ file:///home/conda/feedstock_root/build_artifacts/backports.functools_lru_cache_1618230623929/work beautifulsoup4 @ file:///home/conda/feedstock_root/build_artifacts/beautifulsoup4_1649463573192/work bleach @ file:///home/conda/feedstock_root/build_artifacts/bleach_1656355450470/work boto3==1.24.55 botocore==1.27.55 cached-property @ file:///home/conda/feedstock_root/build_artifacts/cached_property_1615209429212/work cachetools==5.2.0 certifi==2022.6.15 cffi @ file:///Users/runner/miniforge3/conda-bld/cffi_1656782895435/work charset-normalizer==2.1.0 click==8.0.4 cycler @ file:///home/conda/feedstock_root/build_artifacts/cycler_1635519461629/work debugpy @ file:///Users/runner/miniforge3/conda-bld/debugpy_1660619056192/work decorator @ file:///home/conda/feedstock_root/build_artifacts/decorator_1641555617451/work defusedxml @ file:///home/conda/feedstock_root/build_artifacts/defusedxml_1615232257335/work dill==0.3.5.1 distlib==0.3.5 entrypoints @ file:///home/conda/feedstock_root/build_artifacts/entrypoints_1643888246732/work etils==0.7.1 executing @ file:///home/conda/feedstock_root/build_artifacts/executing_1660472483358/work fastjsonschema @ file:///home/conda/feedstock_root/build_artifacts/python-fastjsonschema_1658064924516/work/dist filelock==3.8.0 flatbuffers==1.12 flit_core @ file:///home/conda/feedstock_root/build_artifacts/flit-core_1645629044586/work/source/flit_core fonttools @ file:///Users/runner/miniforge3/conda-bld/fonttools_1660773471789/work frozenlist==1.3.1 gast==0.4.0 getch==1.0 google-auth==2.10.0 google-auth-oauthlib==0.4.6 google-pasta==0.2.0 googleapis-common-protos==1.56.4 grpcio @ file:///Users/runner/miniforge3/conda-bld/grpc-split_1660613987933/work h5py @ file:///Users/runner/miniforge3/conda-bld/h5py_1637964070496/work idna==3.3 importlib-metadata==4.12.0 importlib-resources @ file:///home/conda/feedstock_root/build_artifacts/importlib_resources_1658604161399/work ipykernel @ file:///Users/runner/miniforge3/conda-bld/ipykernel_1657295113967/work ipympl==0.9.1 ipython @ file:///Users/runner/miniforge3/conda-bld/ipython_1653755018507/work ipython-genutils==0.2.0 ipywidgets @ file:///home/conda/feedstock_root/build_artifacts/ipywidgets_1655973868664/work jedi @ file:///home/conda/feedstock_root/build_artifacts/jedi_1659959867326/work Jinja2 @ file:///home/conda/feedstock_root/build_artifacts/jinja2_1654302431367/work jmespath==1.0.1 joblib @ file:///home/conda/feedstock_root/build_artifacts/joblib_1633637554808/work json5 @ file:///home/conda/feedstock_root/build_artifacts/json5_1600692310011/work jsonschema @ file:///home/conda/feedstock_root/build_artifacts/jsonschema-meta_1659525086692/work jupyter @ file:///Users/runner/miniforge3/conda-bld/jupyter_1637233406932/work jupyter-client @ file:///home/conda/feedstock_root/build_artifacts/jupyter_client_1654730843242/work jupyter-console @ file:///home/conda/feedstock_root/build_artifacts/jupyter_console_1655961255101/work jupyter_core @ file:///Users/runner/miniforge3/conda-bld/jupyter_core_1658332635171/work jupyterlab==2.1.0 jupyterlab-pygments @ file:///home/conda/feedstock_root/build_artifacts/jupyterlab_pygments_1649936611996/work jupyterlab-server @ file:///home/conda/feedstock_root/build_artifacts/jupyterlab_server_1589712729745/work jupyterlab-widgets @ file:///home/conda/feedstock_root/build_artifacts/jupyterlab_widgets_1655961217661/work keras==2.9.0 Keras-Preprocessing==1.1.2 kiwisolver @ file:///Users/runner/miniforge3/conda-bld/kiwisolver_1657953153960/work libclang==14.0.6 lightgbm @ file:///Users/runner/miniforge3/conda-bld/lightgbm_1641600046169/work Markdown==3.4.1 MarkupSafe @ file:///Users/runner/miniforge3/conda-bld/markupsafe_1648737587623/work matplotlib @ file:///Users/runner/miniforge3/conda-bld/matplotlib-suite_1660700873572/work matplotlib-inline @ file:///home/conda/feedstock_root/build_artifacts/matplotlib-inline_1660720192888/work mediapipe-silicon==0.8.10.1 mistune @ file:///Users/runner/miniforge3/conda-bld/mistune_1635845001896/work MouseInfo==0.1.3 msgpack==1.0.4 munkres==1.1.4 nbclient @ file:///home/conda/feedstock_root/build_artifacts/nbclient_1646999386773/work nbconvert @ file:///Users/runner/miniforge3/conda-bld/nbconvert_1648465228596/work nbformat @ file:///home/conda/feedstock_root/build_artifacts/nbformat_1651607001005/work nest-asyncio @ file:///home/conda/feedstock_root/build_artifacts/nest-asyncio_1648959695634/work notebook @ file:///home/conda/feedstock_root/build_artifacts/notebook_1654636967533/work numpy @ file:///Users/runner/miniforge3/conda-bld/numpy_1653326072482/work oauthlib==3.2.0 opencv-contrib-python==4.6.0.66 opt-einsum==3.3.0 packaging @ file:///home/conda/feedstock_root/build_artifacts/packaging_1637239678211/work pandas==1.4.3 pandocfilters @ file:///home/conda/feedstock_root/build_artifacts/pandocfilters_1631603243851/work parso @ file:///home/conda/feedstock_root/build_artifacts/parso_1638334955874/work pexpect @ file:///home/conda/feedstock_root/build_artifacts/pexpect_1602535608087/work pickleshare @ file:///home/conda/feedstock_root/build_artifacts/pickleshare_1602536217715/work Pillow @ file:///Users/runner/miniforge3/conda-bld/pillow_1660385908697/work pkgutil_resolve_name @ file:///home/conda/feedstock_root/build_artifacts/pkgutil-resolve-name_1633981968097/work platformdirs==2.5.2 prometheus-client @ file:///home/conda/feedstock_root/build_artifacts/prometheus_client_1649447152425/work promise==2.3 prompt-toolkit @ file:///home/conda/feedstock_root/build_artifacts/prompt-toolkit_1656332401605/work protobuf==3.19.4 psutil @ file:///Users/runner/miniforge3/conda-bld/psutil_1653089418571/work ptyprocess @ file:///home/conda/feedstock_root/build_artifacts/ptyprocess_1609419310487/work/dist/ptyprocess-0.7.0-py2.py3-none-any.whl pure-eval @ file:///home/conda/feedstock_root/build_artifacts/pure_eval_1642875951954/work pyasn1==0.4.8 pyasn1-modules==0.2.8 PyAutoGUI==0.9.53 pycparser @ file:///home/conda/feedstock_root/build_artifacts/pycparser_1636257122734/work PyGetWindow==0.0.9 Pygments @ file:///home/conda/feedstock_root/build_artifacts/pygments_1660666458521/work PyMsgBox==1.0.9 pyobjc==8.5 pyobjc-core==8.5 pyobjc-framework-Accessibility==8.5 pyobjc-framework-Accounts==8.5 pyobjc-framework-AddressBook==8.5 pyobjc-framework-AdServices==8.5 pyobjc-framework-AdSupport==8.5 pyobjc-framework-AppleScriptKit==8.5 pyobjc-framework-AppleScriptObjC==8.5 pyobjc-framework-ApplicationServices==8.5 pyobjc-framework-AppTrackingTransparency==8.5 pyobjc-framework-AudioVideoBridging==8.5 pyobjc-framework-AuthenticationServices==8.5 pyobjc-framework-AutomaticAssessmentConfiguration==8.5 pyobjc-framework-Automator==8.5 pyobjc-framework-AVFoundation==8.5 pyobjc-framework-AVKit==8.5 pyobjc-framework-BusinessChat==8.5 pyobjc-framework-CalendarStore==8.5 pyobjc-framework-CallKit==8.5 pyobjc-framework-CFNetwork==8.5 pyobjc-framework-ClassKit==8.5 pyobjc-framework-CloudKit==8.5 pyobjc-framework-Cocoa==8.5 pyobjc-framework-Collaboration==8.5 pyobjc-framework-ColorSync==8.5 pyobjc-framework-Contacts==8.5 pyobjc-framework-ContactsUI==8.5 pyobjc-framework-CoreAudio==8.5 pyobjc-framework-CoreAudioKit==8.5 pyobjc-framework-CoreBluetooth==8.5 pyobjc-framework-CoreData==8.5 pyobjc-framework-CoreHaptics==8.5 pyobjc-framework-CoreLocation==8.5 pyobjc-framework-CoreMedia==8.5 pyobjc-framework-CoreMediaIO==8.5 pyobjc-framework-CoreMIDI==8.5 pyobjc-framework-CoreML==8.5 pyobjc-framework-CoreMotion==8.5 pyobjc-framework-CoreServices==8.5 pyobjc-framework-CoreSpotlight==8.5 pyobjc-framework-CoreText==8.5 pyobjc-framework-CoreWLAN==8.5 pyobjc-framework-CryptoTokenKit==8.5 pyobjc-framework-DataDetection==8.5 pyobjc-framework-DeviceCheck==8.5 pyobjc-framework-DictionaryServices==8.5 pyobjc-framework-DiscRecording==8.5 pyobjc-framework-DiscRecordingUI==8.5 pyobjc-framework-DiskArbitration==8.5 pyobjc-framework-DVDPlayback==8.5 pyobjc-framework-EventKit==8.5 pyobjc-framework-ExceptionHandling==8.5 pyobjc-framework-ExecutionPolicy==8.5 pyobjc-framework-ExternalAccessory==8.5 pyobjc-framework-FileProvider==8.5 pyobjc-framework-FileProviderUI==8.5 pyobjc-framework-FinderSync==8.5 pyobjc-framework-FSEvents==8.5 pyobjc-framework-GameCenter==8.5 pyobjc-framework-GameController==8.5 pyobjc-framework-GameKit==8.5 pyobjc-framework-GameplayKit==8.5 pyobjc-framework-ImageCaptureCore==8.5 pyobjc-framework-IMServicePlugIn==8.5 pyobjc-framework-InputMethodKit==8.5 pyobjc-framework-InstallerPlugins==8.5 pyobjc-framework-InstantMessage==8.5 pyobjc-framework-Intents==8.5 pyobjc-framework-IntentsUI==8.5 pyobjc-framework-IOSurface==8.5 pyobjc-framework-iTunesLibrary==8.5 pyobjc-framework-KernelManagement==8.5 pyobjc-framework-LatentSemanticMapping==8.5 pyobjc-framework-LaunchServices==8.5 pyobjc-framework-libdispatch==8.5 pyobjc-framework-LinkPresentation==8.5 pyobjc-framework-LocalAuthentication==8.5 pyobjc-framework-LocalAuthenticationEmbeddedUI==8.5 pyobjc-framework-MailKit==8.5 pyobjc-framework-MapKit==8.5 pyobjc-framework-MediaAccessibility==8.5 pyobjc-framework-MediaLibrary==8.5 pyobjc-framework-MediaPlayer==8.5 pyobjc-framework-MediaToolbox==8.5 pyobjc-framework-Metal==8.5 pyobjc-framework-MetalKit==8.5 pyobjc-framework-MetalPerformanceShaders==8.5 pyobjc-framework-MetalPerformanceShadersGraph==8.5 pyobjc-framework-MetricKit==8.5 pyobjc-framework-MLCompute==8.5 pyobjc-framework-ModelIO==8.5 pyobjc-framework-MultipeerConnectivity==8.5 pyobjc-framework-NaturalLanguage==8.5 pyobjc-framework-NetFS==8.5 pyobjc-framework-Network==8.5 pyobjc-framework-NetworkExtension==8.5 pyobjc-framework-NotificationCenter==8.5 pyobjc-framework-OpenDirectory==8.5 pyobjc-framework-OSAKit==8.5 pyobjc-framework-OSLog==8.5 pyobjc-framework-PassKit==8.5 pyobjc-framework-PencilKit==8.5 pyobjc-framework-Photos==8.5 pyobjc-framework-PhotosUI==8.5 pyobjc-framework-PreferencePanes==8.5 pyobjc-framework-PushKit==8.5 pyobjc-framework-Quartz==8.5 pyobjc-framework-QuickLookThumbnailing==8.5 pyobjc-framework-ReplayKit==8.5 pyobjc-framework-SafariServices==8.5 pyobjc-framework-SceneKit==8.5 pyobjc-framework-ScreenCaptureKit==8.5 pyobjc-framework-ScreenSaver==8.5 pyobjc-framework-ScreenTime==8.5 pyobjc-framework-ScriptingBridge==8.5 pyobjc-framework-SearchKit==8.5 pyobjc-framework-Security==8.5 pyobjc-framework-SecurityFoundation==8.5 pyobjc-framework-SecurityInterface==8.5 pyobjc-framework-ServiceManagement==8.5 pyobjc-framework-ShazamKit==8.5 pyobjc-framework-Social==8.5 pyobjc-framework-SoundAnalysis==8.5 pyobjc-framework-Speech==8.5 pyobjc-framework-SpriteKit==8.5 pyobjc-framework-StoreKit==8.5 pyobjc-framework-SyncServices==8.5 pyobjc-framework-SystemConfiguration==8.5 pyobjc-framework-SystemExtensions==8.5 pyobjc-framework-UniformTypeIdentifiers==8.5 pyobjc-framework-UserNotifications==8.5 pyobjc-framework-UserNotificationsUI==8.5 pyobjc-framework-VideoSubscriberAccount==8.5 pyobjc-framework-VideoToolbox==8.5 pyobjc-framework-Virtualization==8.5 pyobjc-framework-Vision==8.5 pyobjc-framework-WebKit==8.5 pyparsing @ file:///home/conda/feedstock_root/build_artifacts/pyparsing_1652235407899/work pyperclip==1.8.2 PyRect==0.2.0 pyrsistent @ file:///Users/runner/miniforge3/conda-bld/pyrsistent_1649013444764/work PyScreeze==0.1.28 pyserial==3.5 python-dateutil @ file:///home/conda/feedstock_root/build_artifacts/python-dateutil_1626286286081/work pytweening==1.0.4 pytz @ file:///home/conda/feedstock_root/build_artifacts/pytz_1660580468234/work PyYAML==6.0 pyzmq @ file:///Users/runner/miniforge3/conda-bld/pyzmq_1660329122596/work ray==1.13.0 requests==2.28.1 requests-oauthlib==1.3.1 rsa==4.9 rubicon-objc==0.4.2 s3transfer==0.6.0 scikit-learn @ file:///Users/runner/miniforge3/conda-bld/scikit-learn_1659726160418/work scipy==1.9.0 seaborn==0.11.2 Send2Trash @ file:///home/conda/feedstock_root/build_artifacts/send2trash_1628511208346/work six @ file:///home/conda/feedstock_root/build_artifacts/six_1620240208055/work soupsieve @ file:///home/conda/feedstock_root/build_artifacts/soupsieve_1658207591808/work stack-data @ file:///home/conda/feedstock_root/build_artifacts/stack_data_1660472507741/work tensorboard==2.9.1 tensorboard-data-server==0.6.1 tensorboard-plugin-wit==1.8.1 tensorflow-datasets==4.6.0 tensorflow-estimator==2.9.0 tensorflow-macos==2.9.2 tensorflow-metadata==1.9.0 tensorflow-metal==0.5.1 termcolor==1.1.0 terminado @ file:///Users/runner/miniforge3/conda-bld/terminado_1652790673205/work testpath @ file:///home/conda/feedstock_root/build_artifacts/testpath_1645693042223/work threadpoolctl @ file:///home/conda/feedstock_root/build_artifacts/threadpoolctl_1643647933166/work toml==0.10.2 tornado @ file:///Users/runner/miniforge3/conda-bld/tornado_1656938134826/work tqdm==4.64.0 traitlets @ file:///home/conda/feedstock_root/build_artifacts/traitlets_1655411388954/work typing_extensions==4.3.0 unicodedata2 @ file:///Users/runner/miniforge3/conda-bld/unicodedata2_1649111980161/work urllib3==1.26.11 virtualenv==20.16.3 wcwidth @ file:///home/conda/feedstock_root/build_artifacts/wcwidth_1600965781394/work webencodings==0.5.1 Werkzeug==2.2.2 widgetsnbextension @ file:///home/conda/feedstock_root/build_artifacts/widgetsnbextension_1655939017940/work wrapt==1.14.1 xgboost==1.6.2 zipp @ file:///home/conda/feedstock_root/build_artifacts/zipp_1659400682470/work

trivialfis commented 2 years ago

What's the shape of your input label?

tomkr000 commented 2 years ago

y_train.shape = (774, 5) X_train.shape = (774, 720)

y_train is one hot encode

trivialfis commented 2 years ago

If you are performing multi-class classification, please provide the labels as one column with class id as the element. For instance, if you have 5 classes in the label, y would look like [0, 3, 1, 4 ...].

If you are performing multi-label classification, please don't specify the objective function (or specify it as binary:logistic, which is done inside XGBoost).