search

intel / ai-containers

This repository contains Dockerfiles, scripts, yaml files, Helm charts, etc. used to scale out AI containers with versions of TensorFlow and PyTorch that have been optimized for Intel platforms. Scaling is done with python, Docker, kubernetes, kubeflow, cnvrg.io, Helm, and other container orchestration frameworks for use in the cloud and on-premise
https://intel.github.io/ai-containers/
Apache License 2.0
19 stars 15 forks source link

Bump the classical-ml group across 1 directory with 4 updates #193

Closed dependabot[bot] closed 2 weeks ago

dependabot[bot] commented 2 weeks ago

Bumps the classical-ml group with 4 updates in the /classical-ml directory: daal4py, scikit-learn-intelex, xgboost and jupyterlab.

Updates daal4py from 2024.4.0 to 2024.5.0

Commits


Updates scikit-learn-intelex from 2024.4.0 to 2024.5.0

Commits


Updates xgboost from 2.0.3 to 2.1.0

Release notes

Sourced from xgboost's releases.

Release 2.1.0 stable

2.1.0 (2024 Jun 20)

We are thrilled to announce the XGBoost 2.1 release. This note will start by summarizing some general changes and then highlighting specific package updates. As we are working on a new R interface, this release will not include the R package. We'll update the R package as soon as it's ready. Stay tuned!

Networking Improvements

An important ongoing work for XGBoost, which we've been collaborating on, is to support resilience for improved scaling and federated learning on various platforms. The existing networking library in XGBoost, adopted from the RABIT project, can no longer meet the feature demand. We've revamped the RABIT module in this release to pave the way for future development. The choice of using an in-house version instead of an existing library is due to the active development status with frequent new feature requests like loading extra plugins for federated learning. The new implementation features:

  • Both CPU and GPU communication (based on NCCL).
  • A reusable tracker for both the Python package and JVM packages. With the new release, the JVM packages no longer require Python as a runtime dependency.
  • Supports federated communication patterns for both CPU and GPU.
  • Supports timeout. The high-level interface parameter is currently hard-coded to 30 minutes, which we plan to improve.
  • Supports significantly more data types.
  • Supports thread-based workers.
  • Improved handling for worker errors, including better error messages when one of the peers dies during training.
  • Work with IPv6. Currently, this is only supported by the dask interface.
  • Built-in support for various operations like broadcast, allgatherV, allreduce, etc.

Related PRs (#9597, #9576, #9523, #9524, #9593, #9596, #9661, #10319, #10152, #10125, #10332, #10306, #10208, #10203, #10199, #9784, #9777, #9773, #9772, #9759, #9745, #9695, #9738, #9732, #9726, #9688, #9681, #9679, #9659, #9650, #9644, #9649, #9917, #9990, #10313, #10315, #10112, #9531, #10075, #9805, #10198, #10414).

The existing option of using MPI in RABIT is removed in the release. (#9525)

NCCL is now fetched from PyPI.

In the previous version, XGBoost statically linked NCCL, which significantly increased the binary size and led to hitting the PyPI repository limit. With the new release, we have made a significant improvement. The new release can now dynamically load NCCL from an external source, reducing the binary size. For the PyPI package, the nvidia-nccl-cu12 package will be fetched during installation. With more downstream packages reusing NCCL, we expect the user environments to be slimmer in the future as well. (#9796, #9804, #10447)

Parts of the Python package now require glibc 2.28+

Starting from 2.1.0, XGBoost Python package will be distributed in two variants:

  • manylinux_2_28: for recent Linux distros with glibc 2.28 or newer. This variant comes with all features enabled.
  • manylinux2014: for old Linux distros with glibc older than 2.28. This variant does not support GPU algorithms or federated learning.

The pip package manager will automatically choose the correct variant depending on your system.

Starting from May 31, 2025, we will stop distributing the manylinux2014 variant and exclusively distribute the manylinux_2_28 variant. We made this decision so that our CI/CD pipeline won't have depend on software components that reached end-of-life (such as CentOS 7). We strongly encourage everyone to migrate to recent Linux distros in order to use future versions of XGBoost.

Note. If you want to use GPU algorithms or federated learning on an older Linux distro, you have two alternatives:

  1. Upgrade to a recent Linux distro with glibc 2.28+. OR
  2. Build XGBoost from the source.

Multi-output

We continue the work on multi-target and vector leaf in this release:

  • Revise the support for custom objectives with a new API, XGBoosterTrainOneIter. This new function supports strided matrices and CUDA inputs. In addition, custom objectives now return the correct shape for prediction. (#9508)
  • The hinge objective now supports multi-target regression (#9850)
  • Fix the gain calculation with vector leaf (#9978)
  • Support graphviz plot for multi-target tree. (#10093)
  • Fix multi-output with alternating strategies. (#9933)

Please note that the feature is still in progress and not suitable for production use.

Federated Learning

... (truncated)

Changelog

Sourced from xgboost's changelog.

XGBoost Change Log

Starting from 2.1.0, release note is recorded in the documentation.

This file records the changes in xgboost library in reverse chronological order.

2.0.0 (2023 Aug 16)

We are excited to announce the release of XGBoost 2.0. This note will begin by covering some overall changes and then highlight specific updates to the package.

Initial work on multi-target trees with vector-leaf outputs

We have been working on vector-leaf tree models for multi-target regression, multi-label classification, and multi-class classification in version 2.0. Previously, XGBoost would build a separate model for each target. However, with this new feature that's still being developed, XGBoost can build one tree for all targets. The feature has multiple benefits and trade-offs compared to the existing approach. It can help prevent overfitting, produce smaller models, and build trees that consider the correlation between targets. In addition, users can combine vector leaf and scalar leaf trees during a training session using a callback. Please note that the feature is still a working in progress, and many parts are not yet available. See #9043 for the current status. Related PRs: (#8538, #8697, #8902, #8884, #8895, #8898, #8612, #8652, #8698, #8908, #8928, #8968, #8616, #8922, #8890, #8872, #8889, #9509) Please note that, only the hist (default) tree method on CPU can be used for building vector leaf trees at the moment.

New device parameter.

A new device parameter is set to replace the existing gpu_id, gpu_hist, gpu_predictor, cpu_predictor, gpu_coord_descent, and the PySpark specific parameter use_gpu. Onward, users need only the device parameter to select which device to run along with the ordinal of the device. For more information, please see our document page (https://xgboost.readthedocs.io/en/stable/parameter.html#general-parameters) . For example, with device="cuda", tree_method="hist", XGBoost will run the hist tree method on GPU. (#9363, #8528, #8604, #9354, #9274, #9243, #8896, #9129, #9362, #9402, #9385, #9398, #9390, #9386, #9412, #9507, #9536). The old behavior of gpu_hist is preserved but deprecated. In addition, the predictor parameter is removed.

hist is now the default tree method

Starting from 2.0, the hist tree method will be the default. In previous versions, XGBoost chooses approx or exact depending on the input data and training environment. The new default can help XGBoost train models more efficiently and consistently. (#9320, #9353)

GPU-based approx tree method

There's initial support for using the approx tree method on GPU. The performance of the approx is not yet well optimized but is feature complete except for the JVM packages. It can be accessed through the use of the parameter combination device="cuda", tree_method="approx". (#9414, #9399, #9478). Please note that the Scala-based Spark interface is not yet supported.

Optimize and bound the size of the histogram on CPU, to control memory footprint

XGBoost has a new parameter max_cached_hist_node for users to limit the CPU cache size for histograms. It can help prevent XGBoost from caching histograms too aggressively. Without the cache, performance is likely to decrease. However, the size of the cache grows exponentially with the depth of the tree. The limit can be crucial when growing deep trees. In most cases, users need not configure this parameter as it does not affect the model's accuracy. (#9455, #9441, #9440, #9427, #9400).

Along with the cache limit, XGBoost also reduces the memory usage of the hist and approx tree method on distributed systems by cutting the size of the cache by half. (#9433)

Improved external memory support

There is some exciting development around external memory support in XGBoost. It's still an experimental feature, but the performance has been significantly improved with the default hist tree method. We replaced the old file IO logic with memory map. In addition to performance, we have reduced CPU memory usage and added extensive documentation. Beginning from 2.0.0, we encourage users to try it with the hist tree method when the memory saving by QuantileDMatrix is not sufficient. (#9361, #9317, #9282, #9315, #8457)

Learning to rank

We created a brand-new implementation for the learning-to-rank task. With the latest version, XGBoost gained a set of new features for ranking task including:

  • A new parameter lambdarank_pair_method for choosing the pair construction strategy.
  • A new parameter lambdarank_num_pair_per_sample for controlling the number of samples for each group.
  • An experimental implementation of unbiased learning-to-rank, which can be accessed using the lambdarank_unbiased parameter.
  • Support for custom gain function with NDCG using the ndcg_exp_gain parameter.
  • Deterministic GPU computation for all objectives and metrics.
  • NDCG is now the default objective function.
  • Improved performance of metrics using caches.
  • Support scikit-learn utilities for XGBRanker.
  • Extensive documentation on how learning-to-rank works with XGBoost.

For more information, please see the tutorial. Related PRs: (#8771, #8692, #8783, #8789, #8790, #8859, #8887, #8893, #8906, #8931, #9075, #9015, #9381, #9336, #8822, #9222, #8984, #8785, #8786, #8768)

Automatically estimated intercept

... (truncated)

Commits


Updates jupyterlab from 4.2.2 to 4.2.3

Release notes

Sourced from jupyterlab's releases.

v4.2.3

4.2.3

(Full Changelog)

Bugs fixed

Documentation improvements

Contributors to this release

(GitHub contributors page for this release)

@​brichet | @​fcollonval | @​github-actions | @​jtpio | @​jupyterlab-probot | @​krassowski | @​lumberbot-app | @​meeseeksmachine | @​Mehak261124 | @​welcome

Changelog

Sourced from jupyterlab's changelog.

4.2.3

(Full Changelog)

Bugs fixed

Documentation improvements

Contributors to this release

(GitHub contributors page for this release)

@​brichet | @​fcollonval | @​github-actions | @​jtpio | @​jupyterlab-probot | @​krassowski | @​lumberbot-app | @​meeseeksmachine | @​Mehak261124 | @​welcome

Commits
  • 1e4b637 [ci skip] Publish 4.2.3
  • b601b10 Backport PR #16526: Fix the description for the main inline completer plugin ...
  • d13971b Backport PR #16508 on branch 4.2.x (Fix inline completer configure calls not ...
  • da74d4e Backport PR #16506: Update JupyterLab 3.x maintenance announcement (#16512)
  • a9bbbd4 Backport PR #16493: Fix the lines placeholder taking up too much space (#16509)
  • 9c22b5b Backport PR #16471: Use correct hub restart URL (#16502)
  • 81549df Backport PR #16482: Fix check link CI failure in README (time zone converter ...
  • 02b336e Backport PR #16476: Fix typo in documentation - spurious single quote prefix ...
  • fec0539 Backport PR #16465: Add a notice for Windows users to activate symbolic links...
  • See full diff in compare view


Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself) - `@dependabot ignore minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself) - `@dependabot ignore ` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself) - `@dependabot unignore ` will remove all of the ignore conditions of the specified dependency - `@dependabot unignore ` will remove the ignore condition of the specified dependency and ignore conditions
github-actions[bot] commented 2 weeks ago

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

OpenSSF Scorecard

PackageVersionScoreDetails
pip/jupyterlab 4.2.3 :green_circle: 5.8
Details
CheckScoreReason
Code-Review:green_circle: 9Found 23/25 approved changesets -- score normalized to 9
Maintained:green_circle: 1030 commit(s) and 21 issue activity found in the last 90 days -- score normalized to 10
License:green_circle: 9license file detected
CII-Best-Practices:warning: 2badge detected: InProgress
Signed-Releases:warning: 0Project has not signed or included provenance with any releases.
Branch-Protection:warning: -1internal error: error during GetBranch(4.2.x): error during branchesHandler.query: internal error: githubv4.Query: Resource not accessible by integration
Packaging:warning: -1packaging workflow not detected
Dangerous-Workflow:green_circle: 10no dangerous workflow patterns detected
Token-Permissions:warning: 0detected GitHub workflow tokens with excessive permissions
SAST:green_circle: 10SAST tool is run on all commits
Binary-Artifacts:green_circle: 10no binaries found in the repo
Security-Policy:green_circle: 10security policy file detected
Fuzzing:warning: 0project is not fuzzed
Pinned-Dependencies:warning: 0dependency not pinned by hash detected -- score normalized to 0
Vulnerabilities:warning: 28 existing vulnerabilities detected
pip/jupyterlab 4.2.2 :green_circle: 5.8
Details
CheckScoreReason
Code-Review:green_circle: 9Found 23/25 approved changesets -- score normalized to 9
Maintained:green_circle: 1030 commit(s) and 21 issue activity found in the last 90 days -- score normalized to 10
License:green_circle: 9license file detected
CII-Best-Practices:warning: 2badge detected: InProgress
Signed-Releases:warning: 0Project has not signed or included provenance with any releases.
Branch-Protection:warning: -1internal error: error during GetBranch(4.2.x): error during branchesHandler.query: internal error: githubv4.Query: Resource not accessible by integration
Packaging:warning: -1packaging workflow not detected
Dangerous-Workflow:green_circle: 10no dangerous workflow patterns detected
Token-Permissions:warning: 0detected GitHub workflow tokens with excessive permissions
SAST:green_circle: 10SAST tool is run on all commits
Binary-Artifacts:green_circle: 10no binaries found in the repo
Security-Policy:green_circle: 10security policy file detected
Fuzzing:warning: 0project is not fuzzed
Pinned-Dependencies:warning: 0dependency not pinned by hash detected -- score normalized to 0
Vulnerabilities:warning: 28 existing vulnerabilities detected
pip/daal4py 2024.5.0 :green_circle: 7.6
Details
CheckScoreReason
Code-Review:green_circle: 10all changesets reviewed
Maintained:green_circle: 1030 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 10
CII-Best-Practices:warning: 0no effort to earn an OpenSSF best practices badge detected
License:green_circle: 10license file detected
Branch-Protection:warning: -1internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration
Signed-Releases:warning: -1no releases found
Token-Permissions:green_circle: 10GitHub workflow tokens follow principle of least privilege
Security-Policy:green_circle: 10security policy file detected
Dangerous-Workflow:green_circle: 10no dangerous workflow patterns detected
Packaging:warning: -1packaging workflow not detected
SAST:warning: 0SAST tool is not run on all commits -- score normalized to 0
Binary-Artifacts:green_circle: 10no binaries found in the repo
Vulnerabilities:green_circle: 100 existing vulnerabilities detected
Fuzzing:warning: 0project is not fuzzed
Pinned-Dependencies:warning: 0dependency not pinned by hash detected -- score normalized to 0
pip/scikit-learn-intelex 2024.5.0 :green_circle: 7.6
Details
CheckScoreReason
Code-Review:green_circle: 10all changesets reviewed
Maintained:green_circle: 1030 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 10
CII-Best-Practices:warning: 0no effort to earn an OpenSSF best practices badge detected
License:green_circle: 10license file detected
Branch-Protection:warning: -1internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration
Signed-Releases:warning: -1no releases found
Token-Permissions:green_circle: 10GitHub workflow tokens follow principle of least privilege
Security-Policy:green_circle: 10security policy file detected
Dangerous-Workflow:green_circle: 10no dangerous workflow patterns detected
Packaging:warning: -1packaging workflow not detected
SAST:warning: 0SAST tool is not run on all commits -- score normalized to 0
Binary-Artifacts:green_circle: 10no binaries found in the repo
Vulnerabilities:green_circle: 100 existing vulnerabilities detected
Fuzzing:warning: 0project is not fuzzed
Pinned-Dependencies:warning: 0dependency not pinned by hash detected -- score normalized to 0
pip/xgboost 2.1.0 :green_circle: 6.5
Details
CheckScoreReason
Binary-Artifacts:green_circle: 10no binaries found in the repo
Branch-Protection:warning: -1internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration
CI-Tests:green_circle: 927 out of 29 merged PRs checked by a CI test -- score normalized to 9
CII-Best-Practices:warning: 0no effort to earn an OpenSSF best practices badge detected
Code-Review:green_circle: 7Found 21/29 approved changesets -- score normalized to 7
Contributors:green_circle: 10project has 56 contributing companies or organizations
Dangerous-Workflow:green_circle: 10no dangerous workflow patterns detected
Dependency-Update-Tool:green_circle: 10update tool detected
Fuzzing:warning: 0project is not fuzzed
License:green_circle: 10license file detected
Maintained:green_circle: 1030 commit(s) and 21 issue activity found in the last 90 days -- score normalized to 10
Packaging:green_circle: 10packaging workflow detected
Pinned-Dependencies:warning: 2dependency not pinned by hash detected -- score normalized to 2
SAST:warning: 0SAST tool is not run on all commits -- score normalized to 0
Security-Policy:green_circle: 10security policy file detected
Signed-Releases:warning: 0Project has not signed or included provenance with any releases.
Token-Permissions:warning: 0detected GitHub workflow tokens with excessive permissions
Vulnerabilities:green_circle: 100 existing vulnerabilities detected
pip/daal4py 2024.4.0 :green_circle: 7.6
Details
CheckScoreReason
Code-Review:green_circle: 10all changesets reviewed
Maintained:green_circle: 1030 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 10
CII-Best-Practices:warning: 0no effort to earn an OpenSSF best practices badge detected
License:green_circle: 10license file detected
Branch-Protection:warning: -1internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration
Signed-Releases:warning: -1no releases found
Token-Permissions:green_circle: 10GitHub workflow tokens follow principle of least privilege
Security-Policy:green_circle: 10security policy file detected
Dangerous-Workflow:green_circle: 10no dangerous workflow patterns detected
Packaging:warning: -1packaging workflow not detected
SAST:warning: 0SAST tool is not run on all commits -- score normalized to 0
Binary-Artifacts:green_circle: 10no binaries found in the repo
Vulnerabilities:green_circle: 100 existing vulnerabilities detected
Fuzzing:warning: 0project is not fuzzed
Pinned-Dependencies:warning: 0dependency not pinned by hash detected -- score normalized to 0
pip/scikit-learn-intelex 2024.4.0 :green_circle: 7.6
Details
CheckScoreReason
Code-Review:green_circle: 10all changesets reviewed
Maintained:green_circle: 1030 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 10
CII-Best-Practices:warning: 0no effort to earn an OpenSSF best practices badge detected
License:green_circle: 10license file detected
Branch-Protection:warning: -1internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration
Signed-Releases:warning: -1no releases found
Token-Permissions:green_circle: 10GitHub workflow tokens follow principle of least privilege
Security-Policy:green_circle: 10security policy file detected
Dangerous-Workflow:green_circle: 10no dangerous workflow patterns detected
Packaging:warning: -1packaging workflow not detected
SAST:warning: 0SAST tool is not run on all commits -- score normalized to 0
Binary-Artifacts:green_circle: 10no binaries found in the repo
Vulnerabilities:green_circle: 100 existing vulnerabilities detected
Fuzzing:warning: 0project is not fuzzed
Pinned-Dependencies:warning: 0dependency not pinned by hash detected -- score normalized to 0
pip/xgboost 2.0.3 :green_circle: 6.5
Details
CheckScoreReason
Binary-Artifacts:green_circle: 10no binaries found in the repo
Branch-Protection:warning: -1internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration
CI-Tests:green_circle: 927 out of 29 merged PRs checked by a CI test -- score normalized to 9
CII-Best-Practices:warning: 0no effort to earn an OpenSSF best practices badge detected
Code-Review:green_circle: 7Found 21/29 approved changesets -- score normalized to 7
Contributors:green_circle: 10project has 56 contributing companies or organizations
Dangerous-Workflow:green_circle: 10no dangerous workflow patterns detected
Dependency-Update-Tool:green_circle: 10update tool detected
Fuzzing:warning: 0project is not fuzzed
License:green_circle: 10license file detected
Maintained:green_circle: 1030 commit(s) and 21 issue activity found in the last 90 days -- score normalized to 10
Packaging:green_circle: 10packaging workflow detected
Pinned-Dependencies:warning: 2dependency not pinned by hash detected -- score normalized to 2
SAST:warning: 0SAST tool is not run on all commits -- score normalized to 0
Security-Policy:green_circle: 10security policy file detected
Signed-Releases:warning: 0Project has not signed or included provenance with any releases.
Token-Permissions:warning: 0detected GitHub workflow tokens with excessive permissions
Vulnerabilities:green_circle: 100 existing vulnerabilities detected

Scanned Manifest Files

classical-ml/jupyter-requirements.txt
  • jupyterlab@4.2.3
  • jupyterlab@4.2.2
classical-ml/requirements.txt
  • daal4py@2024.5.0
  • scikit-learn-intelex@2024.5.0
  • xgboost@2.1.0
  • daal4py@2024.4.0
  • scikit-learn-intelex@2024.4.0
  • xgboost@2.0.3
github-advanced-security[bot] commented 2 weeks ago

This pull request sets up GitHub code scanning for this repository. Once the scans have completed and the checks have passed, the analysis results for this pull request branch will appear on this overview. Once you merge this pull request, the 'Security' tab will show more code scanning analysis results (for example, for the default branch). Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results. For more information about GitHub code scanning, check out the documentation.