Fix a use-after-free bug in SaveInputOutputNamesToNodeMapping function by @snnn in microsoft/onnxruntime#18456 . The issue was found by AddressSanitizer.
ONNX Runtime v1.16.2
The patch release includes updates on:
Performance optimizations for Llama2 on CUDA EP and DirectML EP
Performance optimizations for Stable Diffusion XL model for CUDA EP
Align ONNX Runtime extensions inclusion in source and build
Limit per thread context to 1 in the TensorRT EP to avoid error caused by input shape changes
ONNX Runtime v1.16.0
General
Support for serialization of models >=2GB
APIs
New session option to disable default CPU EP fallback session.disable_cpu_ep_fallback
Java
Support for fp16 and bf16 tensors as inputs and outputs, along with utilities to convert between these and fp32 data. On JDK 20 and newer the fp16 conversion methods use the JDK's Float.float16ToFloat and Float.floatToFloat16 methods which can be hardware accelerated and vectorized on some platforms.
Support for external initializers so that large models that can be instantiated without filesystem access
C#
Expose OrtValue API as the new preferred API to run inference in C#. This reduces garbage and exposes direct native memory access via Slice like interfaces.
Make Float16 and BFloat16 full featured fp16 interfaces that support conversion and expose floating properties (e.g. IsNaN, IsInfinity, etc)
C++
Make Float16_t and BFloat16_t full featured fp16 interfaces that support conversion and expose floating properties (e.g. IsNaN, IsInfinity, etc)
Performance
Improve LLM quantization accuracy with smoothquant
... (truncated)
Commits
2ac381c Cherry pick SDXL demo update to 1.16.3 (#18496)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Bumps onnxruntime-web from 1.14.0 to 1.16.3.
Release notes
Sourced from onnxruntime-web's releases.
... (truncated)
Commits
2ac381c
Cherry pick SDXL demo update to 1.16.3 (#18496)de0e87e
1.16.3 patch release (#18491)0c5b95f
Cherry-pick LLaMA GQA mask to rel-1.16.2 (round 4) (#18350)8f06330
Cherry pick LLaMA or SDXL to 1.16.2 release (round 3) (#18323)0ccca88
Update eigen version (#18308)ad7cecb
Update eigen's URL (#18301)95c20d0
Cherry-pick two pipeline changes for the 1.16.2 patch release (#18249)27b0910
cherry pick resize grad pr (#18255)70b8cda
Cherry pick LLaMA to rel-1.16.2 (round 2) (#18245)2f57f1e
Some cherry-picks for the 1.16.2 release (#18218)Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting
@dependabot rebase
.Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show