coderabbitai[bot]
commented
1 month ago [!IMPORTANT]
Review Skipped
Bot user detected.
To trigger a single review, invoke the
@coderabbitai reviewcommand.You can disable this status message by setting the
reviews.review_statustofalsein the CodeRabbit configuration file.
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?
This PR contains the following updates:
v5.6.1->v5.7.0Release Notes
emqx/emqx (emqx/emqx)
### [`v5.7.0`](https://togithub.com/emqx/emqx/releases/tag/v5.7.0): EMQX v5.7.0 [Compare Source](https://togithub.com/emqx/emqx/compare/v5.6.1...e5.7.0-alpha.1) #### Enhancements ##### MQTT Implemented Durable Sessions, which persists MQTT Persistent Sessions and their messages to disk, and continuously replicates session metadata and MQTT messages among multiple nodes in the EMQX cluster. This achieves effective failover and recovery mechanisms, ensuring service continuity and high availability, thereby enhancing system reliability. Added metrics related to EMQX durable storage to Prometheus: - `emqx_ds_egress_batches` - `emqx_ds_egress_batches_retry` - `emqx_ds_egress_batches_failed` - `emqx_ds_egress_messages` - `emqx_ds_egress_bytes` - `emqx_ds_egress_flush_time` - `emqx_ds_store_batch_time` - `emqx_ds_builtin_next_time` - `emqx_ds_storage_bitfield_lts_counter_seek` - `emqx_ds_storage_bitfield_lts_counter_next` - `emqx_ds_storage_bitfield_lts_counter_collision` Note: these metrics are only visible when session persistence is enabled. The number of persisted messages has also been added to the Dashboard. ##### Security - [#12947](https://togithub.com/emqx/emqx/pull/12947) For JWT authentication, support new `disconnect_after_expire` option. When enabled, the client will be disconnected after the JWT token expires. Note: This is a breaking change. This option is enabled by default, so the default behavior is changed. Previously, the clients with actual JWTs could connect to the broker and stay connected even after the JWT token expired. Now, the client will be disconnected after the JWT token expires. To preserve the previous behavior, set `disconnect_after_expire` to `false`. ##### Data Processing and Integration - [#12671](https://togithub.com/emqx/emqx/pull/12671) An `unescape` function has been added to the rule engine SQL language to handle the expansion of escape sequences in strings. This addition has been done because string literals in the SQL language don't support any escape codes (e.g., `\n` and `\t`). This enhancement allows for more flexible string manipulation within SQL expressions. ##### Extensibility - [#12872](https://togithub.com/emqx/emqx/pull/12872) Implemented Client Attributes feature. It allows setting additional properties for each client using key-value pairs. Property values can be generated from MQTT client connection information (such as username, client ID, TLS certificate) or set from data accompanying successful authentication returns. Properties can be used in EMQX for authentication, authorization, data integration, and MQTT extension functions. Compared to using static properties like client ID directly, client properties offer greater flexibility in various business scenarios, simplifying the development process and enhancing adaptability and efficiency in development work. **Initialization of `client_attrs`** The `client_attrs` fields can be initially populated from one of the following `clientinfo` fields: - `cn`: The common name from the TLS client's certificate. - `dn`: The distinguished name from the TLS client's certificate, that is, the certificate "Subject". - `clientid`: The MQTT client ID provided by the client. - `username`: The username provided by the client. - `user_property`: Extract a property value from 'User-Property' of the MQTT CONNECT packet. **Extension through Authentication Responses** Additional attributes may be merged into `client_attrs` from authentication responses. Supported authentication backends include: - **HTTP**: Attributes can be included in the JSON object of the HTTP response body through a `client_attrs` field. - **JWT**: Attributes can be included via a `client_attrs` claim within the JWT. **Usage in Authentication and Authorization** If `client_attrs` is initialized before authentication, it can be used in external authentication requests. For instance, `${client_attrs.property1}` can be used within request templates directed at an HTTP server for authenticity validation. - The `client_attrs` can be utilized in authorization configurations or request templates, enhancing flexibility and control. Examples include: In `acl.conf`, use `{allow, all, all, ["${client_attrs.namespace}/#"]}` to apply permissions based on the `namespace` attribute. - In other authorization backends, `${client_attrs.namespace}` can be used within request templates to dynamically include client attributes. - [#12910](https://togithub.com/emqx/emqx/pull/12910) Added plugin configuration management and schema validation. For EMQX enterprise edition, one can also annotate the schema with metadata to facilitate UI rendering in the Dashboard. See more details in the [plugin template](https://togithub.com/emqx/emqx-plugin-template/pull/126) and plugin [documentation](../extensions/plugins.md). ##### Operations and Management - [#12923](https://togithub.com/emqx/emqx/pull/12923) Provided more specific error when importing wrong format into builtin authenticate database. - [#12940](https://togithub.com/emqx/emqx/pull/12940) Added `ignore_readonly` argument to `PUT /configs` API. Before this change, EMQX would return 400 (BAD_REQUEST) if the raw config included read-only root keys (`cluster`, `rpc`, and `node`). After this enhancement it can be called as `PUT /configs?ignore_readonly=true`, EMQX will in this case ignore readonly root config keys, and apply the rest. For observability purposes, an info level message is logged if any readonly keys are dropped. Also fixed an exception when config has bad HOCON syntax (returns 500). Now bad syntax will cause the API to return 400 (BAD_REQUEST). - [#12957](https://togithub.com/emqx/emqx/pull/12957) Started building packages for macOS 14 (Apple Silicon) and Ubuntu 24.04 Noble Numbat (LTS). #### Bug Fixes ##### Security - [#12887](https://togithub.com/emqx/emqx/pull/12887) Fixed MQTT enhanced auth with sasl scram. - [#12962](https://togithub.com/emqx/emqx/pull/12962) TLS clients can now verify server hostname against wildcard certificate. For example, if a certificate is issued for host `*.example.com`, TLS clients is able to verify server hostnames like `srv1.example.com`. ##### MQTT - [#12996](https://togithub.com/emqx/emqx/pull/12996) Fixed process leak in `emqx_retainer` application. Previously, client disconnection while receiving retained messages could cause a process leak. ##### Data Processing and Integration - [#12653](https://togithub.com/emqx/emqx/pull/12653) The rule engine function `bin2hexstr` now supports bitstring inputs with a bit size that is not divisible by 8. Such bitstrings can be returned by the rule engine function `subbits`. - [#12657](https://togithub.com/emqx/emqx/pull/12657) The rule engine SQL-based language previously did not allow putting any expressions as array elements in array literals (only constants and variable references were allowed). This has now been fixed so that one can use any expressions as array elements. The following is now permitted, for example: ```bash select [21 + 21, abs(-abs(-2)), [1 + 1], 4] as my_array from "t/#" ``` - [#12932](https://togithub.com/emqx/emqx/pull/12932) Previously, if a HTTP action request received a 503 (Service Unavailable) status, it was marked as a failure and the request was not retried. This has now been fixed so that the request is retried a configurable number of times. - [#12948](https://togithub.com/emqx/emqx/pull/12948) Fixed an issue where sensitive HTTP header values like `Authorization` would be substituted by `******` after updating a connector. - [#13118](https://togithub.com/emqx/emqx/pull/13118) Fix a performance issue in the rule engine template rendering. ##### Observability - [#12765](https://togithub.com/emqx/emqx/pull/12765) Make sure stats `subscribers.count` `subscribers.max` contains shared-subscribers. It only contains non-shared subscribers previously. ##### Operations and Management - [#12812](https://togithub.com/emqx/emqx/pull/12812) Made resource health checks non-blocking operations. This means that operations such as updating or removing a resource won't be blocked by a lengthy running health check. - [#12830](https://togithub.com/emqx/emqx/pull/12830) Made channel (action/source) health checks non-blocking operations. This means that operations such as updating or removing an action/source data integration won't be blocked by a lengthy running health check. - [#12993](https://togithub.com/emqx/emqx/pull/12993) Fixed listener config update API when handling an unknown zone. Before this fix, when a listener config is updated with an unknown zone, for example `{"zone": "unknown"}`, the change would be accepted, causing all clients to crash whens connected. After this fix, updating the listener with an unknown zone name will get a "Bad request" response. - [#13012](https://togithub.com/emqx/emqx/pull/13012) The MQTT listerners config option `access_rules` has been improved in the following ways: - The listener no longer crash with an incomprehensible error message if a non-valid access rule is configured. Instead a configuration error is generated. - One can now add several rules in a single string by separating them by comma (for example, "allow 10.0.1.0/24, deny all"). - [#13041](https://togithub.com/emqx/emqx/pull/13041) Improved HTTP authentication error log message. If HTTP content-type header is missing for POST method, it now emits a meaningful error message instead of a less readable exception with stack trace. - [#13077](https://togithub.com/emqx/emqx/pull/13077) This fix makes EMQX only read action configurations from the global configuration when the connector starts or restarts, and instead stores the latest configurations for the actions in the connector. Previously, updates to action configurations would sometimes not take effect without disabling and enabling the action. This means that an action could sometimes run with the old (previous) configuration even though it would look like the action configuration has been updated successfully. - [#13090](https://togithub.com/emqx/emqx/pull/13090) Attempting to start an action or source whose connector is disabled will no longer attempt to start the connector itself. ##### Gateways - [#12909](https://togithub.com/emqx/emqx/pull/12909) Fixed UDP listener process handling on errors or closure, The fix ensures the UDP listener is cleanly stopped and restarted as needed if these error conditions occur. - [#13001](https://togithub.com/emqx/emqx/pull/13001) Fixed an issue where the syskeeper forwarder would never reconnect when the connection was lost. - [#13010](https://togithub.com/emqx/emqx/pull/13010) Fixed the issue where the JT/T 808 gateway could not correctly reply to the REGISTER_ACK message when requesting authentication from the registration service failed. #### Breaking Changes - [#12947](https://togithub.com/emqx/emqx/pull/12947) For JWT authentication, a new boolean option `disconnect_after_expire` has been added with default value set to `true`. When enabled, the client will be disconnected after the JWT token expires. Previously, the clients with actual JWTs could connect to the broker and stay connected even after the JWT token expired. Now, the client will be disconnected after the JWT token expires. To preserve the previous behavior, set `disconnect_after_expire` to `false`. - [#12957](https://togithub.com/emqx/emqx/pull/12957) Stopped building packages for macOS 12.Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Enabled.
♻ Rebasing: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
This PR has been generated by Mend Renovate. View repository job log here.