Thriftpy / thriftpy2

Pure python approach of Apache Thrift.
MIT License
567 stars 90 forks source link

add 'strict_decode' option for protocols #227

Closed aisk closed 9 months ago

aisk commented 9 months ago

The string field will be encoded into python string (formerly unicode in python2) via UTF-8 encoding, and if the field is not a valid UTF-8 string, current implementatiion will fallback to return the raw bytes.

As a result, there would be a pitfall for users: if someone don't validate the result type and just pass it to database or somewhere else, there would be a dirty data.

This change add a strict_decode flag when create protocol (or protocol factory) to progate the UnicodeDecodeError to users so the PRC call will be failed as expected.

The flag is not enabled by default for compatibility issue.

Protocols needs support:

Summary by CodeRabbit

aisk commented 9 months ago

@coderabbitai review

coderabbitai[bot] commented 9 months ago

Walkthrough

The changes across various files introduce a strict_decode parameter to functions and classes related to binary and compact protocols within a Python library, likely for handling Thrift serialization. This parameter enables strict decoding mode, where decoding errors result in exceptions. New test functions are added to ensure that enabling strict_decode leads to the expected UnicodeDecodeError when decoding operations encounter errors.

Changes

File Path Change Summary
tests/test_aio_protocol_binary.py
tests/test_aio_protocol_compact.py
tests/test_protocol_binary.py
tests/test_protocol_compact.py
tests/test_protocol_cybinary.py
Added new test functions test_strict_decode to verify behavior when strict_decode is True.
thriftpy2/contrib/aio/protocol/binary.py
thriftpy2/contrib/aio/protocol/compact.py
thriftpy2/protocol/binary.py
thriftpy2/protocol/compact.py
thriftpy2/protocol/cybin/cybin.pyx
Introduced strict_decode parameter to various functions and classes to control strict decoding behavior.

🐇✨ In the realm of code where the data streams flow,
A rabbit hopped in, with a strict_decode in tow.
"No more silent fails," it said with a cheer,
"For errors shall speak, whenever they're near!"
🎉🐰


Tips ### Chat with CodeRabbit Bot (`@coderabbitai`) - If you reply to a *review comment* from CodeRabbit, the bot will automatically respond. - To engage with CodeRabbit bot directly around the specific lines of code in the PR, mention @coderabbitai in your review comment - Note: Review comments are made on code diffs or files, not on the PR overview. - Add `@coderabbitai ignore` anywhere in the PR description to prevent this PR from being reviewed. ### CodeRabbit Commands (invoked as PR comments) - `@coderabbitai pause` to pause the reviews on a PR. - `@coderabbitai resume` to resume the paused reviews. - `@coderabbitai review` to trigger a review. This is useful when automatic reviews are disabled for the repository. - `@coderabbitai resolve` resolve all the CodeRabbit review comments. - `@coderabbitai help` to get help. Note: For conversation with the bot, please use the review comments on code diffs or files. ### CodeRabbit Configration File (`.coderabbit.yaml`) - You can programmatically configure CodeRabbit by adding a `.coderabbit.yaml` file to the root of your repository. - The JSON schema for the configuration file is available [here](https://coderabbit.ai/integrations/coderabbit-overrides.v2.json). - If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: `# yaml-language-server: $schema=https://coderabbit.ai/integrations/coderabbit-overrides.v2.json`