Handle UTF-8 Extra Fields Automatically

[ArcticLampyrid]

Goals

Ensure automatic handling of UTF-8 extra fields.
This feature is CRUCIAL for CJK users since most archives created on Windows utilize extra fields to store the UTF-8 path and comment.

Changes

Major

• Introduce an alternative field to ZipString, enabling it to represent a string stored in both UTF-8 and the legacy MBCS format. This provides an exact representation for entities that contain UTF-8 extra fields.
• Support reading and writing of UTF-8 extra fields when available.
• Manage the alternative field appropriately during both reading and writing operations.
• If neither EFS is set nor UTF-8 extra fields are detected, interpret data as ASCII, not UTF-8. This is because some strings in legacy MBCS encoding may appear as valid UTF-8, even if they aren't.

Minor

• Remove the private compute_crc function in entry_whole.rs, as the library function crc32fast::hash accomplishes the same task.
• Transition HeaderId from an enum to a struct with a base type of u16. This eliminates the redundant 2-byte internal field enum_tag. Although this might introduce a breaking change, adding an item to an enum can also cause a break (as per Rust RFC 1105) since we hadn't previously set the #[non_exhaustive] attribute.
• Set the #[non_exhaustive] attribute for the ExtraField enum. This ensures we won't introduce breaking changes in the future if we decide to add more items.

Notes on Breaking Changes

• The addition of certain fields and functions to ZipString will maintain compatibility.
• The shift of HeaderId from an enum to a struct may introduce breaking changes. However, since most users rarely interact with HeaderId, we anticipate this breaking change to be acceptable.
• We've added items to the ExtraField enum. As we hadn't previously assigned it the #[non_exhaustive] attribute, this modification will be considered a breaking change in accordance with Rust RFC 1105.

Related

Issue #103

Acknowledgement

Special thanks to ChatGPT by OpenAI for assisting in refining this report. Your language skills are greatly appreciated!

[ArcticLampyrid]

error: all variants have the same postfix: `ExtraField`
  --> src/spec/header.rs:53:1
   |
53 | / pub enum ExtraField {
54 | |     Zip64ExtendedInformationExtraField(Zip64ExtendedInformationExtraField),
55 | |     InfoZipUnicodeCommentExtraField(InfoZipUnicodeCommentExtraField),
56 | |     InfoZipUnicodePathExtraField(InfoZipUnicodePathExtraField),
57 | |     UnknownExtraField(UnknownExtraField),
58 | | }
   | |_^
   |
   = help: remove the postfixes and use full paths to the variants instead of glob imports
   = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#enum_variant_names

Clippy flags the ExtraField postfix as an issue, even though this naming convention is used previously, as seen in Zip64ExtendedInformationExtraField and UnknownExtraField. Should we consider removing it?

Based on the documentation, the lint regarding variant names is only triggered when there are a minimum of 3 enum variants. This might explain why we didn't receive any warnings earlier.

[Majored]

Apologies for the late response. Yep, if we could remove the postfixes, that would be perfect.

[ArcticLampyrid]

The back-to-school season just made me lazy. Sorry for my late commit. Postfixes are removed in the commit https://github.com/Majored/rs-async-zip/pull/104/commits/4b53dcde729e1914618ec907ba3fec3945337ce0.

[ArcticLampyrid]

I'm sorry to forget to format the code after renaming the fields. 🥲 I believe it should be okey now.

[Majored]

@ArcticLampyrid I've just merged #106 and it looks like that's caused a few conflicts.

Would you be happy to look into those? No worries if not.

[ArcticLampyrid]

Not to worry. I've resolved the conflicts and retested to ensure everything is in order.

[ArcticLampyrid]

Conflicts introduced from PR #110 are also resolved.

[Majored]

Merging, thanks for all your work on this. 👍