Update to bitcode@0.4.0.

[caibear]

Updates to bitcode 0.4.0 which includes both speed and size optimizations.

Additionally some bitcode_hint attributes were moved to their respective fields (which improves performance due to a limitation with bitcode's derive macro).

[djkoloski]

There isn't a clear policy surrounding this, but in general I prefer to avoid crate-specific "hints" and other optimizations. This is mostly just to avoid gaming the benchmarks, but I think that it also reflects the default performance that users can expect from a crate. Do you think that bitcode would be improperly represented in the benchmarks without these hints? i.e. Without the hints, do we get an incorrect general view of bitcode's performance?

[finnbear]

Thanks for responding!

I prefer to avoid crate-specific "hints" and other optimizations. This is mostly just to avoid gaming the benchmarks, but I think that it also reflects the default performance that users can expect from a crate.

I totally agree the benchmarks should not be gamed in any way and that, to the extent possible, they should represent the performance real users can expect to see!

That said, I think the term "default performance" is entirely subjective because performance depends on:

• How the data are distributed:
  • Are are values equally likely or are only a few very small values possible?
  • Do the assumptions made by the format match the distribution of the data?
• Whether optimizations are in place:
  • Using enums instead of strings when appropriate
  • Not using unnecessarily large integer types
  • Using available hints when the data distribution is known in advance

As for "default," some formats require:

• creating protocol files
• installing specialized protocol compilers
• manually implementing functions
• running compression algorithms to get competitive file sizes

I think some optimization is a fair ask for users who care about performance. In other words, the recommended way to integrate bitcode is not just to import the crate and derive the traits, but to

• spend a few minutes replacing string with enums where appropriate
• apply domain knowledge of your dataset into minimal width integers and/or range hints

Without the hints, do we get an incorrect general view of bitcode's performance?

There are a few areas in which I think that the benchmarks unrealistically hold back non-hinted bitcode:

• Log
  • Using strings instead of numbers/enums for month and timezone (collectively the date) and http method/protocol
  • Using a u16 when there are only 63 possible response codes
  • Storing a number that maxes out at 100,000,000 in a u64 helps encodings that do var-int by default
• Mesh
  • Randomly generated floats are always between 0 and 1, allowing compression schemes on bytewise formats to take advantage, while not timing the compression in the speed benchmarks.
• Minecraft Save Data
  • Using strings instead of enums for item/entity ids, recipes, dimensions

The currently-existing bitcode hints can only recover some of the performance lost due to the above.

TL;DR: To get the most realistic view of bitcode's performance (in order of importance):

1. apply the aforementioned optimizations
2. include total time (serialization and compression) next to compressed sizes for all formats
3. do come up with an explicit policy on hinting/optimizing
4. consistently apply that policy, including with respect to bitcode

[djkoloski]

Thanks for the detailed response, I really appreciate the thought and care you put into it. I'm going to move your very helpful comment to a separate issue so we can have a dedicated space for discussion. I think the hints you have right now are acceptable, I'll let you know if that changes in the future.