Efficient serialization by avoiding field names

[siddhartha-gadgil]

The problem

• While serialization as json with upickle is clean and easy, the generated files are very large - huge in the case of the map from paths to unique classes.
• Most of the file is field names.
• This means that when upack is used to binary serialize we get a smaller file, but only by a factor of 2/3.
• Another source of inefficiency is in a normal path, the base surface is stored separately for each normal arc. Actually it should be stored once for the entire collection of PL-Arcs

Solution

• For each of the case classes, have an associated compact type made of tuples (of Integers, BigDecimals, Doubles and Booleans) with two way conversions.
• A clean design is to have
  • a statement in each companion object of the form type Cpt = (Int, BigDecimal, Boolean) or equivalent.
  • a method in the companion object fromCpt(cpt: Cpt): SkewCurve or equivalent.
  • a method in the case class mapping to the corresponding compact types.
  • for the collections and maps we want to store associated to a surface, a special associated compact type avoiding duplication of the associated surface.
• In addition we should have round-trip tests to see that serializing and deserializing gives the same object.
• As these are built from standard scala types, they can be automatically serialized by upickle (with upack for efficiency) or something else (e.g. Boopickle).

[siddhartha-gadgil]

A related change in terms of code (but with different effect). A slight speedup was obtained by memoizing hashes (which are computed a lot since we use maps and group stuff). This is accomplished by using the following code (from StackOverflow) which memoizes the hash code:

override val hashCode: Int = scala.util.hashing.MurmurHash3.productHash(this)

While writing tuple methods for all the case classes, it will be worth doing this too.