The cleaned up version of my prototype for sharded WorkUnit file storage.
these are the changes to just the WorkUnit, for reprojection we'll still need to rewrite the code to load in the data lazily per worker. However, a sharded WorkUnit that's eager loaded will work exactly the same as any other WorkUnit in memory.
The cleaned up version of my prototype for sharded WorkUnit file storage.
WorkUnit, for reprojection we'll still need to rewrite the code to load in the data lazily per worker. However, a shardedWorkUnitthat's eager loaded will work exactly the same as any otherWorkUnitin memory.