build: resource embedding

[petermetz]

Description

As a contributor I want to be able to embed arbitrary type of file resources within the build artifacts of ours so that they are accessible at runtime to the Typescript/NodeJS code without having to deal with the complex issue of file system paths that cannot be depended on even in a relative manner because the different directories where build files are placed such as the ./dist/lib/main/typescript/... path for the Typescript compiler and the ./dist/... path for webpack bundles.

We need a build step that goes through the ./src/main/resources/ and potentially also the ./src/test/resources directories and serializes recursively all files that are in there into JSON files with base64 encoded values. More of this is discussed in an earlier issue that came up with the original idea for this: #510

Acceptance Criteria

1. A pretsc npm script in each package that does the processing/filtering/serialization of resource files.
2. Decision about whether we want to only process the files in the resources directory or in all the other language folders as well such as ./src/main/json/ and ./src/main/solidity/ etc...
3. Test coverage for the serialization where the correct contents of files are verified at runtime by importing a test resource.

cc: @takeutak @sfuji822 @hartm @jonathan-m-hamilton @AzaharaC @jagpreetsinghsasan @jordigiam @kikoncuo

[petermetz]

@AzaharaC @jordigiam @kikoncuo

https://github.com/petermetz/cactus/blob/b05e7eb0854111fb945448ab1b207570db4e539e/tools/render-resources.js#L58

let outFilePath;
if (!path.isAbsolute(outputDir)) {
  outFilePath = path.join(process.cwd(), outputDir, `${entry}.json`);
} else {
  outFilePath = path.join(outputDir, entry, ".json");
}

Just FYI/dumping thoughts regarding the above linked section of the code: This part is a little different from what we have mentioned this morning (evening) in the sense that there is no single resources.json file but instead the original files get mapped down to their .json counterparts into separate files e.g. x.something becomes x.something.json. The idea behind this originally was that we could help bundle size optimizers (Rollup for example) to to cut down on unused assets (tree shaking compilation), BUT what I did not consider is that we may want to be able to loop through a set of resources without knowing in advance what those resources are going to be. Of course in this dynamic iteration there would be no compiler support, but that's not something we can actually solve IMO. We just have to think carefully every time we choose to use dynamic iteration of resources [2] instead of specific imports [1]

[1]:

import ResourcesJson from "../../json/resources.json";

// with these if one of the resource files disappears/gets deleted from the build 
// there will be a compilation error at build time which makes a whole class of
// bugs impossible to happen
const textualNotesBase64 = ResourcesJson["./my/textual-notes-about-stuff.txt"];
const superCoolChaincodeGoBase64 = ResourcesJson["./my/super-cool-chaincode.go"];

[2]:

import ResourcesJson from "../../json/resources.json";

for ([filePath, base64Contents] of ResourcesJson) {
  // do stuff
  // if one of the resources was not here in the loop than you won't find 
  // out about it at compile time, only at run time
}