Add Petastorm/Parquet support, refactor Server

[amacati]

Adds support for Petastorm/Parquet files as datasets and refactors the Server code.

Dataloader change (patching.py):

• Maggy's DataLoader now supports Petastorm datasets. Also supports normal Parquet datasets with Petastorm
• Changed the way datasets automatically get transferred to the GPU. Since Petastorm/Parquet or .csv datasets might consist of multiple nested dictionaries or lists, the DataLoader now recurses into these structures until the underlying torch.Tensor is found. This of course assumes that the encapsulated data is still given as a tensor

Server change (rpc.py): The server previously contained the message processing logic for all experiment types. The refactor aims for increased modularity in the RPC communication. It makes the addition of possible future experiments easy without adding further complexity to the base server.

• Server is now a base class that invokes a collection of registered callbacks
• Each experiment inherits the server and implements its own message callbacks

[moritzmeister]

I have to admit, I do not 100% follow the PetaStorm Dataloader, why you need to traverse through the data.

[amacati]

Are you talking about the _to_cuda(data) function? Petastorm for example returns the loaded Parquet as dicts of torch.Tensors, and these tensors need to be transferred to the GPU in order to work with DDP. Same goes for custom DataLoader classes from users, sometimes they might return data as dicts or lists, or even nested lists etc. In that case the wrapper recurses into these lists/dicts and moves the underlying torch.Tensors to the GPU. An example: Say you have a custom DataLoader that returns the x and y values of some function as input and z values as targets. The DataLoader does not have any .to(device) calls because it was written in a local notebook and executed on the CPU. Then this code still works in Maggy, because of the _to_cuda call.

for data in train_loader:
    x, y, z = data['x'], data['y'], data['z']   
    inputs = torch.stack((x,y)).T  
    outputs = model(inputs)  # Model is on GPU
    loss = criterion(outputs, z.unsqueeze(-1))  

But it's not really traversing the whole data but rather the structure of the output from the DataLoader. There should be no associated performance hit, you'd have to issue the .cuda() calls anyways to get DDP to work.