Closed creativeguitar closed 7 months ago
RTNeural currently has support for "automatically" loading network weights from json files that are exported with the script in RTNeural/python/model_utils.py
, which currently only supports saving for TensorFlow models. In general, for model weights saved in any other format, The user must implement a custom model loader. One example is the code that the GuitarML plugins use to load weights from a json file saved with PyTorch.
I'm not very familiar with the .pt
file format, but from reading the documentation, it looks like the .pt
file should contain a dictionary which should be translatable into a json object. That said, I'm not sure if PyTorch uses some special form of encoding that would need to be reversed in order to access the dictionary data directly.
I managed to extract the model weights from the .pt file. Is the information in this file in the correct format now?
model_verb = torch.load("reverb_full.pt", map_location="cpu").eval()
model_verb.state_dict()
edit: looking online, it looks like I have to convert my model to keras. I am currently searching how to do this
Actually, the OrderedDict you have there is very close. From there it should just be one step to convert to a JSON representation: https://www.google.com/amp/s/www.geeksforgeeks.org/how-to-convert-ordereddict-to-json/amp/
Sharing my progress just in case someone else out there is stuck with the same problem. I was able to convert the .pt file to a json file/format. However, the keys and format of my json file are very different compared to your example "neural_net_weights.json". Therefore, when I try to compile the program, I get an error.
from collections import OrderedDict
import json
#model_verb_cuda = torch.load("reverb_full.pt", map_location="cuda").eval()
model_verb_cpu = torch.load("reverb_full.pt", map_location="cpu").eval()
od1 = model_verb_cpu.state_dict()
od1 = OrderedDict({k: od1[k].detach().cpu().tolist() for k in od1})
od1 = json.dumps(od1)
with open('reverb_full.json', 'w') as outfile:
outfile.write(od1)
Yeah, that JSON file looks good! The next step from there will be to set up a model loading function to parse the JSON file and load the weights from there (see the GuitarML example linked above).
One other think I just noticed, is that the network architecture includes some FiLM adapor layers. RTNeural does not currently support this layer type (see the list of suppored layers), though it would definitely be cool to add support!
I am super new to this. Is there a way within your project to convert .pt files to json ? if so, once done, how do I import the new json file to the plugin ?