lessw2020
commented
4 years ago so FastAI creates 2 param groups to split out l1 and l2 params....I've made a temp function to avoid that:
`def filter_all_params_no_split(layer_groups:Collection[nn.Module])->List[List[nn.Parameter]]:
pure = []
buffer=[]
for l in layer_groups:
for c in l.children():
buffer +=list(trainable_params(c))
pure += [uniqueify(buffer)]
return pure ` though now hitting other issues inside SLS but I still think it's vital that SLS be able to handle param groups as that's the default for most optimizer code...
IssamLaradji
I'm hitting an issue though in using/testing as the code seems to assume no parameter groups?
(from utils.py) def get_grad_list(params): return [p.grad for p in params]
this fails b/c p.grad is inside each param group ala
for group in self.param_groups: for p in group["params"]: if p.grad is None: <-- now you can access p.grad
Is there a way to adjust to handle parameter groups? I'm trying to integrate into FastAI which by default will create two param groups. I'll see if I can avoid that but I think param groups are quite common in most platforms so any tips here would be appreciated.