fastai / course-v3

The 3rd edition of course.fast.ai
https://course.fast.ai/
Apache License 2.0
4.91k stars 3.57k forks source link

model_split seems a little problem. Adam cannot get head_params, and cannot update head params! #598

Open HaronCHou opened 1 year ago

HaronCHou commented 1 year ago

https://github.com/jav0927/course-v3/blob/master/Fastai2_SSD_Model.ipynb def model_split(m): return L(m.body[0], m.body[1], m.body[2], m.body[3], m.body[4], m.body[5], m.body[6], m.body[7], m.drop).map(params)

epoch train_loss valid_loss time
0 48.916576 47.567001 00:55
1 48.897285 47.704815 00:54
2 48.736855 47.622379 00:54

if I change model_split as follows:

`def model_split(m): body_params, head_params = L(),L()

body_params = L(m.body[0], m.body[1], m.body[2], m.body[3], m.body[4], m.body[5], m.body[6], m.body[7], m.drop).map(params)

# head_params = L(m.sconv0, m.sconv1, m.sconv2, m.sconv3, m.out0, m.out1, m.out2, m.out3).map(params)
for p in m.body:
    body_params += params(p)
head_params += params(m.sconv0)
head_params += params(m.sconv1)
head_params += params(m.sconv2)
head_params += params(m.sconv3)
head_params += params(m.out0)
head_params += params(m.out1)
head_params += params(m.out2)
head_params += params(m.out3)
return L(body_params, head_params)`

Then, loss like: ​​​​ image

I think probably this is a problem

review-notebook-app[bot] commented 1 year ago

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB