osuossu8
commented
3 years ago How to use SED
メモ
How to train
- Training with framewise_output need time and classes data
- loss function is usually binary cross entropy
- Then this label data(contains time and class) is called "strong label".
- clipwise_output is the prediction result for classes of the whole clip(for example 60sec)
- It contain no time information(only class information)
- Then this label(contains only class) is called "weak label".
- In general, strong label training have higher accuracy than weak label training. Because strong labels contain time information.
Feature extractor
- It is CNN14 (1CH) in default PANNs architecture.
- CNN14 is strong model and pretrained model(by audio clip).
- But in this competition(and last competition), CNN14 is not strongest model.
- shinmura0 さんのモデルだと EfficientNet(3CH) が良かった
How to predict
- Prediction with clipwise_output is simple to use.
-
But I think that it is weak for short event.
-
Comparatively, framewise_output prediction is good at short sound event.
-
このコンペでは short event が多いので shinmura0 さんは framewise_output を使用
- framewise_output --> max(axis=time) --> final score
pip install efficientnet_pytorch
from efficientnet_pytorch import EfficientNet
class feature_extractor(nn.Module):
def __init__(self, original):
super().__init__()
self.model = original
def forward(self, x):
x= self.model.extract_features(x)
return x
class PANNsCNN14Att(nn.Module):
def __init__():
self.Enet = feature_extractor(EfficientNet.from_pretrained('efficientnet-b0'))
...
def cnn_feature_extractor(self, x):
return self.Enet(x)
def forward(self, input, mixup_lambda=None):
x, frames_num = self.preprocess(input, mixup_lambda=mixup_lambda)
x = torch.cat((x,x,x),1)
x = self.cnn_feature_extractor(x)
x = torch.mean(x, dim=3)
...ぶらさがってるコメントなど
-
Q : have you tried any audio augmentations (like Guassian noise etc.)
-
A : I tried to add white noise and pink noise. But they didn't work.
-
shinmura0 さんが 0.9 届いた時の学習セット
- epoch:30
- time period:10sec
- 60sec, 10sec, 5sec 試したが 10sec が一番よかった
- training 時は、I get random 10 sec clip from 60sec training data
- test 時は、
- I split 60sec test clip per 10 sec.
- And I get mini prediction(max(framewise_output, axis=time)) per 10 sec.
- Final prediction is max of mini prediction(=np.max((6,24), axis=0)).
アイデア、コメントなど
- 60sec, 10sec, 5sec などをアンサンブルすれば良いのでは?
- Feature Extractor も EfficientNet のほか ResNeXt, ResNest もアンサンブルすれば良いのでは?
- audio augmentations は効かなかったそうだけど、画像 (メルスペクトログラム) への augmentation は効くのではないか
スレッドタイトル
メモ
ぶらさがってるコメントなど
それらを受けてのアイデア、コメントなど