bug in Aurora/CP/med.py

[FengWu-PKU]

line 47 "from .adapter import Adapter_Lora"
the point is redundant

[FengWu-PKU]

after I delete the dot, show 'no module named .adapter', then I use command 'pip install adapter',
but there is still an ImportError 'cannot import name 'Adapter_Lora' from 'adapter''

[xinlong-yang]

We include this for baseline comparison (Adapter), so you can directly delete this line, and if you want to do some baseline experiment, we here provide an implementation:

class Adapter_Lora(nn.Module):
    def __init__(self,
                 d_model=768,
                 bottleneck=64,
                 dropout=0.0,
                 init_option="lora",
                 adapter_scalar="learnable_scalar",):
        super().__init__()
        self.n_embd = d_model
        self.down_size = bottleneck

        if adapter_scalar == "learnable_scalar":
            self.scale = nn.Parameter(torch.ones(1))
        else:
            self.scale = float(adapter_scalar)

        self.down_proj = nn.Linear(self.n_embd, self.down_size)
        self.non_linear_func = nn.ReLU()
        self.up_proj = nn.Linear(self.down_size, self.n_embd)

        self.dropout = dropout

    def init_adapter_weights(self,):
        with torch.no_grad():
            nn.init.kaiming_uniform_(self.down_proj.weight, a=math.sqrt(5))
            nn.init.zeros_(self.up_proj.weight)
            nn.init.zeros_(self.down_proj.bias)
            nn.init.zeros_(self.up_proj.bias)

    def forward(self, x, add_residual=True):
        down = self.down_proj(x)
        # down = self.non_linear_func(down)
        up = self.up_proj(down)
        output = up * self.scale
        return output

[maoshanwen]

请问这个适配器类放在哪里呢，自己建立一个adapter名的文件吗

[xinlong-yang]

可以建一个adapter.py文件，然后导入上述代码，但是这不是必须的，你如果想跑baseline实验可以加上去，否则你可以直接删掉这一行

[FengWu-PKU]

Thank you for your prompt response. I have resolved the issue by removing that line of code.
I will appreciate that if you could provide a requirement.txt file so that I don't have to import Python packages one by one.