Adding new model leads to AttributeError: module 'model_zoo' has no attribute 'DCNv2PositiveWeight'

[Sharan123]

I have downloaded the newest version of FuxiCTR library and decided to integrate the work from https://github.com/SkylerLinn/Understanding-the-Ranking-Loss Specifically their DCNv2PositiveWeight and DCNv2ListCE. I have created a folder under model_zoo like this for DCNv2PositiveWeight

model_zoo
    DCNv2PositiveWeight
         config
              dataset_config
              model_config
         src
              \_\_init\_\_.py
              DCNv2PositiveWeight.py
         fuxictr_version.py
         run_expid.py
         README.md 

In the __init__.py under model_zoo I have added: from .DCNv2PositiveWeight.src import DCNv2PositiveWeight

In the __init__.py under DCNv2PositiveWeight/src I have added: from .DCNv2PositiveWeight import DCNv2PositiveWeight

And finally the code for DCNv2PositiveWeight is the same as in the repo Understanding-the-Ranking loss:

```python import torch from torch import nn from fuxictr.pytorch.models import BaseModel from fuxictr.pytorch.layers import FeatureEmbedding, MLP_Block, CrossNetV2, CrossNetMix from fuxictr.metrics import evaluate_metrics from sklearn.metrics import roc_auc_score, log_loss import numpy as np class DCNv2PositiveWeight(BaseModel): def __init__(self, feature_map, model_id="DCNv2PositiveWeight", gpu=-1, model_structure="parallel", use_low_rank_mixture=False, low_rank=32, num_experts=4, learning_rate=1e-3, embedding_dim=10, stacked_dnn_hidden_units=[], parallel_dnn_hidden_units=[], dnn_activations="ReLU", num_cross_layers=3, net_dropout=0, batch_norm=False, embedding_regularizer=None, net_regularizer=None, pos_weight=1., **kwargs): super(DCNv2PositiveWeight, self).__init__(feature_map, model_id=model_id, gpu=gpu, embedding_regularizer=embedding_regularizer, net_regularizer=net_regularizer, **kwargs) self.embedding_layer = FeatureEmbedding(feature_map, embedding_dim) self.pos_weight = pos_weight input_dim = feature_map.sum_emb_out_dim() if use_low_rank_mixture: self.crossnet = CrossNetMix(input_dim, num_cross_layers, low_rank=low_rank, num_experts=num_experts) else: self.crossnet = CrossNetV2(input_dim, num_cross_layers) self.model_structure = model_structure assert self.model_structure in ["crossnet_only", "stacked", "parallel", "stacked_parallel"], \ "model_structure={} not supported!".format(self.model_structure) if self.model_structure in ["stacked", "stacked_parallel"]: self.stacked_dnn = MLP_Block(input_dim=input_dim, output_dim=None, # output hidden layer hidden_units=stacked_dnn_hidden_units, hidden_activations=dnn_activations, output_activation=None, dropout_rates=net_dropout, batch_norm=batch_norm) final_dim = stacked_dnn_hidden_units[-1] if self.model_structure in ["parallel", "stacked_parallel"]: self.parallel_dnn = MLP_Block(input_dim=input_dim, output_dim=None, # output hidden layer hidden_units=parallel_dnn_hidden_units, hidden_activations=dnn_activations, output_activation=None, dropout_rates=net_dropout, batch_norm=batch_norm) final_dim = input_dim + parallel_dnn_hidden_units[-1] if self.model_structure == "stacked_parallel": final_dim = stacked_dnn_hidden_units[-1] + parallel_dnn_hidden_units[-1] if self.model_structure == "crossnet_only": # only CrossNet final_dim = input_dim self.fc = nn.Linear(final_dim, 1) self.compile(kwargs["optimizer"], kwargs["loss"], learning_rate) self.reset_parameters() self.model_to_device() def forward(self, inputs): X = self.get_inputs(inputs) feature_emb = self.embedding_layer(X, flatten_emb=True) cross_out = self.crossnet(feature_emb) if self.model_structure == "crossnet_only": final_out = cross_out elif self.model_structure == "stacked": final_out = self.stacked_dnn(cross_out) elif self.model_structure == "parallel": dnn_out = self.parallel_dnn(feature_emb) final_out = torch.cat([cross_out, dnn_out], dim=-1) elif self.model_structure == "stacked_parallel": final_out = torch.cat([self.stacked_dnn(cross_out), self.parallel_dnn(feature_emb)], dim=-1) y_pred = self.fc(final_out) y_pred = self.output_activation(y_pred) return_dict = {"y_pred": y_pred} return return_dict def compute_loss(self, return_dict, y_true): weight = torch.where(y_true==1., self.pos_weight, 1) loss = self.loss_fn(return_dict["y_pred"], y_true, reduction='mean',weight=weight) loss += self.regularization_loss() return loss def evaluate_metrics(self, y_true, y_pred, metrics, group_id=None): print(metrics) ret_dict = dict() if 'wAUC' in metrics: sample_weight=np.where(y_true==1., self.pos_weight, 1.) ret_dict.update({'wAUC':roc_auc_score(y_true, y_pred, sample_weight=sample_weight, average='samples')}) if 'wlogloss' in metrics: sample_weight=np.where(y_true==1., self.pos_weight, 1.) ret_dict.update({'wlogloss':log_loss(y_true=y_true, y_pred=y_pred,sample_weight=sample_weight)}) tmp = [_ for _ in metrics if _ not in ['wAUC','wlogloss']] ret_dict.update(evaluate_metrics(y_true, y_pred, tmp, group_id)) return ret_dict ```

I call a script that calls autotuner.enumerate_params and autotuner.grid_search. The autotuner.grid_search calls the run_expid.py inside the experiment folder - which I modified to just print

print('model_zoo options: ', dir(model_zoo))

However this doesn't include the DCNv2PositiveWeight instead this is the output:

...
Traceback (most recent call last):
  File "/Users/dsaranovic/Code/FieldEndToEndLoss/experiment/run_expid.py", line 68, in <module>
    model_class = getattr(model_zoo, params['model'])
AttributeError: module 'model_zoo' has no attribute 'DCNv2PositiveWeight'
model_zoo options:  ['AFM', 'AFN', 'AOANet', 'AutoInt', 'BST', 'CCPM', 'DCN', 'DCNv2', 'DESTINE', 'DIEN', 'DIN', 'DLRM', 'DMIN', 'DMR', 'DNN', 'DSSM', 'DeepCrossing', 'DeepFM', 'DeepIM', 'EDCN', 'ETA', 'FFM', 'FFMv2', 'FGCNN', 'FLEN', 'FM', 'FiBiNET', 'FiGNN', 'FinalMLP', 'FinalNet', 'FmFM', 'FwFM', 'HFM', 'HOFM', 'InterHAt', 'LR', 'LorentzFM', 'MMoE', 'MaskNet', 'NFM', 'ONN', 'ONNv2', 'PEPNet', 'PNN', 'PPNet', 'SAM', 'SDIM', 'SharedBottom', 'WideDeep', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', 'multitask', 'xDeepFM']
....

How can we add additional models to the model_zoo such that the module sees those models and that we can run them on full data using autotuner (enumerate_params, grid_search) and run_expid.py ?

I feel like I am missing something obvious?

[zhujiem]

It looks correct. Make sure you run the code in "experiment/" directory. Or another quick solution: you could copy run_param_tuner.py to model_zoo/DCNv2PositiveWeight and run the script at this DCNv2PositiveWeight directory.

[zhujiem]

In the init.py under DCNv2PositiveWeight/src I have added: from .DCNv2PositiveWeight import DCNv2PositiveWeight

Try modify it to from .DCNv2PositiveWeight import *

[Sharan123]

Resolved: Just to note what the issue was in case someone else gets this: .bashrc (or equivalent) had the library path for model zoo appended for the base FuxiCTR library. Which means that it superseded the cloned version which was being modified. After updating the library path in python everything worked ok