deepfm的使用integer_value_sequence报错

yiyibupt commented 6 years ago

报错内容：

I0903 18:05:49.396421 18756 GradientMachine.cpp:101] Init parameters done.
F0903 18:05:53.062909 18756 BaseMatrix.cu:107] Check failed: dimM + offset.bRow_ <= b.height_ (2000 vs. 1000) 
*** Check failure stack trace: ***
    @     0x7fab977d9d2d  google::LogMessage::Fail()
    @     0x7fab977dd7dc  google::LogMessage::SendToLog()
    @     0x7fab977d9853  google::LogMessage::Flush()
    @     0x7fab977decee  google::LogMessageFatal::~LogMessageFatal()
    @     0x7fab97660959  paddle::BaseMatrixT<>::applyBinary<>()
    @     0x7fab97660eca  paddle::BaseMatrixT<>::addAtOffset()
    @     0x7fab975676f7  paddle::ConcatenateLayer::backward()
    @     0x7fab975a02f6  paddle::NeuralNetwork::backward()
    @     0x7fab977b5ce4  GradientMachine::forwardBackward()
    @     0x7fab973fd359  _wrap_GradientMachine_forwardBackward
    @     0x7fac801e845b  PyEval_EvalFrameEx
    @     0x7fac801ea624  PyEval_EvalCodeEx
    @     0x7fac801e8eee  PyEval_EvalFrameEx
    @     0x7fac801ea624  PyEval_EvalCodeEx
    @     0x7fac801e8eee  PyEval_EvalFrameEx
    @     0x7fac801ea624  PyEval_EvalCodeEx
    @     0x7fac801e8eee  PyEval_EvalFrameEx
    @     0x7fac801ea624  PyEval_EvalCodeEx
    @     0x7fac801e8eee  PyEval_EvalFrameEx
    @     0x7fac801ea624  PyEval_EvalCodeEx
    @     0x7fac801ea822  PyEval_EvalCode
    @     0x7fac8020349c  run_mod
    @     0x7fac80203afc  PyRun_FileExFlags
    @     0x7fac80204a11  PyRun_SimpleFileExFlags
    @     0x7fac80214d26   #Py_Main
    @     0x7fac7f44bb45  __libc_start_main
    @           0x4006ca  (unknown)

reader.py脚本：

class Dataset:
    def _reader_creator(self, path, is_infer):
        def reader():
            with open(path, 'r') as f:
                for line in f:
                    features = line.rstrip('\n').split('\t')
                    dense_feature = map(float, features[0].split(','))
                    sparse_feature = map(int, features[1].split(','))
                    test_sparse = [[1], [1,2]]
                    if not is_infer:
                        label = [float(features[2])]
                        yield [dense_feature, sparse_feature] + test_sparse + [label]
                    else:
                        yield [dense_feature, sparse_feature] + sparse_feature

        return reader

    def train(self, path):
        return self._reader_creator(path, False)

    def test(self, path):
        return self._reader_creator(path, False)

    def infer(self, path):
        return self._reader_creator(path, True)

feeding = {
    'dense_input': 0,
    'sparse_input': 1,
    'C1': 2,
    'C2': 3,
    'label': 4
}

network_conf.py如下：

def DeepFM(factor_size, infer=False):
    dense_input = paddle.layer.data(
        name="dense_input",
        type=paddle.data_type.dense_vector(dense_feature_dim))
    sparse_input = paddle.layer.data(
        name="sparse_input",
        type=paddle.data_type.sparse_binary_vector(sparse_feature_dim))
    sparse_input_ids = [
        paddle.layer.data(
            name="C" + str(i),
            type=paddle.data_type.integer_value_sequence(sparse_feature_dim))
        for i in range(1, 3)
    ]

    dense_fm = fm_layer(
        dense_input,
        factor_size,
        fm_param_attr=paddle.attr.Param(name="DenseFeatFactors"))
    sparse_fm = fm_layer(
        sparse_input,
        factor_size,
        fm_param_attr=paddle.attr.Param(name="SparseFeatFactors"))

    def embedding_layer(input):
        return paddle.layer.embedding(
            input=input,
            size=factor_size,
            param_attr=paddle.attr.Param(name="SparseFeatFactors"))

    sparse_embed_seq = map(embedding_layer, sparse_input_ids)
    sparse_embed = paddle.layer.concat(sparse_embed_seq)

    fc1 = paddle.layer.fc(input=[sparse_embed, dense_input],
                          size=400,
                          act=paddle.activation.Relu())
    fc2 = paddle.layer.fc(input=fc1, size=400, act=paddle.activation.Relu())
    fc3 = paddle.layer.fc(input=fc2, size=400, act=paddle.activation.Relu())

    predict = paddle.layer.fc(input=[dense_fm, sparse_fm, fc3],
                              size=1,
                              act=paddle.activation.Sigmoid())

    if not infer:
        label = paddle.layer.data(
            name="label", type=paddle.data_type.dense_vector(1))
        cost = paddle.layer.multi_binary_label_cross_entropy_cost(
            input=predict, label=label)
        paddle.evaluator.classification_error(
            name="classification_error", input=predict, label=label)
        paddle.evaluator.auc(name="auc", input=predict, label=label)
        return cost
    else:
        return predict

Yancey1989 commented 6 years ago

yield [dense_feature, sparse_feature] + test_sparse + [label]

改成下面的试试？

yield [dense_feature, test_sparse] + test_sparse + [label]

yiyibupt commented 6 years ago

尝试了，是不可以的。我的想法就是给deepfm的integer_value更换为integer_value_sequence， sparse_input和 sparse_input_ids可以是不一样的吧 sparse_input为稀疏向量， sparse_input_ids为稀疏整数序列模式报错信息如下： File "train.py", line 108, in train() File "train.py", line 104, in train num_passes=args.num_passes) File "/home/litao/.jumbo/lib/python2.7/site-packages/paddle/v2/trainer.py", line 182, in train in_args = feeder(data_batch) File "/home/litao/.jumbo/lib/python2.7/site-packages/py_paddle/dataprovider_converter.py", line 282, in call return self.convert(dat, argument) File "/home/litao/.jumbo/lib/python2.7/site-packages/paddle/v2/data_feeder.py", line 133, in convert return DataProviderConverter.convert(self, reorder_data(dat), argument) File "/home/litao/.jumbo/lib/python2.7/site-packages/py_paddle/dataprovider_converter.py", line 277, in convert scanner.finish_scan(argument) File "/home/litao/.jumbo/lib/python2.7/site-packages/py_paddle/dataprovider_converter.py", line 238, in finish_scan self.inner_scanner__.finish_scan(argument) File "/home/litao/.jumbo/lib/python2.7/site-packages/py_paddle/dataprovider_converter.py", line 211, in finish_scan ids = swig_paddle.IVector.create(self.ids__, self.data_in_gpu) File "/home/litao/.jumbo/lib/python2.7/site-packages/py_paddle/swig_paddle.py", line 881, in create return _swig_paddle.IVector_create(*args) NotImplementedError: Wrong number or type of arguments for overloaded function 'IVector_create'. Possible C/C++ prototypes are: IVector::create(std::vector< int,std::allocator< int > > const &,bool) IVector::create(std::vector< int,std::allocator< int > > const &)

network_conf.py

import paddle.v2 as paddle

#dense_feature_dim = 2
#sparse_feature_dim = 26

dense_feature_dim = 387
sparse_feature_dim = 30

def fm_layer(input, factor_size, fm_param_attr):
    first_order = paddle.layer.fc(input=input,
                                  size=1,
                                  act=paddle.activation.Linear())
    second_order = paddle.layer.factorization_machine(
        input=input,
        factor_size=factor_size,
        act=paddle.activation.Linear(),
        param_attr=fm_param_attr)
    out = paddle.layer.addto(
        input=[first_order, second_order],
        act=paddle.activation.Linear(),
        bias_attr=False)
    return out

def DeepFM(factor_size, infer=False):
    dense_input = paddle.layer.data(
        name="dense_input",
        type=paddle.data_type.dense_vector(dense_feature_dim))
    sparse_input = paddle.layer.data(
        name="sparse_input",
        type=paddle.data_type.integer_value_sequence(sparse_feature_dim))
#        type=paddle.data_type.sparse_binary_vector(sparse_feature_dim))
    sparse_input_ids = [
        paddle.layer.data(
            name="C" + str(i),
            type=paddle.data_type.integer_value_sequence(sparse_feature_dim))
        for i in range(1, 3)
    ]

    dense_fm = fm_layer(
        dense_input,
        factor_size,
        fm_param_attr=paddle.attr.Param(name="DenseFeatFactors"))
    sparse_fm = fm_layer(
        sparse_input,
        factor_size,
        fm_param_attr=paddle.attr.Param(name="SparseFeatFactors"))

    def embedding_layer(input):
        return paddle.layer.embedding(
            input=input,
            size=factor_size,
            param_attr=paddle.attr.Param(name="SparseFeatFactors"))

    sparse_embed_seq = map(embedding_layer, sparse_input_ids)
    sparse_embed = paddle.layer.concat(sparse_embed_seq)

    fc1 = paddle.layer.fc(input=[sparse_embed, dense_input],
                          size=400,
                          act=paddle.activation.Relu())
    fc2 = paddle.layer.fc(input=fc1, size=400, act=paddle.activation.Relu())
    fc3 = paddle.layer.fc(input=fc2, size=400, act=paddle.activation.Relu())

    predict = paddle.layer.fc(input=[dense_fm, sparse_fm, fc3],
                              size=1,
                              act=paddle.activation.Sigmoid())

    if not infer:
        label = paddle.layer.data(
            name="label", type=paddle.data_type.dense_vector(1))
        cost = paddle.layer.multi_binary_label_cross_entropy_cost(
            input=predict, label=label)
        paddle.evaluator.classification_error(
            name="classification_error", input=predict, label=label)
        paddle.evaluator.auc(name="auc", input=predict, label=label)
        return cost
    else:
        return predict

luotao1 commented 6 years ago

我的想法就是给deepfm的integer_value更换为integer_value_sequence

请问为什么要进行替换呢？

sparse_input和 sparse_input_ids可以是不一样的吧 sparse_input为稀疏向量， sparse_input_ids为稀疏整数序列模式

从原来模型的代码来看，sparse_input_ids是integer_value类型，是整数类型，不是整数序列类型。

yiyibupt commented 6 years ago

模型的demo的整数类型，但是我的项目背景类别特征是序列特征，那么不可以使用fm么？deep+fm是不支持整数序列特征么？

Superjomn commented 6 years ago

原理上，如果是这里的 int 就是想当成数值做，那把数据当成 float就可以直接跑了。

如果id是作为 embedding id，模型里面转成 embedding。看了下这个网络不是太复杂，建议用fluid对应写下，里面的op会更直观一些，能更主流框架对应上。

luotao1 commented 6 years ago

可以参考 https://github.com/PaddlePaddle/models/issues/946 ，models里面加一个pooling来处理不定长特征。

yiyibupt commented 6 years ago

@Superjomn 我这里的序列特征是类别特征，不是数值，后半句话，我不太理解是什么意思？我的输入是整数序列特征了，但是程序报错；我不知道是哪里有问题了？

luotao1 commented 6 years ago

F0903 18:05:53.062909 18756 BaseMatrix.cu:107] Check failed: dimM + offset.bRow_ <= b.height_ (2000 vs. 1000) 
*** Check failure stack trace: ***
    @     0x7fab977d9d2d  google::LogMessage::Fail()
    @     0x7fab977dd7dc  google::LogMessage::SendToLog()
    @     0x7fab977d9853  google::LogMessage::Flush()
    @     0x7fab977decee  google::LogMessageFatal::~LogMessageFatal()
    @     0x7fab97660959  paddle::BaseMatrixT<>::applyBinary<>()
    @     0x7fab97660eca  paddle::BaseMatrixT<>::addAtOffset()
    @     0x7fab975676f7  paddle::ConcatenateLayer::backward()

从报错来看，是concat反向连接时，维度没有匹配上。

yiyibupt commented 6 years ago

加了pooling，然后解决了特征不定长问题

PaddlePaddle / Paddle

deepfm的使用integer_value_sequence报错 #13180