'IdTok' object has no attribute 'as_sing'

Jyonn / Legommenders

A modular recommendation system that allows the selection of different components to be combined into a new recommender.

9 stars 1 forks source link

'IdTok' object has no attribute 'as_sing' #9

Closed Dinaelin-Yip closed 4 weeks ago

Dinaelin-Yip commented 1 month ago

Thank you for your work! In process/mind/processor.py, row 128-142,

        return UniTok().add_col(Column(
            name='nid',
            tokenizer=IdTok(name='nid', vocab=self.nid).as_sing(),
        )).add_col(Column(
            name='cat',
            tokenizer=EntTok(name='cat').as_sing()
        )).add_col(Column(
            name='subcat',
            tokenizer=EntTok(name='subcat').as_sing(),
        )).add_col(Column(
            name='title',
            tokenizer=txt_tok.as_list(max_length=max_title_len),
        )).add_col(Column(
            name='abs',
            tokenizer=txt_tok.as_list(max_length=max_abs_len),
        ))

The error is imported as follows,

AttributeError: 'IdTok' object has no attribute 'as_sing'

Could you tell me how to fix this?

Dinaelin-Yip commented 1 month ago

Meanwhile, it seems that there's no neg directory in the processed dataset. When I run the command

python worker.py \
    --embed config/exp/llama-split.yaml \
    --model config/model/llama-naml.yaml \
    --exp config/exp/llama-split.yaml \
    --data config/data/mind-llama.yaml \
    --version small \
    --llm_ver 7b \
    --hidden_size 64 \
    --layer 0 \
    --lora 0 \
    --fast_eval 0 \
    --embed_hidden_size 4096

The error is imported,

FileNotFoundError: [Errno 2] No such file or directory: 'data/MIND-small/neg/meta.data.json'