TypeError: MoiraiMoEModule.__init__() missing 8 required positional arguments

[Liuwenjing985]

Describe the bug When I build a python demo name testmoe.py with the "get started codes example " in the src directory, the terminal gives the following error like this: "TypeError: MoiraiMoEModule.init() missing 8 required positional arguments: 'distr_output', 'd_model', 'd_ff', 'num_layers', 'patch_sizes', 'max_seq_len', 'attn_dropout_p', and 'dropout_p' ", and I didn't pip install the uni2ts in the virtual environment.

import torch
import matplotlib.pyplot as plt
import pandas as pd
from gluonts.dataset.pandas import PandasDataset
from gluonts.dataset.split import split
from huggingface_hub import hf_hub_download

from uni2ts.eval_util.plot import plot_single
from uni2ts.model.moirai import MoiraiForecast, MoiraiModule
from uni2ts.model.moirai_moe import MoiraiMoEForecast, MoiraiMoEModule

MODEL = "moirai-moe"  # model name: choose from {'moirai', 'moirai-moe'}
SIZE = "small"  # model size: choose from {'small', 'base', 'large'}
PDT = 20  # prediction length: any positive integer
CTX = 200  # context length: any positive integer
PSZ = "auto"  # patch size: choose from {"auto", 8, 16, 32, 64, 128}
BSZ = 32  # batch size: any positive integer
TEST = 100  # test set length: any positive integer

# Read data into pandas DataFrame
url = (
    "https://gist.githubusercontent.com/rsnirwan/c8c8654a98350fadd229b00167174ec4"
    "/raw/a42101c7786d4bc7695228a0f2c8cea41340e18f/ts_wide.csv"
)
df = pd.read_csv(url, index_col=0, parse_dates=True)

# Convert into GluonTS dataset
ds = PandasDataset(dict(df))

# Split into train/test set
train, test_template = split(
    ds, offset=-TEST
)  # assign last TEST time steps as test set

# Construct rolling window evaluation
test_data = test_template.generate_instances(
    prediction_length=PDT,  # number of time steps for each prediction
    windows=TEST // PDT,  # number of windows in rolling window evaluation
    distance=PDT,  # number of time steps between each window - distance=PDT for non-overlapping windows
)

# Prepare pre-trained model by downloading model weights from huggingface hub
if MODEL == "moirai":
    model = MoiraiForecast(
        module=MoiraiModule.from_pretrained(f"Salesforce/moirai-1.1-R-{SIZE}"),
        prediction_length=PDT,
        context_length=CTX,
        patch_size=PSZ,
        num_samples=100,
        target_dim=1,
        feat_dynamic_real_dim=ds.num_feat_dynamic_real,
        past_feat_dynamic_real_dim=ds.num_past_feat_dynamic_real,
    )
elif MODEL == "moirai-moe":
    model = MoiraiMoEForecast(
        module=MoiraiMoEModule.from_pretrained(f"Salesforce/moirai-moe-1.0-R-{SIZE}"),
        prediction_length=PDT,
        context_length=CTX,
        patch_size=16,
        num_samples=100,
        target_dim=1,
        feat_dynamic_real_dim=ds.num_feat_dynamic_real,
        past_feat_dynamic_real_dim=ds.num_past_feat_dynamic_real,
    )

predictor = model.create_predictor(batch_size=BSZ)
forecasts = predictor.predict(test_data.input)

input_it = iter(test_data.input)
label_it = iter(test_data.label)
forecast_it = iter(forecasts)

inp = next(input_it)
label = next(label_it)
forecast = next(forecast_it)

plot_single(
    inp, 
    label, 
    forecast, 
    context_length=200,
    name="pred",
    show_label=True,
)
plt.show()

Expected behavior pleease help me to anasys the problem

Environment

• Operating system: Linux
• Python version: 3.12.7
• PyTorch version: 2.2.0
• uni2ts version (have you pulled the latest version?): I uninstall the unists, and build the demo in the src path

[liuxu77]

Hi @Liuwenjing985, I think this is caused by using different versions of the code. I suggest you first get the latest code, and then reinstall the environment from source. Please do not use pip install uni2ts for now, we will update PyPI later.

[Liuwenjing985]

Hi,thank you for your answer,I have got the latest code today,and  reinstall the virtual environment from source and did't use the "pip install uni2ts"，but it still present the same error:TypeError: MoiraiMoEModule.init() missing 8 required positional arguments: 'distr_output', 'd_model', 'd_ff', 'num_layers', 'patch_sizes', 'max_seq_len', 'attn_dropout_p', and 'dropout_p', "It seems that when using module = MoiraiMoEModule.from_pretrained(f"Salesforce/moirai-moe-1.0-R-{SIZE}") to load the configuration file from Hugging Face, the parameters from the config.json file are not being correctly loaded. What should be done to resolve this?"

------------------ 原始邮件 ------------------ 发件人: "SalesforceAIResearch/uni2ts" @.>; 发送时间: 2024年11月14日(星期四) 晚上11:41 @.>; 抄送: "a @.**@.>; 主题: Re: [SalesforceAIResearch/uni2ts] TypeError: MoiraiMoEModule.init() missing 8 required positional arguments (Issue #144)

Hi @Liuwenjing985, I think this is caused by using different versions of the code. I suggest you first get the latest code, and then reinstall the environment from source. Please do not use pip install uni2ts for now, we will update PyPI later.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

[liuxu77]

@Liuwenjing985 Thank you. I cannot reproduce your problem. Could you check if you can use Moirai instead of Moirai-MoE without any issues?

[Liuwenjing985]

Thank you !,I downloaded the MoE model weights and the config.json file from the Hugging Face website and read the weights from an absolute path. ’‘ elif MODEL == "moirai-moe": model = MoiraiMoEForecast( module=MoiraiMoEModule.from_pretrained(f"/root/autodl-tmp/uni2ts/src/moirai-moe-1.0-R-small"), prediction_length=PDT, context_length=CTX, patch_size=16, num_samples=100, target_dim=1, feat_dynamic_real_dim=ds.num_feat_dynamic_real, past_feat_dynamic_real_dim=ds.num_past_feat_dynamic_real, ) ‘’‘

Now it no longer reports the error 'TypeError: MoiraiMoEModule.init() missing 8 required positional arguments (Issue #144)', but the model still fails to load and reports a new error:

" (moe) @.:~/autodl-tmp/uni2ts/src# python demo.py Loading weights from local directory Traceback (most recent call last): File "/root/autodl-tmp/uni2ts/src/demo.py", line 56, in <module> module=MoiraiMoEModule.from_pretrained(f"/root/autodl-tmp/uni2ts/src/moirai-moe-1.0-R-small"), ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/miniconda3/envs/moe/lib/python3.12/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn return fn(args, kwargs) ^^^^^^^^^^^^^^^^^^^ File "/root/miniconda3/envs/moe/lib/python3.12/site-packages/huggingface_hub/hub_mixin.py", line 569, in from_pretrained instance = cls._from_pretrained( ^^^^^^^^^^^^^^^^^^^^^ File "/root/miniconda3/envs/moe/lib/python3.12/site-packages/huggingface_hub/hub_mixin.py", line 793, in _from_pretrained return cls._load_as_safetensor(model, model_file, map_location, strict) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/miniconda3/envs/moe/lib/python3.12/site-packages/huggingface_hub/hub_mixin.py", line 831, in _load_as_safetensor load_model_as_safetensor(model, model_file, strict=strict) # type: ignore [arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^ NameError: name 'load_model_as_safetensor' is not defined

I suspect that I downloaded the wrong version of Hugging Face. Could you please provide me with the requirements.txt file for this project?

------------------ 原始邮件 ------------------ 发件人: "SalesforceAIResearch/uni2ts" @.>; 发送时间: 2024年11月15日(星期五) 下午2:32 @.>; 抄送: "a @.**@.>; 主题: Re: [SalesforceAIResearch/uni2ts] TypeError: MoiraiMoEModule.init() missing 8 required positional arguments (Issue #144)

@Liuwenjing985 Thank you. I cannot reproduce your problem. Could you check if you can use Moirai instead of Moirai-MoE without any issues?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

[liuxu77]

Hi @Liuwenjing985, the huggingface hub requirement is "huggingface_hub>=0.23.0".

Please carefully follow the installation instruction in the README file: https://github.com/SalesforceAIResearch/uni2ts/blob/main/README.md, it should install correctly. You can also take a look on pyproject.toml for package requirements.

[Suphx]

Maybe you didn't correctly download the pretrain weight files caused by your network environment. You can try to download the pretrain weight file manually and replace the model path with your downloaded weight file path.