ValueError: X.shape[1] = 256 should be equal to 128, the number of features at training time

[tiz-lab]

Rasa version: 1.6.0

Python version: 3.7.4

Operating system: windows

Issue: error while running rasa test

Error (including full traceback):

2020-01-29 12:42:34 INFO     rasa.nlu.test  - Running model for predictions:
  0%|                                                                                                                                                                                                               | 0/504 [00:00<?, ?it/s]c:\users\tizianolabruna\anaconda3\lib\site-packages\rasa\nlu\extractors\crf_entity_extractor.py:590: UserWarning: Number of features (1) for attribute 'text_dense_features' does not match number of tokens (3). Set 'return_sequence' to true in the corresponding featurizer in order to make use of the features in 'CRFEntityExtractor'.
  f"Number of features ({len(features)}) for attribute "
  0%|                                                                                                                                                                                                               | 0/504 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "c:\users\tizianolabruna\anaconda3\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\tizianolabruna\anaconda3\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\TizianoLabruna\Anaconda3\Scripts\rasa.exe\__main__.py", line 9, in <module>
  File "c:\users\tizianolabruna\anaconda3\lib\site-packages\rasa\__main__.py", line 76, in main
    cmdline_arguments.func(cmdline_arguments)
  File "c:\users\tizianolabruna\anaconda3\lib\site-packages\rasa\cli\test.py", line 136, in test_nlu
    exclusion_percentages=args.percentages,
  File "c:\users\tizianolabruna\anaconda3\lib\site-packages\rasa\test.py", line 179, in compare_nlu_models
    runs,
  File "c:\users\tizianolabruna\anaconda3\lib\site-packages\rasa\nlu\test.py", line 1473, in compare_nlu
    test_path, model_path, output_directory=output_path, errors=True
  File "c:\users\tizianolabruna\anaconda3\lib\site-packages\rasa\nlu\test.py", line 1090, in run_evaluation
    interpreter, test_data
  File "c:\users\tizianolabruna\anaconda3\lib\site-packages\rasa\nlu\test.py", line 943, in get_eval_data
    result = interpreter.parse(example.text, only_output_properties=False)
  File "c:\users\tizianolabruna\anaconda3\lib\site-packages\rasa\nlu\model.py", line 376, in parse
    component.process(message, **self.context)
  File "c:\users\tizianolabruna\anaconda3\lib\site-packages\rasa\nlu\classifiers\sklearn_intent_classifier.py", line 155, in process
    intent_ids, probabilities = self.predict(X)
  File "c:\users\tizianolabruna\anaconda3\lib\site-packages\rasa\nlu\classifiers\sklearn_intent_classifier.py", line 198, in predict
    pred_result = self.predict_prob(X)
  File "c:\users\tizianolabruna\anaconda3\lib\site-packages\rasa\nlu\classifiers\sklearn_intent_classifier.py", line 187, in predict_prob
    return self.clf.predict_proba(X)
  File "c:\users\tizianolabruna\anaconda3\lib\site-packages\sklearn\svm\base.py", line 622, in _predict_proba
    X = self._validate_for_predict(X)
  File "c:\users\tizianolabruna\anaconda3\lib\site-packages\sklearn\svm\base.py", line 478, in _validate_for_predict
    (n_features, self.shape_fit_[1]))
ValueError: X.shape[1] = 256 should be equal to 128, the number of features at training time

Command or request that led to error:

rasa test nlu --config supervised.yml spacy.yml --nlu data/nlu.md --runs 3 --percentages 0 25 50 70 90

Content of configuration file (supervised.yml):

language: en
pipeline: supervised_embeddings
policies:
- name: AugmentedMemoizationPolicy
- name: KerasPolicy
- name: MappingPolicy
- name: TwoStageFallbackPolicy
  nlu_threshold: 0.15
  ambiguity_threshold: 0.1
  core_threshold: 0.15
  fallback_core_action_name: action_default_fallback
  fallback_nlu_action_name: action_default_fallback
  deny_suggestion_intent_name: out_of_scope
- name: FormPolicy

Content of configuration file (spacy.yml):

language: en
pipeline: pretrained_embeddings_spacy
policies:
- name: AugmentedMemoizationPolicy
- name: KerasPolicy
- name: MappingPolicy
- name: TwoStageFallbackPolicy
  nlu_threshold: 0.15
  ambiguity_threshold: 0.1
  core_threshold: 0.15
  fallback_core_action_name: action_default_fallback
  fallback_nlu_action_name: action_default_fallback
  deny_suggestion_intent_name: out_of_scope
- name: FormPolicy

Content of configuration file (convert.yml):

language: en
pipeline: pretrained_embeddings_convert
policies:
- name: AugmentedMemoizationPolicy
- name: KerasPolicy
- name: MappingPolicy
- name: TwoStageFallbackPolicy
  nlu_threshold: 0.15
  ambiguity_threshold: 0.1
  core_threshold: 0.15
  fallback_core_action_name: action_default_fallback
  fallback_nlu_action_name: action_default_fallback
  deny_suggestion_intent_name: out_of_scope
- name: FormPolicy

[sara-tagger]

Thanks for the issue, @dakshvar22 will get back to you about it soon!

You may find help in the docs and the forum, too 🤗

[tmbo]

@dakshvar22 do you mind taking a look at this?

[dakshvar22]

@tiz-lab I tested the set of commands and configuration files with sample data that rasa init command produces and everything worked as expected for rasa==1.6.0 and rasa==1.6.2. Can you try the same on your end and see if the problem persists? If it doesn't persist then we should look a bit closer into the dataset on which you are running this command.

[badri-thinker]

I am getting the same error

[badri-thinker]

X.shape[1] = 256 should be equal to 128, the number of features at training time

[badri-thinker]

i even downgraded to rasa 1.6 and still getting the same error

[badri-thinker]

the error occurs only with en_core_web_sm and not with en_core_web_md. I replaced spacy sm with md and it works

[no-response[bot]]

This issue has been automatically closed because there has been no response to our request for more information from the original author. Without this, we don't have enough information to help you. Please comment below with the requested information if you still need help.

[no-response[bot]]

This issue has been automatically closed because there has been no response to our request for more information from the original author. Without this, we don't have enough information to help you. Please comment below with the requested information if you still need help.

[ninjakx]

getting ValueError: X.shape[1] = 300 should be equal to 128, the number of features at training time

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Ahthe45th]

i too have the same error

Traceback (most recent call last):
  File "/root/miniconda3/envs/rasa/bin/rasa", line 8, in <module>
    sys.exit(main())
  File "/root/miniconda3/envs/rasa/lib/python3.6/site-packages/rasa/__main__.py", line 76, in main
    cmdline_arguments.func(cmdline_arguments)
  File "/root/miniconda3/envs/rasa/lib/python3.6/site-packages/rasa/cli/test.py", line 154, in test
    test_nlu(args)
  File "/root/miniconda3/envs/rasa/lib/python3.6/site-packages/rasa/cli/test.py", line 149, in test_nlu
    test_nlu(model_path, nlu_data, output, vars(args))
  File "/root/miniconda3/envs/rasa/lib/python3.6/site-packages/rasa/test.py", line 136, in test_nlu
    run_evaluation(nlu_data, nlu_model, output_directory=output_directory, **kwargs)
  File "/root/miniconda3/envs/rasa/lib/python3.6/site-packages/rasa/nlu/test.py", line 1090, in run_evaluation
    interpreter, test_data
  File "/root/miniconda3/envs/rasa/lib/python3.6/site-packages/rasa/nlu/test.py", line 943, in get_eval_data
    result = interpreter.parse(example.text, only_output_properties=False)
  File "/root/miniconda3/envs/rasa/lib/python3.6/site-packages/rasa/nlu/model.py", line 376, in parse
    component.process(message, **self.context)
  File "/root/miniconda3/envs/rasa/lib/python3.6/site-packages/rasa/nlu/classifiers/sklearn_intent_classifier.py", line 155, in process
    intent_ids, probabilities = self.predict(X)
  File "/root/miniconda3/envs/rasa/lib/python3.6/site-packages/rasa/nlu/classifiers/sklearn_intent_classifier.py", line 198, in predict
    pred_result = self.predict_prob(X)
  File "/root/miniconda3/envs/rasa/lib/python3.6/site-packages/rasa/nlu/classifiers/sklearn_intent_classifier.py", line 187, in predict_prob
    return self.clf.predict_proba(X)
  File "/root/miniconda3/envs/rasa/lib/python3.6/site-packages/sklearn/svm/base.py", line 622, in _predict_proba
    X = self._validate_for_predict(X)
  File "/root/miniconda3/envs/rasa/lib/python3.6/site-packages/sklearn/svm/base.py", line 478, in _validate_for_predict
    (n_features, self.shape_fit_[1]))
ValueError: X.shape[1] = 256 should be equal to 128, the number of features at training time

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed due to inactivity. Please create a new issue if you need more help.

[dakshvar22]

Should be checked if the issue still exists with the latest versions of Rasa.

[tabergma]

This issue does not seem to be valid anymore. Running rasa test nlu --config config1.yml config2.yml --nlu data/nlu.md --runs 3 --percentages 0 25 50 70 90 with different kind of configs that use the SklearnIntentClassifier runs fine without any issues.

[R0B077]

I am getting X.shape[1] = 96 should be equal to 256, the number of features at training time error. My code is:

from rasa_nlu.training_data import load_data
# from rasa_nlu.config import RasaNLUModelConfig
from rasa_nlu.model import Trainer
from rasa_nlu import config
from rasa_nlu.model import Metadata, Interpreter

def train_nlu():
    training_data = load_data('/content/data.json')
    trainer = Trainer(config.load('config_spacy.json'))
    trainer.train(training_data)
    model_directory = trainer.persist('/content/models/nlu' ,  fixed_model_name = 'myfirstbot')
``
def run_nlu():
    interpreter = Interpreter.load('/content/models/nlu/myfirstbot')
    print(interpreter.parse("What's the weather in Berlin at the moment?"))

if __name__ == '__main__':
    run_nlu()
    # run_nlu()

I am using rasa_nlu, version = 0.15.1, and because my program gave me an error, sklearn_crfsuite. Please help