"invalid command name" on running expertise.run

[neubig]

Hello,

I'm getting another error in the next step in the process, do you have an idea why this might be? I tried to debug a little bit myself, but I wasn't easily able to find where the error was being thrown...

$ python -m expertise.run ../config.json
Loading Archives dataset...
Loading Submissions dataset...
03/21/2022 16:37:44 - INFO - pytorch_transformers.modeling_bert -   Better speed can be achieved with apex installed from https://www.github.com/nvidia/apex .
03/21/2022 16:37:44 - INFO - pytorch_transformers.modeling_xlnet -   Better speed can be achieved with apex installed from https://www.github.com/nvidia/apex .
03/21/2022 16:37:45 - INFO - allennlp.common.registrable -   instantiating registered subclass relu of <class 'allennlp.nn.activations.Activation'>
03/21/2022 16:37:45 - INFO - allennlp.common.registrable -   instantiating registered subclass relu of <class 'allennlp.nn.activations.Activation'>
03/21/2022 16:37:45 - INFO - allennlp.common.registrable -   instantiating registered subclass relu of <class 'allennlp.nn.activations.Activation'>
03/21/2022 16:37:45 - INFO - allennlp.common.registrable -   instantiating registered subclass relu of <class 'allennlp.nn.activations.Activation'>
invalid command name '../config.json'

Here's my config for reference:

{
    "name": "acl_arr_acs_march_22_spectermfr",
    "match_group": "aclweb.org/ACL/ARR/2022/March/Area_Chairs",
    "csv_expertise": "ac_expertise.csv",
    "paper_invitation": "aclweb.org/ACL/ARR/2022/March/-/Blind_Submission",
    "exclusion_inv": "aclweb.org/ACL/ARR/2022/March/-/Expertise_Selection",
    "dataset": {
        "directory": "./"
    },
    "model": "specter+mfr",
    "model_params": {
        "specter_dir": "./specter/",
        "average_score": false,
        "max_score": true,
        "specter_batch_size": 16,
        "publications_path": "./",
        "submissions_path": "./",
        "mfr_feature_vocab_file": "./multifacet_recommender_data/feature_vocab_file",
        "mfr_checkpoint_dir": "./multifacet_recommender_data/mfr_model_checkpoint/",
        "mfr_epochs": 100,
        "mfr_batch_size": 50,
        "merge_alpha": 0.8,
        "work_dir": "./",
        "use_cuda": true,
        "scores_path": "./"
    },
    "experiment_dir": "./experiment/"
}

[haroldrubio]

Hi Graham,

I've narrowed down the function call that must be causing this error is when the model becomes initialized: https://github.com/openreview/openreview-expertise/blob/ff18198e0cb0b6930e51ea855549d852897db7c7/expertise/execute_expertise.py#L127-L140 as your output does not indicate that any of the model-specific archives/submissions are being loaded in memory since it is missing this output:

03/22/2022 09:47:47 - INFO - allennlp.common.registrable -   instantiating registered subclass relu of <class 'allennlp.nn.activations.Activation'>
Setting SPECTER archives
Setting MultiFacetRecommender archives
Setting SPECTER submissions
Setting MultiFacetRecommender submissions

I am still trying to find where the program attempts to execute a command on the provided config file.

[neubig]

Thank you! Here are the commands I ran, which are pretty simple... From the directory where config.json is included.

# Create dataset and username list for this venue
cd openreview-expertise/
touch ac_expertise.csv

python -m expertise.create_dataset ../config.json --baseurl https://api.openreview.net --username $ORUSER --password $ORPASS

python -m expertise.run ../config.json

This is using a fresh install of openreview-expertise from a few days ago.