TypeError: sum() got an unexpected keyword argument 'level'

[vpattanaik]

Hi @sebhaan, I am getting this weird error when trying to run DoEgen following Python Setup Installation. I am using the provided test file.

(DoE_workspace) E:\DevProjects\PyCharm_Projects\simulator_DoE>python -m doegen.doegen settings_design.yaml
using settings in: 'settings_design.yaml'
WARNING: Level names not given for factor Factor 5
Setting level names to L1 ... L3
WARNING: Level names not given for factor Factor 8
Setting level names to L1 ... L2
Number of total combinations (Full Factorial): 2916
Sit back and relax. This may take a while.
Total estimated runtime:  38.33minutes
--------------------------------
Optimizing array for 12 runs ...
Niteration: 53455
Traceback (most recent call last):
  File "D:\Workbench\Anaconda3\envs\DoE_workspace\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "D:\Workbench\Anaconda3\envs\DoE_workspace\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "D:\Workbench\Anaconda3\envs\DoE_workspace\lib\site-packages\doegen\doegen.py", line 1105, in <module>
    main_cli()
  File "D:\Workbench\Anaconda3\envs\DoE_workspace\lib\site-packages\doegen\doegen.py", line 1101, in main_cli
    main(**cfg)
  File "D:\Workbench\Anaconda3\envs\DoE_workspace\lib\site-packages\doegen\doegen.py", line 919, in main
    Aopt, effs = optimize_design(
  File "D:\Workbench\Anaconda3\envs\DoE_workspace\lib\site-packages\doegen\doegen.py", line 527, in optimize_design
    effs = evaluate_design2(setup, np.asarray(designs[i]), printopt=False)
  File "D:\Workbench\Anaconda3\envs\DoE_workspace\lib\site-packages\doegen\doegen.py", line 291, in evaluate_design2
    twoleveleff, twolevelmin, Alevel2bal = calc_twofactorbalance(setup, Array)
  File "D:\Workbench\Anaconda3\envs\DoE_workspace\lib\site-packages\doegen\doegen.py", line 206, in calc_twofactorbalance
    pd.get_dummies(dfcat.set_index(factor_name), prefix_sep=" _")
  File "D:\Workbench\Anaconda3\envs\DoE_workspace\lib\site-packages\pandas\core\generic.py", line 11512, in sum
    return NDFrame.sum(self, axis, skipna, numeric_only, min_count, **kwargs)
  File "D:\Workbench\Anaconda3\envs\DoE_workspace\lib\site-packages\pandas\core\generic.py", line 11280, in sum
    return self._min_count_stat_function(
  File "D:\Workbench\Anaconda3\envs\DoE_workspace\lib\site-packages\pandas\core\generic.py", line 11252, in _min_count_stat_function
    nv.validate_sum((), kwargs)
  File "D:\Workbench\Anaconda3\envs\DoE_workspace\lib\site-packages\pandas\compat\numpy\function.py", line 82, in __call__
    validate_args_and_kwargs(
  File "D:\Workbench\Anaconda3\envs\DoE_workspace\lib\site-packages\pandas\util\_validators.py", line 221, in validate_args_and_kwargs
    validate_kwargs(fname, kwargs, compat_args)
  File "D:\Workbench\Anaconda3\envs\DoE_workspace\lib\site-packages\pandas\util\_validators.py", line 162, in validate_kwargs
    _check_for_invalid_keys(fname, kwargs, compat_args)
  File "D:\Workbench\Anaconda3\envs\DoE_workspace\lib\site-packages\pandas\util\_validators.py", line 136, in _check_for_invalid_keys
    raise TypeError(f"{fname}() got an unexpected keyword argument '{bad_arg}'")
TypeError: sum() got an unexpected keyword argument 'level'

Have you encountered this issue before? Would appreciate your help.

[vpattanaik]

For reference, please follow these steps to reproduce the issue.

1. Create project folder.

2. Create Python environment.

3. Install required packages.

   pip install -r requirements.txt
   conda install swig
   pip install DoEgen

   Check pip list and ensure numpy is installed. If not, install it using, pip install numpy.

4. Create experiment folder (example, test_demo) inside project folder.

5. Do design generation.

   python -m doegen.init_config

6. Modify settings_design.yaml and settings_expresults.yaml file. Set path, outpath and inpath, outpath, resp. to the test_demo folder created in Step 4.

7. Fill experiment setup table Experiment_setup.xlsx. For testing use content of Experiment_setup_template.xlsx from repo.

8. Move Experiment_results.xlsx and Experiment_setup.xlsx into test_demo folder.

9. Now run the following command:

   python -m doegen.doegen settings_design.yaml

[vpattanaik]

Installed packages and corresponding versions:

> pip list
Package             Version
------------------- -------
contourpy           1.0.7
cycler              0.11.0
DoEgen              0.4.8
et-xmlfile          1.1.0
fonttools           4.39.3
importlib-resources 5.12.0
joblib              1.2.0
kiwisolver          1.4.4
matplotlib          3.7.1
numpy               1.24.3
OApackage           2.6.6
openpyxl            3.1.2
packaging           23.1
pandas              2.0.1
Pillow              9.5.0
pip                 23.0.1
pyparsing           3.0.9
python-dateutil     2.8.2
pytz                2023.3
PyYAML              6.0
scikit-learn        1.2.2
scipy               1.10.1
seaborn             0.12.2
setuptools          66.0.0
six                 1.16.0
tabulate            0.8.7
threadpoolctl       3.1.0
tzdata              2023.3
wheel               0.38.4
xlrd                1.2.0
XlsxWriter          3.1.0
zipp                3.15.0

[GeoMattB]

@vpattanaik this looks like a pandas version problem. Can you try pip install pandas==1.5.3 pip install tabulate==0.8.9 (I got an addtional error about the version of tabulate)

@sebhaan the deprecation warning is here: /root/.local/lib/python3.9/site-packages/DoEgen-0.4.8-py3.9.egg/doegen/doegen.py:208: FutureWarning: Using the level keyword in DataFrame and Series aggregations is deprecated and will be removed in a future version. Use groupby instead. df.sum(level=1) should use df.groupby(level=1).sum(). I've updated my fork for pandas 2.0 here, there were some other issues with df.plot deprecations as well: https://github.com/GeoMattB/DoEgen/commit/3678770ec30504c0ca2f5cb83afd02fca984d281

[vpattanaik]

Thank you @GeoMattB. Changing versions of pandas and tabulate resolved the issue.