Kaszanas / SC2_Datasets

https://sc2-datasets.readthedocs.io/
GNU General Public License v3.0
8 stars 3 forks source link

OSError: WinError 1455 The paging file is too small #15

Closed Kaszanas closed 1 year ago

Kaszanas commented 2 years ago

While attempting training with these parameters:

datamodule = SC2ReplaypackDataModule(
        transform=economy_average_vs_outcome,
        replaypack_name="2020_IEM_Katowice",
        replaypack_unpack_dir="D:/Projects/SC2EGSet_Experiments/test/test_files/unpack",
        download=False,
        batch_size=256,
        num_workers=12,
)
logistic_regression = LogisticRegression(input_dim=2 * 39, num_classes=2)
trainer = pl.Trainer(
        logger=True,
        accelerator="gpu",
        devices=1,
        auto_select_gpus=True,
        max_epochs=10,
        log_every_n_steps=2,
)

The following error was raised:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\kasza\.pyenv\pyenv-win\versions\3.10.2\lib\multiprocessing\spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "C:\Users\kasza\.pyenv\pyenv-win\versions\3.10.2\lib\multiprocessing\spawn.py", line 125, in _main
    prepare(preparation_data)
  File "C:\Users\kasza\.pyenv\pyenv-win\versions\3.10.2\lib\multiprocessing\spawn.py", line 236, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "C:\Users\kasza\.pyenv\pyenv-win\versions\3.10.2\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path   
    main_content = runpy.run_path(main_path,
  File "C:\Users\kasza\.pyenv\pyenv-win\versions\3.10.2\lib\runpy.py", line 269, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "C:\Users\kasza\.pyenv\pyenv-win\versions\3.10.2\lib\runpy.py", line 96, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "C:\Users\kasza\.pyenv\pyenv-win\versions\3.10.2\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "D:\Projects\SC2EGSet_Experiments\src\experiments\logistic_regression.py", line 3, in <module>
    from pl_bolts.models.regression import LogisticRegression
  File "D:\Projects\SC2EGSet_Experiments\venv_3_10\lib\site-packages\pl_bolts\__init__.py", line 11, in <module>
    from pl_bolts import (  # noqa: E402
  File "D:\Projects\SC2EGSet_Experiments\venv_3_10\lib\site-packages\pl_bolts\callbacks\__init__.py", line 2, in <module>   
    from pl_bolts.callbacks.byol_updates import BYOLMAWeightUpdate
  File "D:\Projects\SC2EGSet_Experiments\venv_3_10\lib\site-packages\pl_bolts\callbacks\byol_updates.py", line 4, in <module>
    from pytorch_lightning import Callback, LightningModule, Trainer
  File "D:\Projects\SC2EGSet_Experiments\venv_3_10\lib\site-packages\pytorch_lightning\__init__.py", line 20, in <module>   
    from pytorch_lightning.callbacks import Callback  # noqa: E402
  File "D:\Projects\SC2EGSet_Experiments\venv_3_10\lib\site-packages\pytorch_lightning\callbacks\__init__.py", line 14, in <module>
    from pytorch_lightning.callbacks.base import Callback
  File "D:\Projects\SC2EGSet_Experiments\venv_3_10\lib\site-packages\pytorch_lightning\callbacks\base.py", line 22, in <module>
    import torch
  File "D:\Projects\SC2EGSet_Experiments\venv_3_10\lib\site-packages\torch\__init__.py", line 126, in <module>
    raise err
OSError: [WinError 1455] The paging file is too small for this operation to complete. Error loading "D:\Projects\SC2EGSet_Experiments\venv_3_10\lib\site-packages\torch\lib\cudnn_cnn_infer64_8.dll" or one of its dependencies.
Kaszanas commented 1 year ago

This was most likely due to Torch attempting to load the library for each of the workers.