search

CMUSTRUDEL / DIRTY

DIRTY: Augmenting Decompiler Output with Learned Variable Names and Types
MIT License
167 stars 24 forks source link

Hang when evaluating test set #6

Closed sei-eschwartz closed 2 years ago

sei-eschwartz commented 2 years ago

When I attempt to process the test set, the script just hangs. Here is the output:

GPU available: True, used: False
TPU available: False, using: 0 TPU cores
/home/ed/Documents/DIRTY/env/lib/python3.6/site-packages/pytorch_lightning/utilities/distributed.py:68: UserWarning: GPU available but not used. Set the --gpus flag when calling the script.
  warnings.warn(*args, **kwargs)
/home/ed/Documents/DIRTY/env/lib/python3.6/site-packages/pytorch_lightning/utilities/distributed.py:68: UserWarning: Your `IterableDataset` has `__len__` defined. In combination with multi-processing data loading (e.g. batch size > 1), this can lead to unintended side effects since the samples will be duplicated.
  warnings.warn(*args, **kwargs)
/home/ed/Documents/DIRTY/env/lib/python3.6/site-packages/webdataset/dataset.py:403: UserWarning: num_workers 8 > num_shards 1
  warnings.warn(f"num_workers {num_workers} > num_shards {len(urls)}")
Testing: 0it [00:00, ?it/s]
sei-eschwartz commented 2 years ago

Still a lot of CPU usage. I am not using --cuda because I didn't have enough memory. Maybe I am not being patient enough.

qibinc commented 2 years ago

Beam search is really slow especially on cpu. Try setting changing beam_size in multitask.xfmr.jsonnet to 0 to disable it.

sei-eschwartz commented 2 years ago

After disabling beam search, I received some output after ~10 minutes. It's taking 32 seconds per iteration though :( I guess DIRTY's model is much more complex than DIRE's.