Low VRAM error

[RutraNickers]

I'm using the openCL version with a 1GB VRAM Radeon RX580, so definitely on the lowers of the low VRAM range. I was trying to use the Ensemble Mode, but it didn't run. I have 16gb of RAM though, so it would be lovely if it could use my RAM instead, but that's a PyTorch problem, isn't it? Could you guys create a low_vram mode akin to what ComfyUI did? Some nodes there also have this thing called tiling, where they process small chunks, one at a time, to limit how much VRAM is being used at each moment. While I can run in CPU Only mode, it is way too slow compared to what I can do in ComfyUI using DirectML and their -low_vram argument.

The error thrown is pasted bellow:

RuntimeError: "Could not allocate tensor with 402653184 bytes. There is not enough GPU video memory available!"
Traceback Error: "
  File "UVR.py", line 6638, in process_start
  File "separate.py", line 1054, in seperate
  File "separate.py", line 1182, in inference_vr
  File "separate.py", line 1149, in _execute
  File "lib_v5\vr_network\nets.py", line 161, in predict_mask
  File "lib_v5\vr_network\nets.py", line 137, in forward
  File "lib_v5\vr_network\nets.py", line 45, in __call__
  File "lib_v5\vr_network\layers.py", line 73, in __call__
  File "torch\nn\functional.py", line 3959, in interpolate
"

Error Time Stamp [2024-08-06 07:52:49]

Full Application Settings:

vr_model: Choose Model
aggression_setting: 5
window_size: 512
mdx_segment_size: 256
batch_size: Default
crop_size: 256
is_tta: False
is_output_image: False
is_post_process: False
is_high_end_process: False
post_process_threshold: 0.2
vr_voc_inst_secondary_model: No Model Selected
vr_other_secondary_model: No Model Selected
vr_bass_secondary_model: No Model Selected
vr_drums_secondary_model: No Model Selected
vr_is_secondary_model_activate: False
vr_voc_inst_secondary_model_scale: 0.9
vr_other_secondary_model_scale: 0.7
vr_bass_secondary_model_scale: 0.5
vr_drums_secondary_model_scale: 0.5
demucs_model: Choose Model
segment: Default
overlap: 0.25
overlap_mdx: Default
overlap_mdx23: 8
shifts: 2
chunks_demucs: Auto
margin_demucs: 44100
is_chunk_demucs: False
is_chunk_mdxnet: False
is_primary_stem_only_Demucs: False
is_secondary_stem_only_Demucs: False
is_split_mode: True
is_demucs_combine_stems: True
is_mdx23_combine_stems: True
demucs_voc_inst_secondary_model: No Model Selected
demucs_other_secondary_model: No Model Selected
demucs_bass_secondary_model: No Model Selected
demucs_drums_secondary_model: No Model Selected
demucs_is_secondary_model_activate: False
demucs_voc_inst_secondary_model_scale: 0.9
demucs_other_secondary_model_scale: 0.7
demucs_bass_secondary_model_scale: 0.5
demucs_drums_secondary_model_scale: 0.5
demucs_pre_proc_model: No Model Selected
is_demucs_pre_proc_model_activate: False
is_demucs_pre_proc_model_inst_mix: False
mdx_net_model: Choose Model
chunks: Auto
margin: 44100
compensate: Auto
denoise_option: None
is_match_frequency_pitch: True
phase_option: Automatic
phase_shifts: None
is_save_align: False
is_match_silence: True
is_spec_match: False
is_mdx_c_seg_def: False
is_invert_spec: False
is_deverb_vocals: False
deverb_vocal_opt: Main Vocals Only
voc_split_save_opt: Lead Only
is_mixer_mode: False
mdx_batch_size: Default
mdx_voc_inst_secondary_model: No Model Selected
mdx_other_secondary_model: No Model Selected
mdx_bass_secondary_model: No Model Selected
mdx_drums_secondary_model: No Model Selected
mdx_is_secondary_model_activate: False
mdx_voc_inst_secondary_model_scale: 0.9
mdx_other_secondary_model_scale: 0.7
mdx_bass_secondary_model_scale: 0.5
mdx_drums_secondary_model_scale: 0.5
is_save_all_outputs_ensemble: True
is_append_ensemble_name: False
chosen_audio_tool: Manual Ensemble
choose_algorithm: Min Spec
time_stretch_rate: 2.0
pitch_rate: 2.0
is_time_correction: True
is_gpu_conversion: True
is_primary_stem_only: False
is_secondary_stem_only: False
is_testing_audio: False
is_auto_update_model_params: True
is_add_model_name: False
is_accept_any_input: False
is_task_complete: False
is_normalization: False
is_use_opencl: True
is_wav_ensemble: False
is_create_model_folder: False
mp3_bit_set: 320k
semitone_shift: 0
save_format: WAV
wav_type_set: PCM_16
device_set: Radeon RX 580 Series:0
help_hints_var: True
set_vocal_splitter: No Model Selected
is_set_vocal_splitter: False
is_save_inst_set_vocal_splitter: False
model_sample_mode: False
model_sample_mode_duration: 10
demucs_stems: All Stems
mdx_stems: All Stems

[ChrisTG742]

This problem is a base problem how PyTorch works. When running in GPU-mode, the whole segments (windows) that have to be processed HAVE TO fit into the VRAM. You can try to set smaller segment-sizes or experiment with other settings the model offers to lower the maximum memory consumption. If that does not work, you need to run CPU mode or get a card with more RAM.