自动缓存清理和文档更新

[HoshikawaYoru]

概述

通过自动缓存清理可以增加显存利用率

更改

• 我让preprocess_hubert_f0.py接受一个新的--automatic_cache_clearing参数，用于开启自动缓存清理
• 在日志添加了相关代码
• 在79行，我让cuda主动释放了缓存
• 文档更改

[ms903x1]

请问你的显卡型号是什么？显存有多大？

[HoshikawaYoru]

请问你的显卡型号是什么？显存有多大？

RTX3060Ti 12G显存

[magic-akari]

目前 PR 的更改处于无法接受的状态。

1. 描述中出现矛盾，无法确认问题是否真的存在。svc team 中也没有人复现问题。建议先开 issue 描述复现问题的条件和步骤，确认问题真实存在可复现。
2. 根据描述，这是一个 bugfix 而非 feature。那么它不应该是用户可选开启的。如果仅仅是因为“不确定会不会影响结果”所以引入一个配置项，这是不可接受的。

[HoshikawaYoru]

目前 PR 的更改处于无法接受的状态。

1. 描述中出现矛盾，无法确认问题是否真的存在。svc team 中也没有人复现问题。建议先开 issue 描述复现问题的条件和步骤，确认问题真实存在可复现。
2. 根据描述，这是一个 bugfix 而非 feature。那么它不应该是用户可选开启的。如果仅仅是因为“不确定会不会影响结果”所以引入一个配置项，这是不可接受的。

@magic-akari 我的描述可能有问题，这是一个feature，开启可以让显存占用降低（大部分情况下是这样的），根据我的测试，它可以正常工作，并不会影响结果，并且有着实质性的效果。 而对于我之前提出的问题，我提了一个Issue

[magic-akari]

根据文档所述，此处 empty_cache() 属于 API 误用。

Releases all unoccupied cached memory currently held by the caching allocator so that those can be used in other GPU application and visible in nvidia-smi.

NOTE

empty_cache() doesn’t increase the amount of GPU memory available for PyTorch. However, it may help reduce fragmentation of GPU memory in certain cases. See Memory management for more details about GPU memory management.

https://pytorch.org/docs/stable/notes/cuda.html#cuda-memory-management

PyTorch uses a caching memory allocator to speed up memory allocations. This allows fast memory deallocation without device synchronizations. However, the unused memory managed by the allocator will still show as if used in nvidia-smi. You can use memory_allocated() and max_memory_allocated() to monitor memory occupied by tensors, and use memory_reserved() and max_memory_reserved() to monitor the total amount of memory managed by the caching allocator. Calling empty_cache() releases all unused cached memory from PyTorch so that those can be used by other GPU applications. However, the occupied GPU memory by tensors will not be freed so it can not increase the amount of GPU memory available for PyTorch.

For more advanced users, we offer more comprehensive memory benchmarking via memory_stats(). We also offer the capability to capture a complete snapshot of the memory allocator state via memory_snapshot(), which can help you understand the underlying allocation patterns produced by your code.

https://pytorch.org/docs/stable/notes/cuda.html#memory-management

注意：Calling empty_cache() releases all unused cached memory from PyTorch so that those can be used by other GPU applications. 调用 empty_cache 会将 memory 占用还给系统，供其他 GPU 应用使用。你所描述的情况“memory不足”和此情况相反。

empty_cache 的合理使用情况是：

1. 程序长期运行，需要提前把 memory 还给系统，而不是等到程序执行结束后被系统回收。
2. 程序在后续较长的时间段内不再需要 memory。注意 PyTorch uses a caching memory allocator to speed up memory allocations. 所以对于程序内的 pytorch 占用和分配，pytorch 会自己管理而无需人工干预。

[HoshikawaYoru]

我是想同时训练模型的同时处理下一个数据集的，然后我发现这样做比较有效果，所以我让新增了一个参数来用于这个情况

---

From: magic-akari @.> Sent: Monday, July 31, 2023 1:31 PM To: svc-develop-team/so-vits-svc @.> Cc: 星川よる @.>; Author @.> Subject: Re: [svc-develop-team/so-vits-svc] 自动显存清理和文档更新 (PR #351)

根据文档所述，此处 empty_cache() 属于 API 误用。

Releases all unoccupied cached memory currently held by the caching allocator so that those can be used in other GPU application and visible in nvidia-smi.

NOTE

empty_cache() doesn’t increase the amount of GPU memory available for PyTorch. However, it may help reduce fragmentation of GPU memory in certain cases. See Memory management for more details about GPU memory management.

https://pytorch.org/docs/stable/notes/cuda.html#cuda-memory-management

PyTorch uses a caching memory allocator to speed up memory allocations. This allows fast memory deallocation without device synchronizations. However, the unused memory managed by the allocator will still show as if used in nvidia-smi. You can use memory_allocated() and max_memory_allocated() to monitor memory occupied by tensors, and use memory_reserved() and max_memory_reserved() to monitor the total amount of memory managed by the caching allocator. Calling empty_cache() releases all unused cached memory from PyTorch so that those can be used by other GPU applications. However, the occupied GPU memory by tensors will not be freed so it can not increase the amount of GPU memory available for PyTorch.

For more advanced users, we offer more comprehensive memory benchmarking via memory_stats(). We also offer the capability to capture a complete snapshot of the memory allocator state via memory_snapshot(), which can help you understand the underlying allocation patterns produced by your code.

https://pytorch.org/docs/stable/notes/cuda.html#memory-management

注意：Calling empty_cache() releases all unused cached memory from PyTorch so that those can be used by other GPU applications. 调用 empty_cache 会讲 memory 占用还给系统，供其他 GPU 应用使用。你所描述的情况“memory不足”和此情况相反。

empty_cache 的合理使用情况是：

1. 程序长期运行，需要提前把 memory 还给系统，而不是等到程序执行结束后被系统回收。
2. 程序在后续较长的时间段内不再需要 memory。注意 PyTorch uses a caching memory allocator to speed up memory allocations. 所以对于程序内的 pytorch 占用和分配，pytorch 会自己管理而无需人工干预。

― Reply to this email directly, view it on GitHubhttps://github.com/svc-develop-team/so-vits-svc/pull/351#issuecomment-1657647736, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A7ZRWVFTZLVLHXSOU4LTSQDXS47KLANCNFSM6AAAAAA24LTMCU. You are receiving this because you authored the thread.Message ID: @.***>

[HoshikawaYoru]

这样做应该会提高内存的利用率

[ms903x1]

不推荐这样的工作流，稳定性太差，而且考虑到大多数使用者的显卡显存都在6g-8g的水平，也不允许在训练的同时预处理。

[HoshikawaYoru]

明白了，但我认为这样做可以满足一部分人的特殊需求，我再更改以下代码和文档，你看这样行吗？

---

From: ms903x1 @.> Sent: Monday, July 31, 2023 1:45 PM To: svc-develop-team/so-vits-svc @.> Cc: 星川よる @.>; Author @.> Subject: Re: [svc-develop-team/so-vits-svc] 自动显存清理和文档更新 (PR #351)

不推荐这样的工作流，稳定性太差，而且考虑到大多数使用者的显卡显存都在6g-8g的水平，也不允许在训练的同时预处理。

― Reply to this email directly, view it on GitHubhttps://github.com/svc-develop-team/so-vits-svc/pull/351#issuecomment-1657671892, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A7ZRWVELDKKFZKCEPUBYW23XS5A7BANCNFSM6AAAAAA24LTMCU. You are receiving this because you authored the thread.Message ID: @.***>

[ms903x1]

我投弃权票，还是要看另外三位的意见.