zixaphir / Stable-Diffusion-Webui-Civitai-Helper

Stable Diffusion Webui Extension for Civitai, to manage your model much more easily.
175 stars 24 forks source link

[Feature] Backup existing metadata before overwriting. #22

Open zixaphir opened 8 months ago

zixaphir commented 8 months ago

Hello, I am currently looking for feedback from users before I undertake developing this. Please reply with any suggestions, alterations, or preferences on the following. :)

Currently when metadata is overwritten, no care is taken to preserve existing data. While ideally an overwrite would only be performed with pristine, flawless data that never removes useful existing data, the fact of the matter is that on top of accidental mistakes on our end, there are also often cases we cannot control, like a model author replacing their model description with garbage data or an API serving an incorrect/mangled response. In these cases, having a backup or even the option to make a backup to restore the old data from can alleviate user anxiety and provide a nice safety net against accidental data loss.

Since our metadata is primarily tiny text files, these backups would also occupy minimal space.

Additionally, an option should exist to remove older backups.

Proposed implementation:

In addition to the current option to replace old metadata formats, an additional option should be added to create a backup of old metadata. This option should be enabled by default.

Backed-up files should be easily identifiable and easily removable. An option to remove backups older than a given time period should also exist, perhaps customizable by the user, but with a sane default, such as "older than 7 days." I propose a filename format of [old-filename].[timestamp].bak that contains the previous full filename and a timestamp, likely in Unix format. Another option is to use a compressed format, named [model-name]-backup-[timestamp].[format], with format being some compressed format, such as gz or zip, and all model-metadata (civitai.info and json) being packaged. However, a compressed implementation such as this would likely accrue additional development costs and require pulling in additional module imports.

aylz10 commented 8 months ago

在我的使用过程中发现,有些模型可能会遭到网站或者作者下架。如果直接覆盖的话,会造成使用空数据去覆盖原有数据的情况,针对这样的情况我建议是执行覆盖操作前,首先判断新数据是否为空数据,可以检查新数据的model id是否存在,存在的再覆盖原有数据。 如果需要备份的话。我的建议是把一个模型的所有更新覆盖节点的备份添加时间后全部放入该模型的同名压缩包里,让用户自己手动删除里面不需要的文件而不是设定一个日期自动删除,毕竟元数据文件特别的小,并且经过压缩后的文件大小对几百M或者几个G的模型来说不值一提,但是数据是无价的,一旦误操作删除又无法找回的话,对比文件占用来说损失太大。