[BUG] Loss of information due to update

[JingJang]

Thank you for the update. Just two issues, one major and the other is minor.

Major bug: If you already have an existing info file that contains information, and it tries to update but cannot as the model have been removed from Civitai or the model never existed as you manually created the JSON. Instead of merging the data, it create a new JSON skeleton data and old data is lost. This is a major bug as models get removed all the time and one creates their one their own data. The data should imported from the current info file if it does not exist on Civitai. A merge would be nice even if the model exists on Civitai, but might not be worth the effort for 99% of the time. Apart from merging trigger words. But if, it does not exist just update the JSON structure.

I lost a lot of data with this. I should've backed up. A mistake on my behalf.

Minor Issue:: For a model that does not exist on Civitai, if the model.name property is empty it should contain the filename of the model.

{
    "id": "",
    "modelId": "",
    "name": "",
    "trainedWords": [],
    "baseModel": "Unknown",
    "description": "",
    "model": {
        "name": "7th_Anime_v3_A",
        "type": "ckp",
        "nsfw": "",
        "poi": "",
        "description": "",
        "tags": []
    },
    "files": [
        {
            "name": "7th_Anime_v3_A.safetensors", <-------- FILENAME
            "sizeKB": 4165133,
            "type": "Model",
            "hashes": {
                "AutoV2": "c330a60ddc",
                "SHA256": "c330a60ddcb49c61109dbd7c28cf326733dadb77984c6e8d162bdc200a05df9e"
            }
        }
    ],
    "downloadUrl": "",
    "extensions": {
        "sd_civitai_helper": {
            "version": "1.7.2"
        }
    }
}

Finally not an issue. I thought you might end up merging the data into the json file and thus not needing the info file. So it would be something like:

{
    "description": "",
    "notes": "",
    "sd version": "Unknown",
    "extensions": {
        "sd_civitai_helper": {
            "version": "1.7.2"
            "data": {
                  ... contains the actual JSON data of the model
             }
        }
    }
}

[JingJang]

Another bug found. It uses the version id I think or maybe modelid to get the JSON data. Not good as sometimes models are removed and then re-uploaded. So the model id changes but the SHA256 hasn't. So maybe check using first the version id and then the AutoV2 if version id json fails. The reason for both, is because for some unknown reason people can upload the exact same files, so you may have multiple models may have files with the same AutoV2 value. Not sure how the API treats this.

[zixaphir]

I lost a lot of data with this. I should've backed up. A mistake on my behalf.

I apologize for this. A scenario where a model has been deleted completely slipped my mind.

Finally not an issue. I thought you might end up merging the data into the json file and thus not needing the info file.

As you can already see with this issue, changing the underlying files can have unforeseen circumstances that I am at high-risk for not considering, so this is not something I'm looking to undertake at the moment. Just off the top of my head, I'd have to consider scenarios where the user already has working JSON, but the model itself was removed from Civitai, so if I forgot to consider that scenario, the user could end up with data that worked previously, but no longer does after a re-scan, just like it did here.

It uses the version id I think or maybe modelid to get the JSON data. Not good as sometimes models are removed and then re-uploaded.

We calculate the hash and then send the hash to Civitai's API, which gives us a model id / version id. There really isn't a way to compare a file against multiple entries in the API. I've written in a check so that if we already have a model id from a previous scan, new data with a mismatched model id will not overwrite old data (tho mismatched version ids are not considered.)

[JingJang]

No problem. This is all for free, so I am very grateful.

`We calculate the hash and then send the hash to Civitai's API, which gives us a model id / version id. There really isn't a way to compare a file against multiple entries in the API. I've written in a check so that if we already have a model id from a previous scan, new data with a mismatched model id will not overwrite old data (tho mismatched version ids are not considered.)

Ignore the second post, it's a mess and wrong.

I was wondering how do you update? What happens when you downloaded the info file, but the model gets removed and then re-uploaded. Do you check using the saved id or save hash? That's it. Not sure how one would handle that. Because the old info file might be more useful than the re-upload.

[zixaphir]

It will use the hash to query Civitai's API and use whatever civitai gives it. Currently the update will fail if the id does not match the current id in stored [model_name].civitai.info.

https://github.com/zixaphir/Stable-Diffusion-Webui-Civitai-Helper/blob/cff09038a7bdc6a22cd46e029d484e82e23b4956/scripts/ch_lib/model.py#L127-L132