Incompatible snapshot.json between 0.4.36 and 0.4.38

[giahung24]

Hi, I have set up an Evidently project to track my data drift. All of my past snapshot are stored with the version 0.4.36 (or below). Today when I accidentally update to 0.4.38, the snapshots saved to the project directory can not be loaded into Dashboard UI (local).

Quick comparison of two json versions gives me this :

Snapshot generated by 0.4.38:

{
    "id": "0192480f-6224-7f94-9c93-0f0372f76cd1",
    "suite": {
        "metrics": [
            {
                **"type": "evidently:metric:DataDriftTable",**
                "options": {
                    "color": null,
                    "render": null,
                    "data_definition": null
                },

Snapshot generated by 0.4.36:

{
    "id": "0192480f-6224-7f94-9c93-0f0372f76cd1",
    "suite": {
        "metrics": [
            {
                **"type": "evidently.metrics.data_drift.data_drift_table.DataDriftTable",**
                "options": {
                    "color": null,
                    "render": null,
                    "data_definition": null
                },

Can you confirm the change in metrics type name between two versions (evidently:metric:DataDriftTable vs. evidently.metrics.data_drift.data_drift_table.DataDriftTable)?

Can you verify if this change breaks the behavior of the local Dashboard UI ? Because if I have these two versions of snapshot in the project dir, when I restart the web ui, I get a 500 error like this :

INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8088 (Press CTRL+C to quit)
INFO:     10.192.4.199:60998 - "GET /projects/60aec1ec-0c0d-4865-8a2c-203a5573b779 HTTP/1.1" 200 OK
INFO:     10.192.4.199:60998 - "GET /static/js/index-DrOTuKLf.js HTTP/1.1" 200 OK
INFO:     10.192.4.199:61000 - "GET /static/css/index-_rehp6Bp.css HTTP/1.1" 200 OK
INFO:     10.192.4.199:60998 - "GET /static/js/vendor-Bhc2M4pH.js HTTP/1.1" 200 OK
INFO:     10.192.4.199:61000 - "GET /static/js/index-BMd0dgrZ.js HTTP/1.1" 200 OK
INFO:     10.192.4.199:60998 - "GET /api/version HTTP/1.1" 200 OK
INFO:     10.192.4.199:61001 - "GET /static/js/createSvgIcon-BCx0yKii.js HTTP/1.1" 200 OK
INFO:     10.192.4.199:60998 - "GET /api/projects/60aec1ec-0c0d-4865-8a2c-203a5573b779/info HTTP/1.1" 500 Internal Server Error
INFO:     10.192.4.199:61000 - "GET /api/projects/60aec1ec-0c0d-4865-8a2c-203a5573b779/dashboard HTTP/1.1" 500 Internal Server Error

[mike0sv]

Hi @giahung24 ! We indeed changed metrics type name in new version, however old typenames should be compatible since they are just classpathes. I just ran this snippet and it worked

from pydantic import parse_obj_as

from evidently.base_metric import Metric
from evidently.metrics import DataDriftTable

def main():
    metric = DataDriftTable(columns=["a"])
    payload = metric.dict()
    print(payload["type"])
    payload["type"] = "evidently.metrics.data_drift.data_drift_table.DataDriftTable"
    parse_obj_as(Metric, payload)

if __name__ == '__main__':
    main()

So 500 errors are something else. Or maybe you run new snapshots with old UI version?

[mike0sv]

You can run UI in debug mode with this code and see full stacktrace

from evidently.ui.app import run
from evidently.ui.local_service import LocalConfig
from evidently.ui.local_service import LocalServiceComponent
config = LocalConfig(service=LocalServiceComponent(debug=True))
run(config)

[giahung24]

Hi @giahung24 ! We indeed changed metrics type name in new version, however old typenames should be compatible since they are just classpathes. I just ran this snippet and it worked

from pydantic import parse_obj_as

from evidently.base_metric import Metric
from evidently.metrics import DataDriftTable

def main():
    metric = DataDriftTable(columns=["a"])
    payload = metric.dict()
    print(payload["type"])
    payload["type"] = "evidently.metrics.data_drift.data_drift_table.DataDriftTable"
    parse_obj_as(Metric, payload)

if __name__ == '__main__':
    main()

So 500 errors are something else. Or maybe you run new snapshots with old UI version?

Hi @mike0sv,

Yeah my UI was always on old version (0.4.36) when I accidentally added the new snapshots from 0.4.38. So, we are agreed that the new shapshots can't be load with old UI ?

[mike0sv]

Yes, you can load old snapshots with new UI but can't load new snapshots with old UI

[giahung24]

Okay, thank you for your verification !