iterative / dvc

🦉 ML Experiments and Data Management with Git
https://dvc.org
Apache License 2.0
13.36k stars 1.16k forks source link

params/metrics: do not fail during dir expansion #10375

Closed skshetry closed 3 months ago

skshetry commented 3 months ago

Closes https://github.com/iterative/dvc/issues/10018.

We have to be careful when expanding paths. This PR fall backs to assume it as a file when either of isdir and find raises error due to some mishaps. The consequence of this assumption happens when we try to read the file. In that case, if the directory exists, and we happen to try to open it as a file, dvc will raise IsADirectoryError, but that is handled correctly and won't fail catastrophically like this did.

This is what I get now:

$ dvc metrics show --json
DVC failed to load some metrics for following revisions: ''.
{
  "": {
    "data": {
      "results/train/metrics.json": {
        "error": {
          "type": "DataIndexDirError",
          "msg": "failed to load directory ('results', 'train')"
        }
      },
      "results/evaluate/metrics.json": {
        "data": {
          "dice_multi": 0.8789161007343483
        }
      }
    }
  }
}

The error message is not great, but at least it won't fail.

codecov[bot] commented 3 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 90.71%. Comparing base (5379aa2) to head (02453e0).

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #10375 +/- ## ======================================= Coverage 90.70% 90.71% ======================================= Files 501 501 Lines 38837 38865 +28 Branches 5616 5618 +2 ======================================= + Hits 35229 35257 +28 Misses 2964 2964 Partials 644 644 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.