iterative / cml

♾️ CML - Continuous Machine Learning | CI/CD for ML
http://cml.dev
Apache License 2.0
4k stars 339 forks source link

[Failed guessing mime type of file] Unable to publish ".png" image to the report. #401

Closed MaxKumundzhiev closed 3 years ago

MaxKumundzhiev commented 3 years ago

Hi there !

Description

Based on the proposed by the DVCorg community MLOps tutorial I was attaching the calculated and saved confusion_matrix.png image to the report. But I continuously obtained the error:

Error: Failed guessing mime type of file result/confusion_matrix.png
    at mime_type (/cml/src/utils.js:49:11)
Error: Process completed with exit code 1.

The .github/workflow/expirement.cml file content:

name: experiment-workflow
on: [push]
jobs:
  run:
    runs-on: [ubuntu-latest]
    container: docker://dvcorg/cml-py3:latest
    steps:
      - uses: actions/checkout@v2
      - name: cml_run
        env:
          repo_token: ${{ secrets.GITHUB_TOKEN }}
        run: |

          pip install -r requirements.txt
          python expirement.py

          # CML REPORT
          echo "## Model Metrics" >> report.md
          cat results/metrics.txt >> report.md

          echo "## Confusion Matrix" >> report.md
          cml-publish results/confusion_matrix.png --md >> report.md

          cml-send-comment report.md

The sample of the expirement.py file content:

# imports
...

_FOLDER = Path('./results')
_METRICS = Path('metrics.txt')
_CONFUSION_MATRIX = Path('confusion_matrix.png')

shutil.rmtree(_FOLDER, ignore_errors=False)
Path(_FOLDER).mkdir(exist_ok=True)

# example of model output
target = [1, 1, 1, 0, 0, 1]
predicted = [1, 1, 1, 1, 1, 0]

results_dict = {
    'Precision': MetricsFactory.precision(target=target, predicted=predicted),
    'Recall': MetricsFactory.recall(target=target, predicted=predicted),
    'ROC AUC': MetricsFactory.roc_auc(target=target, predicted=predicted),
    'F1 Score': MetricsFactory.f1(target=target, predicted=predicted),
    'Confusion Matrix': MetricsFactory.confusion_matrix(target=target, predicted=predicted)
}

with open(_FOLDER / _METRICS, 'w') as f:
    f.write(str(results_dict))

_confusion_matrix = pd.DataFrame(results_dict['Confusion Matrix'])

plt.figure(figsize=(10, 7))
sn.heatmap(_confusion_matrix, annot=True)
plt.savefig(_FOLDER / _CONFUSION_MATRIX)

Summary

The error denotes an issue with the confusion_matrix.png file. In the meantime I can open and validate the image exists and format is not corrupted.
Could you please give a hint what's going wrong ? What did I miss if so ?

DavidGOrtega commented 3 years ago

👋 @KumundzhievMaxim At first sight everything seems to be right in your workflow. Let me do a check. Also, is there any chance to see the repo? Is it public?

DavidGOrtega commented 3 years ago

Could you please check that results/confusion_matrix.png is relative to cml-publish?

DavidGOrtega commented 3 years ago

@KumundzhievMaxim did you find the issue?

DavidGOrtega commented 3 years ago

related to #308

MaxKumundzhiev commented 3 years ago

👋 @DavidGOrtega, apologies for the delayed response.

Also, is there any chance to see the repo? Is it public?

Yes, the repo is private

MaxKumundzhiev commented 3 years ago

Good news @DavidGOrtega, the issue can be closed I presume !

Explanation

Could you please check that results/confusion_matrix.png is relative to cml-publish?

Yes, the path is correct.

Did you find the issue?

Eventually I entirely reassembled the project and the results appeared in the generated with GitHub actions report. Appreciate for rapid response and effort @DavidGOrtega

P.S. The sample report output snapshot: Screenshot 2021-02-28 at 12 55 53