search

iterative / mlem

🐶 A tool to package, serve, and deploy any ML model on any platform. Archived to be resurrected one day🤞
https://mlem.ai
Apache License 2.0
717 stars 44 forks source link

Fix spammish warnings #592

Open aguschin opened 1 year ago

aguschin commented 1 year ago

A while ago we introduced a better way to find requirements to be installed, when one serializes a model. This doesn't work perfect, so we decided to issue warnings that should be useful for one to check, if we found something incorrectly.

But now I feel those are too spammish:

⏳️ Loading deployment from docker_container.mlem
⏳️ Loading model from lyrics2emoji.mlem
🛠 Creating docker image lyrics2emoji
2023-01-25 13:24:55,599 [WARNING] mlem.utils.module: Fail to determine package name for module 'click<8.2', using module name instead. If that's incorrect, fix metadata manually, either in `.mlem` file or in your MLEM Python object.
2023-01-25 13:24:55.599 WARNING mlem.utils.module: Fail to determine package name for module 'click<8.2', using module name instead. If that's incorrect, fix metadata manually, either in `.mlem` file or in your MLEM Python object.
2023-01-25 13:24:56,074 [WARNING] mlem.utils.module: Fail to determine package name for module 'aiohttp<4', using module name instead. If that's incorrect, fix metadata manually, either in `.mlem` file or in your MLEM Python object.
2023-01-25 13:24:56.074 WARNING mlem.utils.module: Fail to determine package name for module 'aiohttp<4', using module name instead. If that's incorrect, fix metadata manually, either in `.mlem` file or in your MLEM Python object.
2023-01-25 13:24:56,325 [WARNING] mlem.utils.module: Fail to determine package name for module 'aiohttp_swagger<2', using module name instead. If that's incorrect, fix metadata manually, either in `.mlem` file or in your MLEM Python object.
2023-01-25 13:24:56.325 WARNING mlem.utils.module: Fail to determine package name for module 'aiohttp_swagger<2', using module name instead. If that's incorrect, fix metadata manually, either in `.mlem` file or in your MLEM Python object.
2023-01-25 13:24:56,572 [WARNING] mlem.utils.module: Fail to determine package name for module 'Jinja2', using module name instead. If that's incorrect, fix metadata manually, either in `.mlem` file or in your MLEM Python object.
2023-01-25 13:24:56.572 WARNING mlem.utils.module: Fail to determine package name for module 'Jinja2', using module name instead. If that's incorrect, fix metadata manually, either in `.mlem` file or in your MLEM Python object.
2023-01-25 13:24:57,061 [WARNING] mlem.utils.module: Fail to determine package name for module 'pyparsing<4', using module name instead. If that's incorrect, fix metadata manually, either in `.mlem` file or in your MLEM Python object.
2023-01-25 13:24:57.061 WARNING mlem.utils.module: Fail to determine package name for module 'pyparsing<4', using module name instead. If that's incorrect, fix metadata manually, either in `.mlem` file or in your MLEM Python object.
2023-01-25 13:24:57,302 [WARNING] mlem.utils.module: Fail to determine package name for module 'cached-property', using module name instead. If that's incorrect, fix metadata manually, either in `.mlem` file or in your MLEM Python object.
2023-01-25 13:24:57.302 WARNING mlem.utils.module: Fail to determine package name for module 'cached-property', using module name instead. If that's incorrect, fix metadata manually, either in `.mlem` file or in your MLEM Python object.
2023-01-25 13:24:57,801 [WARNING] mlem.utils.module: Fail to determine package name for module 'gitpython', using module name instead. If that's incorrect, fix metadata manually, either in `.mlem` file or in your MLEM Python object.
2023-01-25 13:24:57.801 WARNING mlem.utils.module: Fail to determine package name for module 'gitpython', using module name instead. If that's incorrect, fix metadata manually, either in `.mlem` file or in your MLEM Python object.
2023-01-25 13:24:58,043 [WARNING] mlem.utils.module: Fail to determine package name for module 'python-gitlab', using module name instead. If that's incorrect, fix metadata manually, either in `.mlem` file or in your MLEM Python object.
2023-01-25 13:24:58.043 WARNING mlem.utils.module: Fail to determine package name for module 'python-gitlab', using module name instead. If that's incorrect, fix metadata manually, either in `.mlem` file or in your MLEM Python object.
2023-01-25 13:24:58,539 [WARNING] mlem.utils.module: Fail to determine package name for module 'iterative-telemetry', using module name instead. If that's incorrect, fix metadata manually, either in `.mlem` file or in your MLEM Python object.
2023-01-25 13:24:58.539 WARNING mlem.utils.module: Fail to determine package name for module 'iterative-telemetry', using module name instead. If that's incorrect, fix metadata manually, either in `.mlem` file or in your MLEM Python object.
2023-01-25 13:24:58,789 [WARNING] mlem.utils.module: Fail to determine package name for module 'python-multipart', using module name instead. If that's incorrect, fix metadata manually, either in `.mlem` file or in your MLEM Python object.
2023-01-25 13:24:58.789 WARNING mlem.utils.module: Fail to determine package name for module 'python-multipart', using module name instead. If that's incorrect, fix metadata manually, either in `.mlem` file or in your MLEM Python object.
2023-01-25 13:24:59,026 [WARNING] mlem.utils.module: Fail to determine package name for module 'importlib-metadata', using module name instead. If that's incorrect, fix metadata manually, either in `.mlem` file or in your MLEM Python object.
2023-01-25 13:24:59.026 WARNING mlem.utils.module: Fail to determine package name for module 'importlib-metadata', using module name instead. If that's incorrect, fix metadata manually, either in `.mlem` file or in your MLEM Python object.
  🛠 Building MLEM wheel file...
  💼 Adding model files...
  🛠 Generating dockerfile...
  💼 Adding sources...
  💼 Generating requirements file...
  🛠 Building docker image lyrics2emoji:latest...

Not talking about deduplicating them (should be done 100%), some options, IMO:

  1. remove them - faster
  2. group them to appear as a single warning.
  3. make them appear at manual mlem.api.save only - and not appear for mlem build docker.
shcheklein commented 1 year ago

I think it's fine to have as many logs as needed. They should though never be visible by default in CLI output (at least not in this form and style). Only if run it in a verbose mode. In this case you could make them TRACE for example. Also make sure that the default logger doesn't print into stdout at all. Warnings, e.g. if we want to group them, should appear as a regular message (w/o timestamps, w/o [WARNING], etc)

mike0sv commented 1 year ago

Can you just make them .debug messages? Btw the ones you see are because you are using dev mlem version I guess - those are mlem requirements and if mlem install from pip you should not see them