How to track system logs manually & legitimately, when cli convert implemented?

[hjoonjang]

❓Question

Following #2348, I'd like to implement several features more on cli convert wandb. But for now, I have some pain points to implement those ideas, which, I think, the maintainer might have the keys to solve. The followings are describing each of my implementation goal and its related questions.

(Assuming I'm on the way implementing cli convert wandb specifically.)

• Tracking system logs
  • Bringing system logs from wandb api has no big problem.
  • And cpu, gpu utils are successfully tracked & loaded with aim.ext.resource.configs.AIM_RESOURCE_METRIC_PREFIX(i.e. '__system__') setup in my implementation referring aim/ext/resource/tracker.py.
  • But except for cpu / gpu utils, other system logs, e.g. memory, temp, powerPercent, etc., belong to Metrics tab, not to System tab, and they don't seem to be shown normally even in the Metric tab.
  • 
  • Question:
  • Are there hidden rules to classify system logs internally? Then how can we register (inject) system logs legitimately into aim_run client seamlessly? I need your help..🫠
• ~Tracking metric / system logs with time info~ (solved!)
  • ~Tracking both step & epoch seems to be done by passing steps= & epochs= args to run.track().~
  • ~Questions:~
  • ~How can we pass relative & wall time info to the tracker?~

[alberttorosyan]

Hi @hjoonjang. Seems the reason of other system metrics not being shown is the fact that we use the pre-defined list of names in UI. Here's for example the part where the metric names are mapped to more user-friendly texts: https://github.com/aimhubio/aim/blob/cb5472f9cc43317a8e4b0ef013f03cec127a5baf/aim/web/ui/src/config/systemMetrics/systemMetrics.ts#L4 The quick fix would be to map metric names defined in wandb to the ones expected by Aim UI. For more robust solution, @roubkar, can you please take a look?

[hjoonjang]

Hi @hjoonjang. Seems the reason of other system metrics not being shown is the fact that we use the pre-defined list of names in UI. Here's for example the part where the metric names are mapped to more user-friendly texts:

https://github.com/aimhubio/aim/blob/cb5472f9cc43317a8e4b0ef013f03cec127a5baf/aim/web/ui/src/config/systemMetrics/systemMetrics.ts#L4

The quick fix would be to map metric names defined in wandb to the ones expected by Aim UI. For more robust solution, @roubkar, can you please take a look?

@alberttorosyan, thanks for your help! I've applied your 'quick fix' suggestion to my implementation (#2403), and of course, there were remaining unpaired names which I think are not perfectly robust but inevitable in lossy conversion.

If there is another way to make this better, please let me know again. Thanks 👍🏻