Open SGevorg opened 3 years ago
Hey @SGevorg,
let's use the existing image classification w/ PyTorch template as an example here. As far as I can see, using Aim would just mean adding
aim.set_params({"lr": lr, "batch_size": batch_size, "num_epochs": num_epochs}, name="hparams")
around line 85 and adding
aim.track(metrics["loss"], name="loss", subset=name, epoch=epoch)
aim.track(metrics["accuracy"], name="accuracy", subset=name, epoch=epoch)
around line 220 (+ adding import and pip install of course).
Questions:
track
method just use subset to differentiate between train/val/test or should the metrics also be named train_loss/val_loss/test_loss etc?@jrieke thanks for the instructions. Adding @gorarakelyan here too to help with the code.
re train/val/test
they are provided as arguments to track - any additional argument converted into a context (key-value pairs). In this case you woul just call aim.track(metrics["loss"], name="loss", epoch=epoch, subset="train")
and subsequent ones for test, val too.
Hi @jrieke ,
All correct, thanks for your effort!
hparams
to be more specifictrack
function, which I see has been perfectly done here:
aim.track(metrics["loss"], name="loss", -> subset=name <- , epoch=epoch)
Would be happy to help in case of more questions. Do you want me to open a PR?
This is an awesome project!
Hi @gorarakelyan,
yes, feel free to open a PR if you have time, otherwise, I'll do it later (not that much work anyway) and will send it to you for testing. Make sure to add Aim as an option in the sidebar here. You can also update the scikit-learn template if you like.
Best, Johannes
Great, thanks for the instructions, @jrieke. I will work on the PR and ping you once it is ready.
Hi @jrieke ,
I've added Aim to PyTorch image classification template and opened a PR. Can you have a look at it? Thanks a lot!
@gorarakelyan Sorry for the slight delay. I only changed 2 small things on your fork (hope that was OK; I added aim to the pip install
line at the top of the generated code + moved the aim import a bit further down) and merged. It's now live on the website! 🥳
Thanks again for the contribution ❤️ Feel free to also add it to the scikit-learn template if you like.
Btw I noticed 2 minor issues with aim when running on my machine:
This warning
/Users/jrieke/opt/anaconda3/lib/python3.7/site-packages/aimrecords/artifact_storage/storage.py:3: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working.
from collections import Iterator`
When I delete the .aim
folder before closing aim_session
or finishing the script (which is what happens when I run pytest because I create a temporary dir for each test run, which gets deleted before the tests finish), I get this error:
Something went wrong in _close. The process will continue to execute.
_close: [Errno 2] No such file or directory: '/private/var/folders/1j/szl2n8g55053cfpnpdr_bpfc0000gn/T/tmp65jegpiw/.aim/default/8a8bbb00-5a00-11eb-8eac-8c8590b82ad8/objects/.aimrecords_storage/loss/current_bucket.bin'`
Nothing major but thought I'll mention it in case you want to look into it :)
Awesome! 🎉
Thanks a lot for the review and for reporting issues.
Thanks for the great work. It would be great to add Aim to train generator as an experiment logger option.
Aim is the most advanced open source experiment comparison tool available at the moment.