search

microsoft / nni

An open source AutoML toolkit for automate machine learning lifecycle, including feature engineering, neural architecture search, model compression and hyper-parameter tuning.
https://nni.readthedocs.io
MIT License
14k stars 1.81k forks source link

[feature_request] `nnictl` with batch operation #5647

Open busFred opened 1 year ago

busFred commented 1 year ago

I have conducted multiple experiments with nnictl create and below is a list of all the experiments that I have conducted.

(ace) fred@fred-ThinkPad-T480s:~$ nnictl experiment list --all
----------------------------------------------------------------------------------------
                Experiment information
Id: xfjd2uhs    Name: p_1.0_niter_5000    Status: STOPPED    Port: 8080    Platform: local    StartTime: 2023-07-19 21:55:13    EndTime: N/A
Id: p4jsgcdm    Name: p_0.8_niter_5000    Status: STOPPED    Port: 8080    Platform: local    StartTime: 2023-07-19 17:24:00    EndTime: N/A
Id: kzlxhp9b    Name: p_0.5_niter_5000    Status: STOPPED    Port: 8080    Platform: local    StartTime: 2023-07-19 17:25:58    EndTime: N/A
Id: n7duobet    Name: p_0.3_niter_5000    Status: STOPPED    Port: 8080    Platform: local    StartTime: 2023-07-19 17:27:00    EndTime: N/A

----------------------------------------------------------------------------------------

I need to do some post processing with the experiment result in python, so I need to export experiment results into csv files. However, I currently have to manually run nnictl view <exp_id> && nnictl export <exp_id> --type csv --filename <exp_name>_<exp_id>.csv && nnictl close --all for each experiment on the list, which is extremely tedious.

Is it possible to have a command that do these operation in batch? I also don't quite understand why I have to "view" the experiment before exporting. Can't nnictl export view and close the experiment on its own if the service isn't running yet?