search

google / model_search

Apache License 2.0
3.26k stars 462 forks source link

Evaluating Model Search Results #47

Open malekinho8 opened 3 years ago

malekinho8 commented 3 years ago

As a test, I just ran the supplied code with all of the default parameters for the "Getting Started" example, just to make sure I installed everything correctly and that the code would run smoothly. The test has now finished, and I saw that the model search put all the outputs in a 'tmp\run_example' directory, with each model getting its own folder numbered from 1 - 200. I would now like to evaluate the results of this run, but I have no idea where to even start. I would like to see things like optimal model architecture, architectures evaluated, architecture with highest ratio of accuracy to # parameters, etc., but I am just lost as to how to go about doing that. Would anyone be able to offer any guidance in that area?

I tried loading in a model taking inspiration from the following tutorial on Tensorflow, image

but ultimately I got an error after running the code below:

M = tf.keras.models.load_model("C:\\tmp\\run_example\\tuner-1\\200\\saved_model\\1616871647") M.summary() Traceback (most recent call last): File "<string>", line 1, in <module> AttributeError: 'AutoTrackable' object has no attribute 'summary'

Does anyone have any ideas how to get information about the model architectures from these runs?

Xiaoping777 commented 3 years ago

it can be identified from the tensorboard

malekinho8 commented 3 years ago

it can be identified from the tensorboard

@Xiaoping777 Would you be able to paste a couple lines of code to show what you mean exactly? I've never use Tensorboard so I'm not sure where to even start.

Xiaoping777 commented 3 years ago

You can Run TensorBoard from command line: 1) cd tensorflow folder: cd /data/.local/lib/python3.6/site-packages/tensorboard/ 2) python3 -m tensorboard.main --logdir="/data/tmp/run_sample/" --host='localhost' --port=6010

malekinho8 commented 3 years ago

Thanks for the help @Xiaoping777

Is this what you are seeing with Tensorboard? I've zoomed in and I don't see where I'm supposed to find the model architecture. I just see a bunch of seemingly randomly labeled blocks...

image

Xiaoping777 commented 3 years ago

Main Graph part gives you the Graph, if you check Scalars tab, it gives the model performance

Andy546 commented 3 years ago

Thanks for the help @Xiaoping777

Is this what you are seeing with Tensorboard? I've zoomed in and I don't see where I'm supposed to find the model architecture. I just see a bunch of seemingly randomly labeled blocks...

image

how can i build network with this tensorboar?

anaptoro commented 2 years ago

@Xiaoping777 My tensorflow scalars bar is only showing things like this: image

With no difference between the models but the time they were executed... (As input I used the example csv provided in the repo)