problem to plot the learning curve

[RoroKA]

I'm trying to execute "plot_learning_curve.py" to plot the learning curve but it gives me this error message

AttributeError: 'Series' object has no attribute 'find'

Could you please help me?

[mattm401]

I am also looking into this at the moment, don't know if you were able to solve this yourself. A similarly named script also exists in the main caffe repo: https://github.com/BVLC/caffe/blob/master/tools/extra/plot_training_log.py.example

[mattm401]

So, I got this to work by doing the following:

1. Go into caffe\tools\extra\ and edit the parse_log.py file so that you can run it manually so that you can get the two output CSV files. Create an output direction and run "python parse_log.py model_1_train.log ./ouput"

2. Update plot_learning_curve.py, I commented out everything above the loading of the two csv files; had to update header row to match current version of caffe. Then, I ran "python plot_learning_curve.py <path/to/output.png>"

[mattm401]

Looks like I only had to edit pare_log.py so that the header rows matched recent Caffe updates to the log file itself:

row = OrderedDict([ ('NumIters', iteration), ('Seconds', seconds), ('LearningRate', learning_rate) ])

[kmpujar]

That's what my parse_log.py already has. Could you please elaborate on what has to be done? This is my file:

!/usr/bin/env python

""" Parse training log

Evolved from parse_log.sh """

import os import re import extract_seconds import argparse import csv from collections import OrderedDict

def parse_log(path_to_log): """Parse log file Returns (train_dict_list, test_dict_list)

train_dict_list and test_dict_list are lists of dicts that define the table
rows
"""

regex_iteration = re.compile('Iteration (\d+)')
regex_train_output = re.compile('Train net output #(\d+): (\S+) = ([\.\deE+-]+)')
regex_test_output = re.compile('Test net output #(\d+): (\S+) = ([\.\deE+-]+)')
regex_learning_rate = re.compile('lr = ([-+]?[0-9]*\.?[0-9]+([eE]?[-+]?[0-9]+)?)')

# Pick out lines of interesta
iteration = -1
learning_rate = float('NaN')
train_dict_list = []
test_dict_list = []
train_row = None
test_row = None

logfile_year = extract_seconds.get_log_created_year(path_to_log)
with open(path_to_log) as f:
    start_time = extract_seconds.get_start_time(f, logfile_year)
    last_time = start_time

    for line in f:
        iteration_match = regex_iteration.search(line)
        if iteration_match:
            iteration = float(iteration_match.group(1))
        if iteration == -1:
            # Only start parsing for other stuff if we've found the first
            # iteration
            continue

        try:
            time = extract_seconds.extract_datetime_from_line(line,
                                                              logfile_year)
        except ValueError:
            # Skip lines with bad formatting, for example when resuming solver
            continue

        # if it's another year
        if time.month < last_time.month:
            logfile_year += 1
            time = extract_seconds.extract_datetime_from_line(line, logfile_year)
        last_time = time

        seconds = (time - start_time).total_seconds()

        learning_rate_match = regex_learning_rate.search(line)
        if learning_rate_match:
            learning_rate = float(learning_rate_match.group(1))

        train_dict_list, train_row = parse_line_for_net_output(
            regex_train_output, train_row, train_dict_list,
            line, iteration, seconds, learning_rate
        )
        test_dict_list, test_row = parse_line_for_net_output(
            regex_test_output, test_row, test_dict_list,
            line, iteration, seconds, learning_rate
        )

fix_initial_nan_learning_rate(train_dict_list)
fix_initial_nan_learning_rate(test_dict_list)

return train_dict_list, test_dict_list

def parse_line_for_net_output(regex_obj, row, row_dict_list, line, iteration, seconds, learning_rate): """Parse a single line for training or test output

Returns a a tuple with (row_dict_list, row)
row: may be either a new row or an augmented version of the current row
row_dict_list: may be either the current row_dict_list or an augmented
version of the current row_dict_list
"""

output_match = regex_obj.search(line)
if output_match:
    if not row or row['NumIters'] != iteration:
        # Push the last row and start a new one
        if row:
            # If we're on a new iteration, push the last row
            # This will probably only happen for the first row; otherwise
            # the full row checking logic below will push and clear full
            # rows
            row_dict_list.append(row)

        row = OrderedDict([
            ('NumIters', iteration),
            ('Seconds', seconds),
            ('LearningRate', learning_rate)
        ])

    # output_num is not used; may be used in the future
    # output_num = output_match.group(1)
    output_name = output_match.group(2)
    output_val = output_match.group(3)
    row[output_name] = float(output_val)

if row and len(row_dict_list) >= 1 and len(row) == len(row_dict_list[0]):
    # The row is full, based on the fact that it has the same number of
    # columns as the first row; append it to the list
    row_dict_list.append(row)
    row = None

return row_dict_list, row

def fix_initial_nan_learning_rate(dict_list): """Correct initial value of learning rate

Learning rate is normally not printed until after the initial test and
training step, which means the initial testing and training rows have
LearningRate = NaN. Fix this by copying over the LearningRate from the
second row, if it exists.
"""

if len(dict_list) > 1:
    dict_list[0]['LearningRate'] = dict_list[1]['LearningRate']

def save_csv_files(logfile_path, output_dir, train_dict_list, test_dict_list, delimiter=',', verbose=False): """Save CSV files to output_dir

If the input log file is, e.g., caffe.INFO, the names will be
caffe.INFO.train and caffe.INFO.test
"""

log_basename = os.path.basename(logfile_path)
train_filename = os.path.join(output_dir, log_basename + '.train')
write_csv(train_filename, train_dict_list, delimiter, verbose)

test_filename = os.path.join(output_dir, log_basename + '.test')
write_csv(test_filename, test_dict_list, delimiter, verbose)

def write_csv(output_filename, dict_list, delimiter, verbose=False): """Write a CSV file """

if not dict_list:
    if verbose:
        print('Not writing %s; no lines to write' % output_filename)
    return

dialect = csv.excel
dialect.delimiter = delimiter

with open(output_filename, 'w') as f:
    dict_writer = csv.DictWriter(f, fieldnames=dict_list[0].keys(),
                                 dialect=dialect)
    dict_writer.writeheader()
    dict_writer.writerows(dict_list)
if verbose:
    print 'Wrote %s' % output_filename

def parse_args(): description = ('Parse a Caffe training log into two CSV files ' 'containing training and testing information') parser = argparse.ArgumentParser(description=description)

parser.add_argument('logfile_path',
                    help='Path to log file')

parser.add_argument('output_dir',
                    help='Directory in which to place output CSV files')

parser.add_argument('--verbose',
                    action='store_true',
                    help='Print some extra info (e.g., output filenames)')

parser.add_argument('--delimiter',
                    default=',',
                    help=('Column delimiter in output files '
                          '(default: \'%(default)s\')'))

args = parser.parse_args()
return args

def main(): args = parse_args() train_dict_list, test_dict_list = parse_log(args.logfile_path) save_csv_files(args.logfile_path, args.output_dir, train_dict_list, test_dict_list, delimiter=args.delimiter, verbose=args.verbose)

if name == 'main': main()

[mattm401]

parse_log runs on .log files from Caffe... so just point it at your log file and grab the two output files. Comment out the parts of plot_learning_curve that use the .sh script... and then just fiddle with python until you can run plot_learning_curve manually and it only takes the two output files. I think I also manually set what files plot_learning_curve was looking for.

[indsak]

is this problem solved?

[mattm401]

Is for me, but I don't think this GitHub is getting updated as there is currently another pull request that is very old.

[indsak]

i tried to do as per your suggestion but still my problm is not cleared

[mattm401]

What error message are you getting?

[indsak]

When i tried to execute "plot_learning_curve.py" to plot the learning curve, it gave the error messageAttributeError: 'Series' object has no attribute 'find'

Then i added the following lines as i saw in the issue reported by Fareeha in the site http://adilmoujahid.com/posts/2016/06/introduction-deep-learning-python-caffe/

train_log = train_log.astype(float) test_log = test_log.astype(float) But that was giving errr for float. Would liketo know how to solve the issue.

I also tried your suggestion to edit the parse_log.py in caffe\tools\extra\ . BUt i didnt quite understand that. It would be helpful if i get any input to clear the issue.

On Wed, Feb 22, 2017 at 12:11 AM, Matthew Louis Mauriello < notifications@github.com> wrote:

What error message are you getting?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/adilmoujahid/deeplearning-cats-dogs-tutorial/issues/3#issuecomment-281436903, or mute the thread https://github.com/notifications/unsubscribe-auth/AWAQiZyoeLjN6bR2eDqdmgEp8ImTOKz3ks5rey_jgaJpZM4LJ53b .

[mattm401]

I edited my plot_learning_curvey.py file as followed:

import os import sys import subprocess import pandas as pd

import matplotlib matplotlib.use('Agg') import matplotlib.pylab as plt

plt.style.use('ggplot')

caffe_path = 'C:\Users\mattm401\Desktop\caffe\' model_log_path = sys.argv[1] learning_curve_path = sys.argv[2]

Get directory where the model logs is saved, and move to it

model_log_dir_path = os.path.dirname(model_log_path)

os.chdir(model_log_dir_path)

''' Generating training and test logs DID THIS PART MANUALLY '''

Parsing training/validation logs

command = caffe_path + 'tools\extra\parse_log.sh ' + model_log_path

process = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE)

process.wait()

Read training and test logs

train_log_path = model_log_path + 'model_1_train.log.train' test_log_path = model_log_path + 'model_1_train.log.test' train_log = pd.read_csv(train_log_path, delim_whitespace=True) test_log = pd.read_csv(test_log_path, delim_whitespace=True)

''' Making learning curve ''' fig, ax1 = plt.subplots()

Plotting training and test losses

train_loss, = ax1.plot(train_log['NumIters'], train_log['loss'], color='red', alpha=.5) test_loss, = ax1.plot(test_log['NumIters'], test_log['loss'], linewidth=2, color='green') ax1.set_ylim(ymin=0, ymax=1) ax1.set_xlabel('Iterations', fontsize=15) ax1.set_ylabel('Loss', fontsize=15) ax1.tick_params(labelsize=15)

Plotting test accuracy

ax2 = ax1.twinx() test_accuracy, = ax2.plot(test_log['NumIters'], test_log['accuracy'], linewidth=2, color='blue') ax2.set_ylim(ymin=0, ymax=1) ax2.set_ylabel('Accuracy', fontsize=15) ax2.tick_params(labelsize=15)

Adding legend

plt.legend([train_loss, test_loss, test_accuracy], ['Training Loss', 'Test Loss', 'Test Accuracy'], bbox_to_anchor=(1, 0.8)) plt.title('Training Curve', fontsize=18)

Saving learning curve

plt.savefig(learning_curve_path)

''' Deleting training and test logs '''

command = 'del ' + train_log_path

process = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE)

process.wait()

command = command = 'del ' + test_log_path

process = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE)

process.wait()

[mattm401]

First, I ran parse_log.py (C:\Users\mattm401\Desktop\caffe\tools\extra) on the *_train.log file and directed to an output folder where it produced the test and training files used in plot_learning_curve.py. Because I ran parse_log.py manually, I commented out some things in plot_learning_curve.py (see above), set some file names manually, and I think I had to update the Column Names (Headers) in the ax#.plot() function calls.

Sorry for not posting better notes in previous comments... but with a little editing this should work.

[mattm401]

I thought I also had to make some modifications to plot_learning_curve.py as well, but this may be unnecessary.

[indsak]

thank you for posting this. I wil try this once my training gets over. As it is CPU looks like is going to take one more day to finish

On Thu, Feb 23, 2017 at 4:22 AM, Matthew Louis Mauriello < notifications@github.com> wrote:

I thought I also had to make some modifications to plot_learning_curve.py as well, but this may be unnecessary.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/adilmoujahid/deeplearning-cats-dogs-tutorial/issues/3#issuecomment-281831885, or mute the thread https://github.com/notifications/unsubscribe-auth/AWAQiYIAjYbUtrx4Hs1YsyM790JLDMwpks5rfLxJgaJpZM4LJ53b .

[clancyian]

A small change to caffe parse_log.sh file resoved this issue for me. On line 42 change : grep ', loss = ' $1 | awk '{print $9}' > aux1.txt to grep ', loss = ' $1 | awk '{print $13}' > aux1.txt

[kmpujar]

That works like a charm.. Thankyou clancyian. But how did you come up with the solution?

[clancyian]

Hi kmpujar, Your welcome. Not much too it really. Just some good old fashioned follow the code. My guess is that the logging output in caffe was updated at some point and the parse_log.sh was not updated at the same time to reflect this. The extra log data pushed out the columd we were looking for from 9 to 13. The shell script is quite brittle though and suceptable to to any future changes in the logging format. Maybe there is simiar issue with the python script parse_log.py, I have not looked yet.

[indsak]

Thank you very much

On Wed, Mar 8, 2017 at 10:20 PM, clancyian notifications@github.com wrote:

A small change to caffe parse_log.sh file resoved this issue for me. On line 42 change : grep ', loss = ' $1 | awk '{print $9}' > aux1.txt to grep ', loss = ' $1 | awk '{print $13}' > aux1.txt

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/adilmoujahid/deeplearning-cats-dogs-tutorial/issues/3#issuecomment-285097591, or mute the thread https://github.com/notifications/unsubscribe-auth/AWAQia7jP-m0W0qE9y9NxGEwTczy5FTxks5rjtxxgaJpZM4LJ53b .

[marikhu]

Thanks clancyian. You rock!

[MFarooqAit]

"parse_log.py" is not run and gave the too few argument error as shown below:

siraj@siraj-System-Product-Name:~/Programs/caffe-master/tools/extra$ parse_log.py lenet_train.log usage: parse_log.py [-h] [--verbose] [--delimiter DELIMITER] logfile_path output_dir parse_log.py: error: too few arguments

Could you kind me in this regards.