search

codalab / chalab

1 stars 4 forks source link

Bug in metric loading #89

Closed madclam closed 7 years ago

madclam commented 7 years ago

On this page:

image

It is possible to upload files starting with # comments, nor with ''' comments, like:

'''Examples of organizer-provided metrics.
You can just replace this code by your own.
Make sure to indicate the name of the function that you chose as metric function
in the file metric.txt. E.g. mse_metric, because this file may contain more 
than one function, hence you must specify the name of the function that is your metric.'''

import numpy as np
import scipy as sp

def mse_metric(solution, prediction):
    '''Mean-square error.
    Works even if the target matrix has more than one column'''
    mse = np.mean((solution-prediction)**2)
    return np.mean(mse)
Tthomas63 commented 7 years ago

So If I understand correctly, also look for a description with ''' triple docstring comments instead of just singular comments # ?

Tthomas63 commented 7 years ago

I am pushing the fix for this to the LRI server right now. The issue was a null reference, added a check to handle it. Triple doc-strings will not be recognized as a description yet.