search

jeff1evesque / machine-learning

Web-interface + rest API for classification and regression (https://jeff1evesque.github.io/machine-learning.docs)
Other
256 stars 85 forks source link

Partition 'base_data.py' into dedicated modules #2582

Closed jeff1evesque closed 8 years ago

jeff1evesque commented 8 years ago

We will move some logic from base_data.py into designated modules, then include them respectively, within base_data.py.

jeff1evesque commented 8 years ago

Our unit tests generate the following traceback:

vagrant@vagrant-ubuntu-trusty-64:/vagrant/test$ py.test
============================= test session starts ==============================

platform linux2 -- Python 2.7.6, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /vagrant/test, inifile: pytest.ini
collected 4 items

programmatic_interface/pytest_session.py ..FF

=================================== FAILURES ===================================

_____________________________ check_model_generate _____________________________

    def check_model_generate():
        """@check_model_generate

        This method tests the 'model_generate' session.

        """

>       assert requests.post(
            endpoint_url,
            headers=headers,
            data=get_sample_json('svm-model-generate.json')
        )
E       assert <Response [500]>
E        +  where <Response [500]> = <function post at 0x7f887fa20ed8>('http://l
ocalhost:5000/load-data/', headers={'Content-Type': 'application/json'}, data='{
"properties": {"model_type": "classification", "sv_kernel_type": "rbf", "session
_id": "1", "session_type": "model_generate"}}')
E        +    where <function post at 0x7f887fa20ed8> = requests.post
E        +    and   '{"properties": {"model_type": "classification", "sv_kernel_
type": "rbf", "session_id": "1", "session_type": "model_generate"}}' = get_sampl
e_json('svm-model-generate.json')

programmatic_interface/pytest_session.py:97: AssertionError
_____________________________ check_model_predict ______________________________

    def check_model_predict():
        """@check_model_predict

        This method tests the 'model_predict' session.

        """

>       assert requests.post(
            endpoint_url,
            headers=headers,
            data=get_sample_json('svm-model-predict.json')
        )
E       assert <Response [500]>
E        +  where <Response [500]> = <function post at 0x7f887fa20ed8>('http://l
ocalhost:5000/load-data/', headers={'Content-Type': 'application/json'}, data='{
"properties": {"model_id": "1", "prediction_input[]": ["22.22", "96.24", "338",
"72.55", "0.001", "28", "0.678"], "session_type": "model_predict"}}')
E        +    where <function post at 0x7f887fa20ed8> = requests.post
E        +    and   '{"properties": {"model_id": "1", "prediction_input[]": ["22
.22", "96.24", "338", "72.55", "0.001", "28", "0.678"], "session_type": "model_p
redict"}}' = get_sample_json('svm-model-predict.json')

programmatic_interface/pytest_session.py:111: AssertionError
====================== 2 failed, 2 passed in 6.18 seconds ======================

The following is the corresponding error traceback:

vagrant@vagrant-ubuntu-trusty-64:/vagrant$ python app.py
/vagrant/brain/database/db_query.py:119: Warning: Field 'model_type' doesn't hav
e a default value
  self.cursor.execute(sql_statement, sql_args)
Traceback (most recent call last):
  File "app.py", line 66, in <module>
    app.run(host='0.0.0.0')
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 843, in run
    run_simple(host, port, self, **options)
  File "/usr/local/lib/python2.7/dist-packages/werkzeug/serving.py", line 694, i
n run_simple
    inner()
  File "/usr/local/lib/python2.7/dist-packages/werkzeug/serving.py", line 659, i
n inner
    srv.serve_forever()
  File "/usr/local/lib/python2.7/dist-packages/werkzeug/serving.py", line 499, i
n serve_forever
    HTTPServer.serve_forever(self)
  File "/usr/lib/python2.7/SocketServer.py", line 238, in serve_forever
    self._handle_request_noblock()
  File "/usr/lib/python2.7/SocketServer.py", line 297, in _handle_request_nobloc
k
    self.handle_error(request, client_address)
  File "/usr/lib/python2.7/SocketServer.py", line 295, in _handle_request_nobloc
k
    self.process_request(request, client_address)
  File "/usr/lib/python2.7/SocketServer.py", line 321, in process_request
    self.finish_request(request, client_address)
  File "/usr/lib/python2.7/SocketServer.py", line 334, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib/python2.7/SocketServer.py", line 651, in __init__
    self.finish()
  File "/usr/lib/python2.7/SocketServer.py", line 710, in finish
    self.wfile.close()
  File "/usr/lib/python2.7/socket.py", line 279, in close
    self.flush()
  File "/usr/lib/python2.7/socket.py", line 303, in flush
    self._sock.sendall(view[write_offset:write_offset+buffer_size])
socket.error: [Errno 32] Broken pipe
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ python app.py
/vagrant/brain/database/db_query.py:119: Warning: Field 'model_type' doesn't hav
e a default value
  self.cursor.execute(sql_statement, sql_args)
[2016-06-05 23:13:01,589] ERROR in app: Exception on /load-data/ [POST]
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1988, in wsgi
_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1641, in full
_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1544, in hand
le_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1639, in full
_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1625, in disp
atch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/vagrant/interface/views.py", line 68, in load_data
    response = loader.load_model_generate()
  File "/vagrant/brain/load_data.py", line 141, in load_model_generate
    session.generate_model()
  File "/vagrant/brain/session/model_generate.py", line 62, in generate_model
    self.list_error
  File "/vagrant/brain/session/model/svm.py", line 54, in svm_model
    feature_count = feature_count['result'][0][0]
IndexError: tuple index out of range
[2016-06-05 23:13:01,630] ERROR in app: Exception on /load-data/ [POST]
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1988, in wsgi
_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1641, in full
_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1544, in hand
le_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1639, in full
_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1625, in disp
atch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/vagrant/interface/views.py", line 70, in load_data
    response = loader.load_model_predict()
  File "/vagrant/brain/load_data.py", line 165, in load_model_predict
    my_prediction = session.predict()
  File "/vagrant/brain/session/model_predict.py", line 66, in predict
    return svm_prediction('svm', 'rbf', self.model_id, self.predictors)
  File "/vagrant/brain/session/predict/svm.py", line 32, in svm_prediction
    model_id + '_' + title
TypeError: coercing to Unicode: need string or buffer, NoneType found
jeff1evesque commented 8 years ago

Last evening, the unit tests took longer than usual to run, so it was manually killed during execution. Successive attempts to run the unit tests, yielded the above trace. However, I manually truncated all tables within the db_machine_learning. Now, successive unit test implementation succeeds:

vagrant@vagrant-ubuntu-trusty-64:/vagrant/test$ py.test
============================= test session starts ==============================

platform linux2 -- Python 2.7.6, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /vagrant/test, inifile: pytest.ini
collected 4 items

programmatic_interface/pytest_session.py ....

========================== 4 passed in 17.79 seconds ===========================