Benchmarking against milk library

[Iron-Stark]

@zoq

In one of my earlier PR's you had suggested a link for installing milk from some other link which seemed less cryptic. Can you please give that to me again. The files in that PR got deleted because I had mistakenly performed a rebase.

[zoq]

It's https://github.com/luispedro/milk/archive/release-0.6.1.tar.gz on the github page: https://github.com/luispedro/milk you can find a section called releases. Also, here is the original PR: https://github.com/mlpack/benchmarks/pull/55

[zoq]

Looks like there is something wrong with the install script (milk):

error: can't create or remove files in install directory

The following error occurred while trying to add or remove files in the installation directory:

    [Errno 2] No such file or directory: '..//lib/python3.5/site-packages/test-easy-install-20182.write-test'

The installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:

    ..//lib/python3.5/site-packages

This directory does not currently exist.  Please create it and try again, or
choose a different installation directory (using the -d or --install-dir
option).

Error installing Milk!
Makefile:188: recipe for target '.setup' failed
make: *** [.setup] Error 1

[Iron-Stark]

@zoq @rcurtin

Why were the shogun tests failing in the previous build. Any ideas on what might be causing them?

[zoq]

Looks like there is an issue with shogun 6.0, I temporarily fixed the issue by testing against 5.0.

[zoq]

@mlpack-jenkins test this

[Iron-Stark]

@zoq

Milk installed successfully so why are we getting the error that there is no module named milk during tests?

[zoq]

I'm not sure the last build installed milk successfully, here is part of the log:

writing build/bdist.linux-x86_64/egg/EGG-INFO/native_libs.txt
zip_safe flag not set; analyzing archive contents...
Traceback (most recent call last):
  File "setup.py", line 97, in <module>
    test_suite = 'nose.collector',
  File "/usr/lib/python3/dist-packages/numpy/distutils/core.py", line 169, in setup
    return old_setup(**new_attr)
  File "/usr/lib/python3.5/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.5/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/usr/lib/python3/dist-packages/numpy/distutils/command/install.py", line 62, in run
    r = self.setuptools_run()
  File "/usr/lib/python3/dist-packages/numpy/distutils/command/install.py", line 56, in setuptools_run
    self.do_egg_install()
  File "/usr/lib/python3/dist-packages/setuptools/command/install.py", line 109, in do_egg_install
    self.run_command('bdist_egg')
  File "/usr/lib/python3.5/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/usr/lib/python3/dist-packages/setuptools/command/bdist_egg.py", line 209, in run
    os.path.join(archive_root, 'EGG-INFO'), self.zip_safe()
  File "/usr/lib/python3/dist-packages/setuptools/command/bdist_egg.py", line 245, in zip_safe
    return analyze_egg(self.bdist_dir, self.stubs)
  File "/usr/lib/python3/dist-packages/setuptools/command/bdist_egg.py", line 355, in analyze_egg
    safe = scan_module(egg_dir, base, name, stubs) and safe
  File "/usr/lib/python3/dist-packages/setuptools/command/bdist_egg.py", line 392, in scan_module
    code = marshal.load(f)
ValueError: bad marshal data (unknown type code)
Error installing Milk!

[zoq]

Looks like you fixed the build error and the only test that fails is the KMeans test, let us know if you need any help with the debugging.

[zoq]

Can you post mlpack's and milk's KMeans output here (e.g. by using covtype and covtype_centroids)? I think once we made sure the output is reasonable it's ready to be merged.

[Iron-Stark]

@zoq

Here are the results:

Dataset mlpack milk wine 0.000119 - - - - - 0.000748 iris 0.000126 - - - - - 0.000671 waveform 0.008536 - - - - - 0.011266 isolet 3.594024 - - - - - 1.102464 covtype 21.906052 - - - - - 13.829839

[Iron-Stark]

@zoq @rcurtin

Here are the numpy arrays of centroids of milk and mlpack after running on wine dataset mlpack: https://paste.ubuntu.com/24901050/ milk: https://paste.ubuntu.com/24901094/

Looks good to me please verify.

[rcurtin]

Looks good to me, thanks for digging in!