Modify new CI setup files to pass tests

[nikohansen]

Copied from https://github.com/numbbo/coco/pull/1478#issue-248248007

Hi @nikohansen and @brockho. Now that you have a new CI set up, we should make it useful. I suggest to run all the available COCO unit tests. I have already tried to add your unit tests to .circleci/config.yml and appveyor.yml, however, I struggle to make the tests pass.

1. Windows fails to pass test-c with this error (WindowsError: [Error 2] The system cannot find the file specified). I am not sure what file is it missing, maybe there is something more that has to be done before running test-c? @brockho will probably know more.
2. Ubuntu Python 2 fails to pass postprocessing tests with this error (Test failed: error while running bibtex on templateBBOBarticle.). I may be missing some special latex dependency installed, but I have no Idea which one.
3. Ubuntu Python 3 fails to pass some preprocessing or postprocessing tests with this error (ImportError: No module named setuptools). This one may be caused by the fact that somewhere in the preprocessing/postprocessing code, python 2 is hardcoded (instead of using the python interpreter, which is used to run the main script). I suppose that this error does not appear in Jenkins and your personal computers, since they have both Python 2 and 3 installed. However, in the CircleCI environment, I have deliberately installed only the Python that is really tested.

If you are interested, please push some more edits to .circleci/config.yml and appveyor.yml into this branch to make it pass all the tests. You have the permission to push into it. I believe that you are more experienced with your own unit tests.

P.S.: I had to put in a few more Python 2 to Python 3 conversions to make the tests compile under Python 3, but nothing serious.

[nikohansen]

When calling do.py test-python instead of test-python3 in the Ubuntu Python 3 test, it fails with

  File "do.py", line 803, in test_preprocessing
    python('code-preprocessing/archive-update', ['-m', 'pytest'], verbose=_verbosity)

see here, while pytest has been installed in the setup. Same reason as above? Is the env argument in the python function of code-experiments/tools/cocoutils.py really needed? EDIT: probably when we make a sandbox test-installation.

EDIT: before the above error, we see

PYTHON  setup.py install in code-experiments/build/python

An exception occurred while trying to install packages.

A common reason for this error is insufficient access rights
to the installation directory. The original exception message
is as follows:

/----------------------------< EXCEPTION MESSAGE >----------------------------\
|                                                                             |
| Traceback (most recent call last):                                          |
|   File "setup.py", line 8, in <module>                                      |
|     import setuptools                                                       |
| ImportError: No module named setuptools                                     |
\-----------------------------------------------------------------------------/

which may also have the same reason.

We should also be aware that the default way to install python modules via do.py changed from user-installation to system installation by default, see here.

[nikohansen]

The Ubuntu Python 2 test (above point 2.) now succeeds, see here. For the time being the bibtex is disabled in cocopp/test.py.

I don't feel it is really necessary to run bibtex as a regular CI test-case. Manual testing of LaTeX templates is simple and the possible dependency complexity with the entire code base is very limited.

[nikohansen]

FTR, appveyor/coco and circleci/coco show the output of the new tests.

[nikohansen]

Latest error on Windows (calling make seems to work now after the cywin/bin path has been added in the setup):

MAKE    all in code-experiments/test/unit-test
CHDIR   C:\projects\coco\code-experiments\test\unit-test
COPY    Makefile.in -> Makefile
ERROR: return value=2
cc -c -g -ggdb -std=c89 -pedantic -Wall -Wextra -Wstrict-prototypes -Wshadow -Wno-sign-compare -Wconversion  -o unit_test.o unit_test.c 
In file included from unit_test.c:4:0:
cmocka.h:81:45: warning: ISO C90 does not support 'long long' [-Wlong-long]
 #  define LargestIntegralType unsigned long long int
                                             ^
cmocka.h:1863:42: note: in expansion of macro 'LargestIntegralType'
 typedef int (*CheckParameterValue)(const LargestIntegralType value,
                                          ^
cmocka.h:81:45: warning: ISO C90 does not support 'long long' [-Wlong-long]
 #  define LargestIntegralType unsigned long long int
                                             ^
cmocka.h:1864:42: note: in expansion of macro 'LargestIntegralType'
                                    const LargestIntegralType check_value_data);

https://ci.appveyor.com/project/nikohansen/coco/build/1.0.54/job/ptryfs7q0bt3yd94

[brockho]

Seems that now cmocka is missing. I tried to find out how to install this, but I did not find an obvious answer with Google in reasonable time. Unfortunately, the cmocka webpage also does not give much information about how to install the library.

But maybe, it is as simple as adding the path to code-experiments\test\unit-test\lib\win32_cygwin because the Windows .dll, appveyor is complaing about, is already contained in the Coco repository:

cc -g -ggdb -std=c89 -pedantic -Wall -Wextra -Wstrict-prototypes -Wshadow -Wno-sign-compare -Wconversion  -o unit_test unit_test.o -L. -lm -lcmocka  
./cmocka.dll: file not recognized: File format not recognized
collect2: error: ld returned 1 exit status
make: *** [Makefile:14: unit_test] Error 1

[ttusar]

All explicit calls to Python environments have been removed from do.py, but the error ImportError: No module named setuptools persists on CIrcleCI with Python 3.

[brockho]

I guess I found the problem with the Windows Appveyor test-c: The do.py checks for the sys.platform and platform.machine() to decide which .dll of cmocka to load. The Appveyor machine has the following setting:

sys.platform: win32
platform.machine(): AMD64

a combination which is not yet covered by our do.py. Unfortunately, we cannot connect to the machine at all in order to generate the missing .dll. Has anybody an idea how to get one compiled for an AMD64?

[dtusar]

I have the same values for sys.platform and platform.machine() so the dll should be fine. I think that cmocka must be installed on the machine but I don't know how to do it.

[brockho]

Decision of today: we take out the remaining not working two tests for the moment until we figure out how to deal with them. I will take care of this.

[islandblue]

The message we get from the appveyor machine suggests a corrupt or unrecognized library. @dtusar runs with the same library without problems, so that kind of rules corrupt out. I tried hard-coding the other two cmocka library files just in case the appveyor machine reports sys.platform and platform.machine() erroneously. That got rid of the ./cmocka.dll: file not recognized: File format not recognized message, but the build still failed, with a much fancier error code. So that may or may not be meaningful.

http://git.cryptomilk.org/projects/cmocka.git/tree/INSTALL may be relevant. http://github.com/clibs/clib may help.

I will see if I can get it built from source on the appveyor machine.

[brockho]

Can you point us to the output of the test? Is it this one here https://ci.appveyor.com/project/nikohansen/coco-j53aywshl8udzvb/build/1.0.389/job/uw7vn3i9t4ylxa6y you refer to?

[islandblue]

Yes and thank you. For some reason, I could not access the project on appveyor (says access denied). Your link seems to work though.

[brockho]

I now removed the test-c from appveyor.yml and added MikTeX to the mix in order to pass all remaining tests. Next up: the CircleCI python3 problem.

[islandblue]

After a long day at it, CircleCI now installs packages and runs the example experiment. However, post-processing fails with the following output:

Post-processing (1) /usr/local/lib/python3.6/site-packages/cocopp-2.0.737-py3.6.egg/cocopp/pproc.py:1327: UserWarning: logger is an additional attribute. warnings.warn('%s is an additional attribute.' % (attrname)) /usr/local/lib/python3.6/site-packages/cocopp-2.0.737-py3.6.egg/cocopp/dataformatsettings.py:71: RuntimeWarning: invalid value encountered in greater assert np.nansum(dataset.evals_function > -1) == np.nansum(dataset.evals_constraints > -1) Data consistent according to consistency_check() in pproc.DataSet Will generate output data in folder ./postproc/exdata this might take several minutes. Scaling figures... Loading best algorithm data from refalgs/best2009-bbob.tar.gz ... archive extracted to folder /usr/local/lib/python3.6/site-packages/cocopp-2.0.737-py3.6.egg/cocopp/refalgs/.extracted_best2009-bbob ... Data consistent according to consistency_check() in pproc.DataSet using: /usr/local/lib/python3.6/site-packages/cocopp-2.0.737-py3.6.egg/cocopp/refalgs/best2009-bbob.tar.gz done (Thu Oct 12 13:53:57 2017). done (Thu Oct 12 13:54:14 2017). Generating LaTeX tables... done (Thu Oct 12 13:54:14 2017). ECDF graphs... Traceback (most recent call last): File "/usr/local/lib/python3.6/runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "/usr/local/lib/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals) File "/usr/local/lib/python3.6/site-packages/cocopp-2.0.737-py3.6.egg/cocopp/main.py", line 30, in res = rungeneric.main(args) File "/usr/local/lib/python3.6/site-packages/cocopp-2.0.737-py3.6.egg/cocopp/rungeneric.py", line 311, in main dsld = rungeneric1.main(genopts + ["-o", outputdir, alg]) File "/usr/local/lib/python3.6/site-packages/cocopp-2.0.737-py3.6.egg/cocopp/rungeneric1.py", line 419, in main pprldistr.main(sliceNoise, True, algoutputdir, '%s' % noise) File "/usr/local/lib/python3.6/site-packages/cocopp-2.0.737-py3.6.egg/cocopp/pprldistr.py", line 839, in main rldStyles[j % len(rldStyles)]) File "/usr/local/lib/python3.6/site-packages/cocopp-2.0.737-py3.6.egg/cocopp/pprldistr.py", line 498, in plotRLDistr res = plotECDF(x, nn, kwargs) File "/usr/local/lib/python3.6/site-packages/cocopp-2.0.737-py3.6.egg/cocopp/pprldistr.py", line 381, in plotECDF drawstyle='steps', plotArgs) File "/usr/local/lib/python3.6/site-packages/cocopp-2.0.737-py3.6.egg/cocopp/ppfig.py", line 540, in plotUnifLogXMarkers res3 = plt.plot([], [], kwargs) File "/usr/local/lib/python3.6/site-packages/matplotlib/pyplot.py", line 3240, in plot ret = ax.plot(*args, *kwargs) File "/usr/local/lib/python3.6/site-packages/matplotlib/init.py", line 1710, in inner return func(ax, args, *kwargs) File "/usr/local/lib/python3.6/site-packages/matplotlib/axes/_axes.py", line 1438, in plot self.add_line(line) File "/usr/local/lib/python3.6/site-packages/matplotlib/axes/_base.py", line 1759, in add_line self._update_line_limits(line) File "/usr/local/lib/python3.6/site-packages/matplotlib/axes/_base.py", line 1781, in _update_line_limits path = line.get_path() File "/usr/local/lib/python3.6/site-packages/matplotlib/lines.py", line 951, in get_path self.recache() File "/usr/local/lib/python3.6/site-packages/matplotlib/lines.py", line 684, in recache xy = STEP_LOOKUP_MAP[self._drawstyle](self._xy.T) File "/usr/local/lib/python3.6/site-packages/matplotlib/cbook/init.py", line 2185, in pts_to_prestep steps = np.zeros((1 + len(args), 2 * len(x) - 1)) ValueError: negative dimensions are not allowed Exited with code 1

Good news is, I can now connect to the CircleCI container to debug. Bad news is, I don't know where to begin. Any insight?

[nikohansen]

The only thing which comes into my mind is to print kwargs before line 540 of ppfig.py and go from there.

File "/usr/local/lib/python3.6/site-packages/cocopp-2.0.737-py3.6.egg/cocopp/ppfig.py", line 540, in plotUnifLogXMarkers
res3 = plt.plot([], [], **kwargs)

[nikohansen]

This is BTW also a bug:

/usr/local/lib/python3.6/site-packages/cocopp-2.0.737-py3.6.egg/cocopp/dataformatsettings.py:71: RuntimeWarning: invalid value encountered in greater
assert np.nansum(dataset.evals_function > -1) == np.nansum(dataset.evals_constraints > -1)

Will have to fix it.

[islandblue]

@nikohansen that would be {'drawstyle': 'steps', 'color': 'k', 'ls': '-', 'label': '1: 6/24'}

[islandblue]

Tried to imitate it locally.

Python 3.6.1 |Anaconda 4.4.0 (64-bit)| (default, May 11 2017, 13:25:24) [MSC v.1900 64 bit (AMD64)] Type "copyright", "credits" or "license" for more information.

IPython 5.3.0 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object', use 'object??' for extra details.

In [1]: from matplotlib import pyplot as plt

In [2]: kwargs = dict( ...: drawstyle = 'steps', ...: color = 'k', ...: ls = '-', ...: label = '1: 6/24')

In [3]: print(kwargs) {'drawstyle': 'steps', 'color': 'k', 'ls': '-', 'label': '1: 6/24'}

In [4]: res3 = plt.plot([], [], **kwargs)

In [5]: res3 Out[5]: [<matplotlib.lines.Line2D at 0x1b60bc7c978>]

[islandblue]

Well, my local environment had matplotlib 2.0.2 and CircleCI had 2.1.0 so I tried upgrading my local matplotlib as well. As a result, I was able to reproduce the error locally. So it's a version compatibility problem now, probably totally unrelated to CircleCI.

[nikohansen]

Right, that is most likely it. So the trick to plot nothing to get the legend right is gone. Bad news.

[islandblue]

Can maybe plot an invisible dot instead? i.e. plt.plot([nan], [nan], **kwargs)

[islandblue]

Well, out of breath for today. Here's the last checkpoint. At this stage, each try takes ridiculously long (>10 min).

AML ['code-experiments/src/coco_random.c', 'code-experiments/src/coco_suite.c', 'code-experiments/src/coco_observer.c', 'code-experiments/src/coco_archive.c', 'code-experiments/src/coco_runtime_c.c'] -> code-experiments/build/c/coco.c EXPAND code-experiments/build/c/coco.c.in to code-experiments/build/c/coco.c EXPAND code-experiments/src/coco.h to code-experiments/build/c/coco.h COPY code-experiments/build/c/coco.c -> code-experiments/examples/bbob2009-c-cmaes/coco.c EXPAND code-experiments/build/c/coco.h to code-experiments/examples/bbob2009-c-cmaes/coco.h WRITE code-experiments/build/c/REVISION WRITE code-experiments/build/c/VERSION MAKE clean in code-experiments/build/c CHDIR /home/circleci/coco/code-experiments/build/c COPY Makefile.in -> Makefile MAKE all in code-experiments/build/c CHDIR /home/circleci/coco/code-experiments/build/c COPY Makefile.in -> Makefile COPY code-experiments/build/c/coco.c -> code-experiments/test/unit-test/coco.c EXPAND code-experiments/src/coco.h to code-experiments/test/unit-test/coco.h MAKE clean in code-experiments/test/unit-test CHDIR /home/circleci/coco/code-experiments/test/unit-test COPY Makefile.in -> Makefile MAKE all in code-experiments/test/unit-test CHDIR /home/circleci/coco/code-experiments/test/unit-test COPY Makefile.in -> Makefile ERROR: return value=2 cc -c -g -ggdb -std=c89 -pedantic -Wall -Wextra -Wstrict-prototypes -Wshadow -Wno-sign-compare -Wconversion -o unit_test.o unit_test.c In file included from unit_test.c:5:0: code-experiments/src/c_linear.c: In function ‘c_linear_shuffle’: code-experiments/src/c_linear.c:229:79: warning: conversion to ‘double’ from ‘size_t’ may alter its value [-Wconversion] In file included from unit_test.c:5:0: code-experiments/src/suite_cons_bbob_problems.c: In function ‘obj_function_type’: code-experiments/src/suite_cons_bbob_problems.c:102:39: warning: conversion to ‘double’ from ‘size_t’ may alter its value [-Wconversion] cc -g -ggdb -std=c89 -pedantic -Wall -Wextra -Wstrict-prototypes -Wshadow -Wno-sign-compare -Wconversion -o unit_test unit_test.o -L. -lm -lcmocka
unit_test.o: In function test_all_coco_string': /home/circleci/coco/code-experiments/test/unit-test/test_coco_string.c:52: undefined reference to_cmocka_run_group_tests' unit_test.o: In function test_all_coco_archive': /home/circleci/coco/code-experiments/test/unit-test/test_coco_archive.c:279: undefined reference to_cmocka_run_group_tests' unit_test.o: In function test_all_coco_observer': /home/circleci/coco/code-experiments/test/unit-test/test_coco_observer.c:134: undefined reference to_cmocka_run_group_tests' unit_test.o: In function test_all_coco_problem': /home/circleci/coco/code-experiments/test/unit-test/test_coco_problem.c:145: undefined reference to_cmocka_run_group_tests' unit_test.o: In function test_all_coco_suite': /home/circleci/coco/code-experiments/test/unit-test/test_coco_suite.c:95: undefined reference to_cmocka_run_group_tests' unit_test.o:/home/circleci/coco/code-experiments/test/unit-test/test_coco_utilities.c:457: more undefined references to `_cmocka_run_group_tests' follow collect2: error: ld returned 1 exit status Makefile:14: recipe for target 'unit_test' failed make: *** [unit_test] Error 1

Traceback (most recent call last): File "do.py", line 999, in main(sys.argv[1:]) File "do.py", line 980, in main elif cmd == 'test-c': test_c() File "do.py", line 87, in test_c build_c_unit_tests() File "do.py", line 151, in build_c_unit_tests make("code-experiments/test/unit-test", "all", verbose=_verbosity) File "/home/circleci/coco/code-experiments/tools/cocoutils.py", line 213, in make universal_newlines=True) File "/home/circleci/coco/code-experiments/tools/cocoutils.py", line 42, in check_output_with_print output = check_output(*popenargs, kwargs) File "/usr/local/lib/python3.6/subprocess.py", line 336, in check_output kwargs).stdout File "/usr/local/lib/python3.6/subprocess.py", line 418, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['make', 'all']' returned non-zero exit status 2. Exited with code 1

[islandblue]

So it appears this is again a problem with the cmocka library. Should we take the same approach as appveyor?

[brockho]

I am fine with leaving this test (test-c) out also on the CircleCI. We have it running on three different systems right now on Jenkins.

And again: in the long run, we might want to abandon the library anyway...

[islandblue]

Well, seems it's because the image I'm working with has debian linux and we don't handle this distribution. I'll switch back to the ubuntu image.

[islandblue]

For some reason, RS-4.tgz is never downloaded properly.

Build output:

Post-processing (1)
  downloading http://coco.gforge.inria.fr/data-archive/bbob/2009/BFGS_ros_noiseless.tgz to /root/.cocopp/data-archive/bbob/2009/BFGS_ros_noiseless.tgz
  Using:
    /root/.cocopp/data-archive/bbob/2009/BFGS_ros_noiseless.tgz

Post-processing (1)
    archive extracted to folder /root/.cocopp/data-archive/bbob/2009/.extracted_BFGS_ros_noiseless ...
  Data consistent according to consistency_check() in pproc.DataSet
  Will generate output data in folder ppdata/BFGS_ros_noiseless
    this might take several minutes.
Scaling figures...
Loading best algorithm data from refalgs/best2009-bbob.tar.gz ...
/usr/local/lib/python3.5/dist-packages/cocopp-2.0.763-py3.5.egg/cocopp/rungeneric1.py:234: UserWarning: grayscalesettings disregarded
  warnings.warn("grayscalesettings disregarded")
    archive extracted to folder /usr/local/lib/python3.5/dist-packages/cocopp-2.0.763-py3.5.egg/cocopp/refalgs/.extracted_best2009-bbob ...
  Data consistent according to consistency_check() in pproc.DataSet
  using: /usr/local/lib/python3.5/dist-packages/cocopp-2.0.763-py3.5.egg/cocopp/refalgs/best2009-bbob.tar.gz
  done (Mon Oct 16 20:27:52 2017).
  done (Mon Oct 16 20:28:21 2017).
Generating LaTeX tables...
  done (Mon Oct 16 20:28:22 2017).
ECDF graphs...
  done (Mon Oct 16 20:28:40 2017).
ECDF graphs per function...
  done (Mon Oct 16 20:29:16 2017).
aRT loss ratio figures and tables...
  done (Mon Oct 16 20:29:25 2017).
Output data written to folder /root/coco/code-postprocessing/cocopp/ppdata/BFGS_ros_noiseless
Setting changes in `cocopp.genericsettings` compared to default:
    inputsettings: from "color" to "grayscale"
    simulated_runlength_bootstrap_sample_size: from 10 to 10.098990100989901
ALL done (Mon Oct 16 20:29:25 2017).
Traceback (most recent call last):
  File "/usr/lib/python3.5/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.5/dist-packages/cocopp-2.0.763-py3.5.egg/cocopp/__main__.py", line 30, in <module>
    res = rungeneric.main(args)
  File "/usr/local/lib/python3.5/dist-packages/cocopp-2.0.763-py3.5.egg/cocopp/rungeneric.py", line 279, in main
    clean_extended_args.append(data_archive.get(name))  # download if necessary
  File "/usr/local/lib/python3.5/dist-packages/cocopp-2.0.763-py3.5.egg/cocopp/findfiles.py", line 645, in get
    self.check_hash(full_name)
  File "/usr/local/lib/python3.5/dist-packages/cocopp-2.0.763-py3.5.egg/cocopp/findfiles.py", line 710, in check_hash
    '' % (name, self.full_path(name)))
ValueError: wrong checksum for "/root/.cocopp/data-archive/bbob-biobj/2016/RS-4.tgz".Consider to (re)move file
   /root/.cocopp/data-archive/bbob-biobj/2016/RS-4.tgz
as it may be a partial/unsuccessful download.
A missing file will be downloaded again by `get`.
Post-processing (1)
  downloading http://coco.gforge.inria.fr/data-archive/bbob-biobj/2016/RS-4.tgz to /root/.cocopp/data-archive/bbob-biobj/2016/RS-4.tgz
LaTeX templates copied.
*** testing module cocopp ***
python3 -m  cocopp --no-svg --settings=grayscale BFGS_ros_noiseless
**  subtest 1 finished in  250.68950295448303  seconds
python3 -m  cocopp --no-svg --settings=grayscale RS-4.tgz
**  subtest 2 finished in  0.9687111377716064  seconds
Traceback (most recent call last):
  File "test.py", line 389, in <module>
    main(sys.argv[1:])
  File "test.py", line 231, in main
    assert result == 0, 'Test failed: rungeneric on one bi-objective algorithm.'
AssertionError: Test failed: rungeneric on one bi-objective algorithm.

Exited with code 255

Debug session on build image:

root@dc70eee83735:~/.cocopp/data-archive# ls bbob/2009/
BFGS_ros_noiseless.tgz
root@dc70eee83735:~/.cocopp/data-archive# ls -s bbob/2009/
total 14676
14676 BFGS_ros_noiseless.tgz
root@dc70eee83735:~/.cocopp/data-archive# ls -s bbob-biobj/2016/
total 24
24 RS-4.tgz
root@dc70eee83735:~/.cocopp/data-archive# tar -xf bbob/2009/BFGS_ros_noiseless.tgz
root@dc70eee83735:~/.cocopp/data-archive# ls BFGS/
bfgs1e5_five   ppfigdim_f001.pdf  ppfigdim_f006.pdf  ppfigdim_f011.pdf  ppfigdim_f016.pdf  ppfigdim_f021.pdf      ppfvdistr_05D_hcond.pdf         ppfvdistr_05D_separ.pdf         pprldistr_05D_lcond.pdf         pprldistr_20D_all.pdf           templateBBOBarticle.html
bfgs1e5_four   ppfigdim_f002.pdf  ppfigdim_f007.pdf  ppfigdim_f012.pdf  ppfigdim_f017.pdf  ppfigdim_f022.pdf      ppfvdistr_05D_lcond.pdf         ppfvdistr_20D_all.pdf           pprldistr_05D_mult2.pdf         pprldistr_20D_noiselessall.pdf
bfgs1e5_one    ppfigdim_f003.pdf  ppfigdim_f008.pdf  ppfigdim_f013.pdf  ppfigdim_f018.pdf  ppfigdim_f023.pdf      ppfvdistr_05D_mult2.pdf         ppfvdistr_20D_noiselessall.pdf  pprldistr_05D_multi.pdf         pptable_05D_noiselessall.tex
bfgs1e5_three  ppfigdim_f004.pdf  ppfigdim_f009.pdf  ppfigdim_f014.pdf  ppfigdim_f019.pdf  ppfigdim_f024.pdf      ppfvdistr_05D_multi.pdf         pprldistr_05D_all.pdf           pprldistr_05D_noiselessall.pdf  pptable_20D_noiselessall.tex
bfgs1e5_two    ppfigdim_f005.pdf  ppfigdim_f010.pdf  ppfigdim_f015.pdf  ppfigdim_f020.pdf  ppfvdistr_05D_all.pdf  ppfvdistr_05D_noiselessall.pdf  pprldistr_05D_hcond.pdf         pprldistr_05D_separ.pdf         sorttable.js
root@dc70eee83735:~/.cocopp/data-archive# tar -xf bbob-biobj/2016/RS-4.tgz

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now

[nikohansen]

The reason is that there is no such file in this part of the archive http://coco.gforge.inria.fr/data-archive/bbob-biobj/2016/

[brockho]

Sorry that was my mistake. I deleted some data files this morning after adding renamed versions of them according to issues #1580 and #69, but I forgot to update the tests accordingly :-( A fix is now running on devel-test1 and I hope to merge it later tonight to development.

[brockho]

The development branch is green again.

[nikohansen]

Well, my local environment had matplotlib 2.0.2 and CircleCI had 2.1.0 so I tried upgrading my local matplotlib as well. As a result, I was able to reproduce the error locally. So it's a version compatibility problem now, probably totally unrelated to CircleCI.

this should have been solved immediately with critical priority, as it is a bug which makes the current code unusable with an up-to-date installation of matplotlib. Unsurprisingly it now re-surfaced during the weekend in #1636.

[islandblue]

CircleCI tests are green on development. Now appveyor is broken. See #1651

[brockho]

The last four builds were successful today. Hence, I would guess, the issue can be closed?