Optimization readme sample/test needs improving

[woodsp-ibm]

The readme sample unit test, that ensures the sample continues to work with whatever changes are going on, seems to fail on occasion in the build. I have not looked to see whether the test is failing with perhaps a valid alternate answer that should be catered to in the assertion its correct, or whether this sample needs seeding to ensure the output is consistent each time. Here is a failure from earlier today as an example

{1} test.optimization.test_readme_sample.TestReadmeSample.test_readme_sample [30.941167s] ... FAILED

Captured traceback:

~~~~~~~~~~~~~~~~~~~

    Traceback (most recent call last):
      File "/home/travis/build/Qiskit/qiskit-aqua/test/optimization/test_readme_sample.py", line 78, in test_readme_sample
    np.testing.assert_array_almost_equal(result.x, [1, 0, 1, 0])
      File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/numpy/testing/_private/utils.py", line 1044, in assert_array_almost_equal
    precision=decimal)
      File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/numpy/testing/_private/utils.py", line 840, in assert_array_compare
    raise AssertionError(msg)
    AssertionError: 

Arrays are not almost equal to 6 decimals

Mismatched elements: 2 / 4 (50%)
Max absolute difference: 1.
Max relative difference: 1.
 x: array([1., 1., 0., 0.])
 y: array([1, 0, 1, 0])

I have also seen it fail where the issue is around NaN in the samples, which seems of greater concern since one would hope no matter whether it gave a correct answer or not NaN seems more like some logic bug.

[woodsp-ibm]

Here is one that has just happened with NaN, as I mentioned I had seen in original message here. Though this error seems to originate from BasicAer as the first source of failure

 Captured traceback:
 ~~~~~~~~~~~~~~~~~~~
     Traceback (most recent call last):

     concurrent.futures.process._RemoteTraceback: 
 """
 Traceback (most recent call last):
   File "/opt/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/concurrent/futures/process.py", line 239, in _process_worker
     r = call_item.fn(*call_item.args, **call_item.kwargs)
   File "/opt/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/site-packages/qiskit/providers/basicaer/qasm_simulator.py", line 413, in _run_job
     result_list.append(self.run_experiment(experiment))
   File "/opt/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/site-packages/qiskit/providers/basicaer/qasm_simulator.py", line 587, in run_experiment
     memory = self._add_sample_measure(measure_sample_ops, self._shots)
   File "/opt/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/site-packages/qiskit/providers/basicaer/qasm_simulator.py", line 203, in _add_sample_measure
     num_samples, p=probabilities)
   File "mtrand.pyx", line 928, in numpy.random.mtrand.RandomState.choice
 ValueError: probabilities contain NaN
 """

 The above exception was the direct cause of the following exception:

      Traceback (most recent call last):

       File "/home/runner/work/qiskit-aqua/qiskit-aqua/test/optimization/test_readme_sample.py", line 74, in test_readme_sample
     result = algorithm.solve(problem)
        File "/home/runner/work/qiskit-aqua/qiskit-aqua/qiskit/optimization/algorithms/minimum_eigen_optimizer.py", line 163, in solve
     eigen_result = self._min_eigen_solver.compute_minimum_eigenvalue(operator)
        File "/home/runner/work/qiskit-aqua/qiskit-aqua/qiskit/aqua/algorithms/minimum_eigen_solvers/vqe.py", line 457, in compute_minimum_eigenvalue
     return self._run()
        File "/home/runner/work/qiskit-aqua/qiskit-aqua/qiskit/aqua/algorithms/minimum_eigen_solvers/vqe.py", line 399, in _run
     optimizer=self.optimizer)
        File "/home/runner/work/qiskit-aqua/qiskit-aqua/qiskit/aqua/algorithms/vq_algorithm.py", line 206, in find_minimum
     gradient_function=gradient_fn)
        File "/home/runner/work/qiskit-aqua/qiskit-aqua/qiskit/aqua/components/optimizers/spsa.py", line 134, in optimize
     **self._options)
        File "/home/runner/work/qiskit-aqua/qiskit-aqua/qiskit/aqua/components/optimizers/spsa.py", line 189, in _optimization
     cost_plus = obj_fun(theta_plus)
        File "/home/runner/work/qiskit-aqua/qiskit-aqua/qiskit/aqua/algorithms/minimum_eigen_solvers/vqe.py", line 488, in _energy_evaluation
     sampled_expect_op = self._circuit_sampler.convert(self._expect_op, params=param_bindings)
        File "/home/runner/work/qiskit-aqua/qiskit-aqua/qiskit/aqua/operators/converters/circuit_sampler.py", line 198, in convert
     param_bindings=p_b)
        File "/home/runner/work/qiskit-aqua/qiskit-aqua/qiskit/aqua/operators/converters/circuit_sampler.py", line 276, in sample_circuits
     had_transpiled=self._transpile_before_bind)
        File "/home/runner/work/qiskit-aqua/qiskit-aqua/qiskit/aqua/quantum_instance.py", line 374, in execute
     self._skip_qobj_validation, self._job_callback)
        File "/home/runner/work/qiskit-aqua/qiskit-aqua/qiskit/aqua/utils/run_circuits.py", line 312, in run_qobj
     results.append(job.result(**qjob_config))
        File "/opt/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/site-packages/qiskit/providers/basicaer/basicaerjob.py", line 38, in _wrapper
     return func(self, *args, **kwargs)
        File "/opt/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/site-packages/qiskit/providers/basicaer/basicaerjob.py", line 93, in result
     return self._future.result(timeout=timeout)
        File "/opt/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/concurrent/futures/_base.py", line 435, in result
     return self.__get_result()
        File "/opt/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
     raise self._exception

     ValueError: probabilities contain NaN

[t-imamichi]

How about replacing BasicAer with Aer?

[woodsp-ibm]

The above issue has not been seen in a while. An issue was fixed during Cvar implementation #1345 in SPSA that had potential to be the source of this failure. Since the issue has not been seen in a while the issue, that was noticed/fixed for a different problem, may indeed have been the source. I am therefore closing this issue since it no longer seems to be valid.