pip download failed in post commit

[kennknowles]

A post commit failed with a pip failure: https://builds.apache.org/job/beam_PostCommit_Python_Verify/1801/consoleFull

Captured output is not clear enough to tell what went wrong:

\==\==\==\==\==\==\==\==\==\==\==\==\==\==\==\==\==\==\==\==\==\==\==\==\==\==\==\==\==\==\==\==\==\==\== ERROR: test_par_do_with_multiple_outputs_and_using_return (apache_beam.transforms.ptransform_test.PTransformTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/transforms/ptransform_test.py", line 207, in test_par_do_with_multiple_outputs_and_using_return pipeline.run() File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/test_pipeline.py", line 91, in run result = super(TestPipeline, self).run() File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/pipeline.py", line 160, in run self.to_runner_api(), self.runner, self.options).run(False) File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/pipeline.py", line 169, in run return self.runner.run(self) File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py", line 39, in run self.result = super(TestDataflowRunner, self).run(pipeline) File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py", line 175, in run self.dataflow_client.create_job(self.job), self) File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/utils/retry.py", line 174, in wrapper return fun(*args, *kwargs) File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py", line 425, in create_job self.create_job_description(job) File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py", line 446, in create_job_description job.options, file_copy=self._gcs_file_copy) File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/runners/dataflow/internal/dependency.py", line 306, in stage_job_resources setup_options.requirements_file, requirements_cache_path) File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/runners/dataflow/internal/dependency.py", line 242, in _populate_requirements_cache processes.check_call(cmd_args) File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/utils/processes.py", line 40, in check_call return subprocess.check_call(args, **kwargs) File "/usr/lib/python2.7/subprocess.py", line 540, in check_call raise CalledProcessError(retcode, cmd) CalledProcessError: Command '['/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/bin/python', '-m', 'pip', 'install', '--download', '/tmp/dataflow-requirements-cache', '-r', 'postcommit_requirements.txt', '--no-binary', ':all:']' returned non-zero exit status 2

Any ideas on how we can debug this failure or improve for the future? (cc: [~markflyhigh] [~sb2nov])

Imported from Jira BEAM-1924. Original Jira may contain additional context. Reported by: altay.

[Abacn]

obsolete