search

aria-jpl / scihub_acquisition_scraper

Apache License 2.0
0 stars 5 forks source link

Unable to build latest release-20190710 docker #7

Open shitong01 opened 4 years ago

shitong01 commented 4 years ago

From release-20190710 onwards, unable to install pandas in pge-base/python2 docker image. From Jenkins:

Started by an SCM change
Building in workspace /home/ops/jenkins/workspace/ops_container-builder_aria-jpl_scihub_acquisition_scraper
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/aria-jpl/scihub_acquisition_scraper # timeout=10
Fetching upstream changes from https://github.com/aria-jpl/scihub_acquisition_scraper
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/aria-jpl/scihub_acquisition_scraper +refs/tags/*:refs/remotes/origin/tags/*
Seen branch in repository origin/#1245
Seen branch in repository origin/ARIA-7
Seen branch in repository origin/dev
Seen branch in repository origin/dev-malarout
Seen branch in repository origin/develop
Seen branch in repository origin/handle_duplicates
Seen branch in repository origin/master
Seen branch in repository origin/tags/release-20190313
Seen branch in repository origin/tags/release-20190314
Seen branch in repository origin/tags/release-20190401
Seen branch in repository origin/tags/release-20190403
Seen branch in repository origin/tags/release-20190404
Seen branch in repository origin/tags/release-20190405
Seen branch in repository origin/tags/release-20190410
Seen branch in repository origin/tags/release-20190429
Seen branch in repository origin/tags/release-20190503
Seen branch in repository origin/tags/release-20190504
Seen branch in repository origin/tags/release-20190520
Seen branch in repository origin/tags/release-20190618
Seen branch in repository origin/tags/release-20190710
Seen 20 remote branches
 > git show-ref --tags -d # timeout=10
Checking out Revision 13d0ae207d4e25c7900cfb7e59cc185cc7322f39 (origin/tags/release-20190710)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 13d0ae207d4e25c7900cfb7e59cc185cc7322f39
Commit message: "malarout: Corrected location field in report"
First time build. Skipping changelog.
[ops_container-builder_aria-jpl_scihub_acquisition_scraper] $ /bin/bash /tmp/jenkins2453716622219723132.sh
[CI] Is checkout a tag: 1 release-20190710
[CI] Last log: commit 10917c8b89e3da4cd2156d13171e3fa86197c1f4
Author: Malarout <namrata.malarout@jpl.nasa.gov>
Date:   Wed Jul 10 11:46:32 2019 -0700

    malarout: Corrected location field in report
[CI] Skip image build flag: 0
[WARNING] ipf-scraper-asf defines hysds-io parameter acq_id without a type
[WARNING] ipf-scraper-asf defines hysds-io parameter acq_met without a type
[WARNING] ipf-scraper-asf defines hysds-io parameter index without a type
[WARNING] ipf-scraper-asf defines hysds-io parameter dataset_type without a type
[WARNING] ipf-scraper-asf defines hysds-io parameter endpoint without a type
[WARNING] ipf-scraper-asf defines hysds-io parameter ds_cfg without a type
[WARNING] ipf-scraper-scihub defines hysds-io parameter acq_id without a type
[WARNING] ipf-scraper-scihub defines hysds-io parameter acq_met without a type
[WARNING] ipf-scraper-scihub defines hysds-io parameter index without a type
[WARNING] ipf-scraper-scihub defines hysds-io parameter dataset_type without a type
[WARNING] ipf-scraper-scihub defines hysds-io parameter endpoint without a type
[WARNING] ipf-scraper-scihub defines hysds-io parameter ds_cfg without a type
[WARNING] aoi_validate_acquisitions defines job-spec with deprecated 'required-queues'
[WARNING] aoi_validate_acquisitions defines hysds-io parameter ds_cfg without a type
[WARNING] aoi_validate_acquisitions defines hysds-io parameter starttime without a type
[WARNING] aoi_validate_acquisitions defines hysds-io parameter endtime without a type
[WARNING] aoi_validate_acquisitions defines hysds-io parameter polygon_flag without a type
[WARNING] aoi_validate_acquisitions defines hysds-io parameter polygon without a type
[WARNING] aoi_validate_acquisitions defines hysds-io parameter ingest_flag without a type
[WARNING] aoi_validate_acquisitions defines hysds-io parameter purpose_flag without a type
[WARNING] aoi_validate_acquisitions defines hysds-io parameter purpose without a type
[WARNING] acquisition_ingest-aoi defines job-spec with deprecated 'required-queues'
[WARNING] acquisition_ingest-aoi defines hysds-io parameter aoi_name without a type
[WARNING] acquisition_ingest-aoi defines hysds-io parameter ds_cfg without a type
[WARNING] acquisition_ingest-aoi defines hysds-io parameter starttime without a type
[WARNING] acquisition_ingest-aoi defines hysds-io parameter endtime without a type
[WARNING] acquisition_ingest-aoi defines hysds-io parameter polygon_flag without a type
[WARNING] acquisition_ingest-aoi defines hysds-io parameter polygon without a type
[WARNING] acquisition_ingest-aoi defines hysds-io parameter ds_flag without a type
[WARNING] acquisition_ingest-aoi defines hysds-io parameter ds_version without a type
[WARNING] acquisition_ingest-aoi defines hysds-io parameter ingest_flag without a type
[WARNING] acquisition_ingest-aoi defines hysds-io parameter purpose_flag without a type
[WARNING] acquisition_ingest-aoi defines hysds-io parameter purpose without a type
[WARNING] acquisition_ingest-aoi defines hysds-io parameter report_flag without a type
[WARNING] acquisition_ingest-asf defines hysds-io parameter starttime without a type
[WARNING] acquisition_ingest-asf defines hysds-io parameter endtime without a type
[WARNING] acquisition_ingest_by_id-scihub defines job-spec with deprecated 'required-queues'
[WARNING] acquisition_ingest_by_id-scihub defines hysds-io parameter es_dataset_url without a type
[WARNING] acquisition_ingest_by_id-scihub defines hysds-io parameter ds_cfg without a type
[WARNING] acquisition_ingest_by_id-scihub defines hysds-io parameter slc_id without a type
[WARNING] acquisition_ingest_by_id-scihub defines hysds-io parameter ingest_flag without a type
[WARNING] acquisition_ingest-scihub defines job-spec with deprecated 'required-queues'
[WARNING] acquisition_ingest-scihub defines hysds-io parameter ds_cfg without a type
[WARNING] acquisition_ingest-scihub defines hysds-io parameter starttime without a type
[WARNING] acquisition_ingest-scihub defines hysds-io parameter endtime without a type
[WARNING] acquisition_ingest-scihub defines hysds-io parameter ingest_flag without a type
[WARNING] aoi_based_ipf_submitter defines job-spec with deprecated 'required-queues'
[WARNING] aoi_based_ipf_submitter defines hysds-io parameter AOI_name without a type
[WARNING] aoi_based_ipf_submitter defines hysds-io parameter spatial_extent without a type
[WARNING] aoi_based_ipf_submitter defines hysds-io parameter start_time without a type
[WARNING] aoi_based_ipf_submitter defines hysds-io parameter end_time without a type
[WARNING] aoi_based_acq_submitter defines job-spec with deprecated 'required-queues'
[WARNING] aoi_based_acq_submitter defines hysds-io parameter AOI_name without a type
[WARNING] aoi_based_acq_submitter defines hysds-io parameter spatial_extent without a type
[WARNING] aoi_based_acq_submitter defines hysds-io parameter start_time without a type
[WARNING] aoi_based_acq_submitter defines hysds-io parameter end_time without a type
[WARNING] aoi_based_acq_submitter defines hysds-io parameter dataset_version without a type
[CI] Build for: container-aria-jpl_scihub_acquisition_scraper:release-20190710 and file aria-jpl_scihub_acquisition_scraper
 docker build --rm --force-rm -f docker/Dockerfile -t container-aria-jpl_scihub_acquisition_scraper:release-20190710  .
Sending build context to Docker daemon  1.001MB

Step 1/11 : FROM hysds/pge-base:python2
 ---> c44b44cbe06e
Step 2/11 : MAINTAINER aria-dev "aria-dev@jpl.nasa.gov"
 ---> Using cache
 ---> 7e1ce3e624ed
Step 3/11 : LABEL description="SciHub/ApiHub scraper PGE"
 ---> Using cache
 ---> 66ec980088f2
Step 4/11 : USER ops
 ---> Using cache
 ---> 473ec1203cfc
Step 5/11 : RUN sudo pip install shapely
 ---> Using cache
 ---> aa5a40c3653d
Step 6/11 : RUN sudo pip install elasticsearch
 ---> Using cache
 ---> 5edaad163f77
Step 7/11 : RUN sudo pip install pandas
 ---> Running in cfcd8a6a8ac0
Collecting pandas
  Downloading https://files.pythonhosted.org/packages/07/cf/1b6917426a9a16fd79d56385d0d907f344188558337d6b81196792f857e9/pandas-0.25.1.tar.gz (12.6MB)
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-jN93u0/pandas/setup.py", line 21, in <module>
        import versioneer
      File "versioneer.py", line 1629
        print("Adding sample versioneer config to setup.cfg", file=sys.stderr)
                                                                  ^
    SyntaxError: invalid syntax

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-jN93u0/pandas/
You are using pip version 8.1.2, however version 19.2.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Removing intermediate container cfcd8a6a8ac0
The command '/bin/sh -c sudo pip install pandas' returned a non-zero code: 1
[ERROR] Failed to build docker container for: container-aria-jpl_scihub_acquisition_scraper:release-20190710
Build step 'Execute shell' marked build as failure
Finished: FAILURE
pymonger commented 4 years ago

@shitong01: that seems to be a python 2 vs. 3 issue. The latest major version of Pandas 0.25.x only supports python 3.5.3 and above: https://pandas.pydata.org/pandas-docs/version/0.25/whatsnew/v0.25.0.html. We'll probably have to pin pandas to the 0.24.2 version since that's the last release to support Python 2: https://pandas.pydata.org/pandas-docs/version/0.24/whatsnew/v0.24.2.html.

shitong01 commented 4 years ago

I see, so we gotta try sudo pip install pandas==0.24.2

pymonger commented 4 years ago

Yes, as a stopgap. However we should be moving over the python 3 and porting all of the ARIA PGEs over. I'll discuss with our team today.