SearchTrends data shows an error in loading intermediate output

themonk911 commented 4 years ago

{
  "jsonPayload": {
    "exception": "FileNotFoundError: [Errno 2] File /tmp/tmpugulfbpe/intermediate/31944cb0-9b4f-5939-b83b-5d3d90ecf299.csv does not exist: '/tmp/tmpugulfbpe/intermediate/31944cb0-9b4f-5939-b83b-5d3d90ecf299.csv'",
    "source_config": {
      "name": "pipelines._common.google_search_trends.GoogleSearchTrendsL2DataSource",
      "parse": {
        "low_memory": false
      },
      "fetch": [
        {
          "url": "https://raw.github.com/google-research/open-covid-19-data/master/data/inputs/downloaded/search_trends_symptoms_dataset/2020-08-30/United%20States%20of%20America/subregions/{subregion1_name_path}/2020_US_{subregion1_name_file}_daily_symptoms_dataset.csv"
        }
      ],
      "test": {
        "skip": true
      },
      "automation": {
        "job_group": "0"
      }
    },
    "classname": "DataPipeline",
    "source_name": "GoogleSearchTrendsL2DataSource",
    "message": "Failed to load intermediate output",
    "timestamp": "2020-09-03T13:20:14.8559"
  },
  "resource": {
    "type": "gae_app",
    "labels": {
      "zone": "",
      "project_id": "github-open-covid-19",
      "module_id": "default",
      "version_id": "20200903t025702"
    }
  },
  "timestamp": "2020-09-03T13:20:14Z",
  "logName": "projects/github-open-covid-19/logs/appengine.googleapis.com%2Fstderr",
  "receiveTimestamp": "2020-09-03T13:20:16.273289578Z"
}

owahltinez commented 4 years ago

Thanks for reporting, it looks like the job is timing out. I submitted this commit to fix it: https://github.com/GoogleCloudPlatform/covid-19-open-data/commit/90b8d0ea0d39cabbb578326ac0c3c123b15d57dc

owahltinez commented 4 years ago

It looks like this has now been fixed

GoogleCloudPlatform / covid-19-open-data

SearchTrends data shows an error in loading intermediate output #148