google / fhir-data-pipes

A collection of tools for extracting FHIR resources and analytics services on top of that data.
https://google.github.io/fhir-data-pipes/
Apache License 2.0
151 stars 84 forks source link

Attempting to fix the e2e failure #892

Closed bashir2 closed 10 months ago

bashir2 commented 10 months ago

Description of what I changed

The e2e is failing since Nov. 24th in incremental run phase of step 12, i.e., testing the controller and thrift-server together. We don't get the log of the controller in Cloud Build logs (since we run it in detached mode). But from a local run it seems it fails in the merge step because of memory issues. Not sure why this might have been triggered now since PR #859 was merged more than a month ago. But anyways, reducing number of threads to see if that helps. (Note this is built on top of #887 so only the second commit belongs to this PR).

E2E test

TESTED:

Ran the incremental pipeline locally through the controller.

Checklist: I completed these to help reviewers :)

bashir2 commented 10 months ago

This is now obsolete because after merging #874 the e2e has been fixed. This is probably because that PR makes changes to the flink memory configuration parameters automatically, fixing the incremental build failures.