Extremely large responses

[jawalonoski]

Crucible currently does not save extremely large server responses to the database due to Mongo limitations.

However, it does not consider the size of entire test runs, where each individual test is below the size limit, but taken together they exceed the size limitations.

This line causes the error: https://github.com/fhir-crucible/crucible/blob/de2854124dce802692c2bb402371761b3d953269/app/models/test_run.rb#L146

I have been able to recreate this bug repeatedly. Let me know if you need a demonstration.

[arscan]

Well that's a frustrating oversight. We probably should add some unit tests for this.

[jawalonoski]

One possible solution is to reference/link to the result (all of which are under the size limit) rather than embed them within the test run.

But that would likely have downstream impacts... most notably processing the server score.

[arscan]

Results aren't embedded within test runs. Take a look at the test_runs and test_results collections in your local db.

I believe that line (self.test_results << result) saves individual results to the test_result collection, and has each test_result reference back to its test_run with the test_run_id field. Maybe the fact that we are saving it all at once in one batch is causing this problem? We could try saving the test result individually, and manually set the test_run_id to point back to the test_run (instead of using mongoid's build in functionality for handling these relationships using array appends).

As an aside, if I was to do it again, I would also put requests/responses in it's own collection. That contains 99% of the data in our database, and would be nice split out into it's own collection so we could exclude it from database dumps, etc. It would also significantly speed up looking at test results by only downloading the request/response data when you click on 'view requests'.

[jawalonoski]

Huh. Worth refactoring?