runTest promise initialization. 300ms sleep time added before each runTest promise is initialized. Why? At this point I suspected the problem was the container couldn't handle 5 incognito instances initialized at the same time. This totally solved the problem.
runTest(): if the test times-out (more than 50s to complete), it closes itself so that runMultipleTests() can restart it . Why? After creating the logger functions that write to S3 I found that some of the promises could get randomly stuck at iteration 0. This ensures that if one promise get stuck, it won't be for more than a minute.
Complete list of changes
coordinator.js
max_memory_restart limit increased from 250 to 500mb. Why? I thought that maybe coordinator was restarting the runner.js at a high rate. It wasn't the case.
normalizer.js
await writeToS3(): normalizer now waits for writeToS3 to complete before continuing. Why? To avoid a situation where the data in memory is wiped out from memory before it's sent to S3.
writeToS3 now uses Promise.allSettled to upload files to S3 concurrently. Why? Performance.
runner.js
runMultipleTests(): if during the test the number of promises < numberOfUser -> runner.js closes the browser and finish execution (pm2 will restart it immediately). Why? I thought that maybe one of the runTest promises was never initialized or it was dropped prematurely.
runTest(): if the test of one of the users throws an exception for any reason, the incognito context is closed and the promise is fulfilled. runMultipleTests will take replace the fulfilled promise with a new one. Why? I thought that maybe one of the tests where erroing and wasn't able to continue after.
functions updatePromiseStatusLog and writePromiseLogToS3IfCounter added. They can be used to send logs to S3.
runTest(): if the test times-out (more than 50s to complete), it closes itself so that runMultipleTests() can restart it . Why? After creating the logger functions that write to S3 I found that some of the promises could get randomly stuck at iteration 0.
runTest promise initialization. 300ms sleep time added before each runTest promise is initialized. Why? At this point I suspected the problem was the container couldn't handle 5 incognito instances initialized at the same time.
Changes that solved the problem:
runTestpromise initialization. 300ms sleep time added before eachrunTestpromise is initialized. Why? At this point I suspected the problem was the container couldn't handle 5 incognito instances initialized at the same time. This totally solved the problem.runTest(): if the test times-out (more than 50s to complete), it closes itself so thatrunMultipleTests()can restart it . Why? After creating the logger functions that write to S3 I found that some of the promises could get randomly stuck at iteration 0. This ensures that if one promise get stuck, it won't be for more than a minute.Complete list of changes coordinator.js
normalizer.js
await writeToS3(): normalizer now waits forwriteToS3to complete before continuing. Why? To avoid a situation where the data in memory is wiped out from memory before it's sent to S3.writeToS3now uses Promise.allSettled to upload files to S3 concurrently. Why? Performance.runner.js
runMultipleTests(): if during the test the number of promises < numberOfUser ->runner.jscloses the browser and finish execution (pm2 will restart it immediately). Why? I thought that maybe one of therunTestpromises was never initialized or it was dropped prematurely.runTest(): if the test of one of the users throws an exception for any reason, the incognito context is closed and the promise is fulfilled. runMultipleTests will take replace the fulfilled promise with a new one. Why? I thought that maybe one of the tests where erroing and wasn't able to continue after.updatePromiseStatusLogandwritePromiseLogToS3IfCounteradded. They can be used to send logs to S3.runTest(): if the test times-out (more than 50s to complete), it closes itself so thatrunMultipleTests()can restart it . Why? After creating the logger functions that write to S3 I found that some of the promises could get randomly stuck at iteration 0.runTestpromise initialization. 300ms sleep time added before eachrunTestpromise is initialized. Why? At this point I suspected the problem was the container couldn't handle 5 incognito instances initialized at the same time.