Windows tests frequently fails to compile in CI

[jszwedko]

Example: https://github.com/timberio/vector/runs/1392147593#step:3:1139

    Compiling rdkafka v0.24.0
memory allocation of 150994944 bytes failed
error: could not compile `vector`.

This appears to happen pretty frequently on master (most recent commits have failed with this).

[jszwedko]

Notes so far:

I tried to replicate this using a similarly sized m5.large in AWS running Windows_Server-2019-English-Full-Base-2020.10.14. It also has 2 vCPU but 8 GB of RAM rather than 7 GB (github runner specs. It compiled fine, but definitely has a lot of memory pressure (at peak it was using almost 100% of RAM and 110% of swap/page file).

My next thought is to check disk space as Ana suggests. It's possible it is running out of space to swap. The initial 30 GB EBS volume I had turned out to be too small. Github runners have 14 GB (but also have preinstalled software).

[jszwedko]

It doesn't appear to be due to running out of disk space:

Before compilation:

Run Get-CimInstance -Class CIM_LogicalDisk

DeviceID DriveType ProviderName VolumeName        Size         FreeSpace
-------- --------- ------------ ----------        ----         ---------
A:       2                                                     
C:       3                      Windows           274351517696 93256208384
D:       3                      Temporary Storage 15030284288  13083041792

After failed compilation:

Run Get-CimInstance -Class CIM_LogicalDisk

DeviceID DriveType ProviderName VolumeName        Size         FreeSpace
-------- --------- ------------ ----------        ----         ---------
A:       2                                                     
C:       3                      Windows           274351517696 92219043840
D:       3                      Temporary Storage 15030284288  6980259840

Run: https://github.com/timberio/vector/runs/1397419100

[jszwedko]

Fixed by #5108