microsoft / vstest

Visual Studio Test Platform is the runner and engine that powers test explorer and vstest.console.
MIT License
898 stars 323 forks source link

Providing a lot of dll files ends up as stack overflow #3514

Closed nohwnd closed 4 months ago

nohwnd commented 2 years ago

Using a response file and providing 10k or so dlls to run will result in StackOverflow in vstest.console.exe. There is no report of where it is happening.

This is unlikely to happen to anyone, but it might be highlighting some possible perf improvement.

acerock commented 1 year ago

Having the same issue. With 216 assemblies it works, but with 217 in the response file I get "Process is terminated due to StackOverflowException"

Running latest vstest.console.exe shipped with VS 2022 Enterprise Edition from Developer Command Prompt for VS 2020: vstest.console @test.rsp

nohwnd commented 1 year ago

Mkay.

C:\Program Files\Microsoft Visual Studio\2022\Enterprise>vstest.console p1.dll p2.dll p3.dll p4.dll p5.dll p6.dll p7.dll p8.dll p9.dll p10.dll p11.dll p12.dll p13.dll p14.dll p15.dll p16.dll p17.dll p18.dll p19.dll p20.dll p21.dll p22.dll p23.dll p24.dll p25.dll p26.dll p27.dll p28.dll p29.dll p30.dll p31.dll p32.dll p33.dll p34.dll p35.dll p36.dll p37.dll p38.dll p39.dll p40.dll p41.dll p42.dll p43.dll p44.dll p45.dll p46.dll p47.dll p48.dll p49.dll p50.dll p51.dll p52.dll p53.dll p54.dll p55.dll p56.dll p57.dll p58.dll p59.dll p60.dll p61.dll p62.dll p63.dll p64.dll p65.dll p66.dll p67.dll p68.dll p69.dll p70.dll p71.dll p72.dll p73.dll p74.dll p75.dll p76.dll p77.dll p78.dll p79.dll p80.dll p81.dll p82.dll p83.dll p84.dll p85.dll p86.dll p87.dll p88.dll p89.dll p90.dll p91.dll p92.dll p93.dll p94.dll p95.dll p96.dll p97.dll p98.dll p99.dll p100.dll p101.dll p102.dll p103.dll p104.dll p105.dll p106.dll p107.dll p108.dll p109.dll p110.dll p111.dll p112.dll p113.dll p114.dll p115.dll p116.dll p117.dll p118.dll p119.dll p120.dll p121.dll p122.dll p123.dll p124.dll p125.dll p126.dll p127.dll p128.dll p129.dll p130.dll p131.dll p132.dll p133.dll p134.dll p135.dll p136.dll p137.dll p138.dll p139.dll p140.dll p141.dll p142.dll p143.dll p144.dll p145.dll p146.dll p147.dll p148.dll p149.dll p150.dll p151.dll p152.dll p153.dll p154.dll p155.dll p156.dll p157.dll p158.dll p159.dll p160.dll p161.dll p162.dll p163.dll p164.dll p165.dll p166.dll p167.dll p168.dll p169.dll p170.dll p171.dll p172.dll p173.dll p174.dll p175.dll p176.dll p177.dll p178.dll p179.dll p180.dll p181.dll p182.dll p183.dll p184.dll p185.dll p186.dll p187.dll p188.dll p189.dll p190.dll p191.dll p192.dll p193.dll p194.dll p195.dll p196.dll p197.dll p198.dll p199.dll p200.dll p201.dll p202.dll p203.dll p204.dll p205.dll p206.dll p207.dll p208.dll p209.dll p210.dll p211.dll p212.dll p213.dll p214.dll p215.dll p216.dll p217.dll p218.dll p219.dll p220.dll Microsoft (R) Test Execution Command Line Tool Version 17.7.2 (x64) Copyright (c) Microsoft Corporation. All rights reserved.

Process is terminated due to StackOverflowException.

nohwnd commented 1 year ago

Actually only p1.dll - p55.dll is enough on my system to SO.

nohwnd commented 12 months ago

This is a side effect of the argument processor and argument executor design. Argument processor in vstest is responsible for parsing out an argument that you provide on commandline. And Argument executor is acting on the argument.

So in this case ArgumentProcessor for TestSource is created 220 times, and every time it wraps the argument executor in a lazy wrapper, capturing the argument value (the dll path) as a closure.

image (1) image (2) image (3)

is this a pressing issue for you @acerock?

nohwnd commented 4 months ago

This would require changing architecture of argument processor and won't be implemented, we are focusing on adding new features to Testing.Platform instead. https://aka.ms/testingplatform