search

microsoft / VirtualClient

Benchmarking and monitoring automation, designed for cloud, built with .NET.
http://microsoft.github.io/VirtualClient/
MIT License
85 stars 38 forks source link

DeathStarBench Profile: Getting `Error response from daemon: manifest for consul:latest not found: manifest unknown: manifest unknown` #320

Closed dimoibiehg closed 2 months ago

dimoibiehg commented 4 months ago

Describe the bug When running the following command for DeathStarBench workload in Ubuntu: 

sudo /usr/bin/virtualclient --profile=./PERF-NETWORK-DEATHSTARBENCH.json --packages=https://virtualclient.blob.core.windows.net/packages

get the following error:

[05/16/2024 14:38:02] Profile: Initialize
[05/16/2024 14:38:02] Profile: Install Dependencies
[05/16/2024 14:38:02] Profile: Dependency = DependencyPackageInstallation (scenario=InstallDeathStarBenchPackage)
[05/16/2024 14:38:02] Profile: Dependency = DockerInstallation (scenario=InstallDocker)
[05/16/2024 14:38:12] Profile: Dependency = LinuxPackageInstallation (scenario=InstallLinuxPackages)
[05/16/2024 14:38:35] Profile: Dependency = ApiServer (scenario=StartAPIServer)
[05/16/2024 14:38:35] Profile: Dependency = LinuxPackageInstallation (scenario=InstallEpelPackage)
[05/16/2024 14:38:35] Profile: Dependency = LinuxPackageInstallation (scenario=InstallAtop)
Hosting environment: Production
Content root path: /opt/virtualclient/
Now listening on: http://[::]:4500
Application started. Press Ctrl+C to shut down.
[05/16/2024 14:38:47] Profile: Execute Monitors
[05/16/2024 14:38:47] Profile: Monitor = PerfCounterMonitor (scenario=CaptureCounters)
[05/16/2024 14:38:47] Profile: Monitor = LspciMonitor (scenario=CaptureDeviceInformation)
[05/16/2024 14:38:47] Profile: Execute Actions
[05/16/2024 14:38:47] Profile: Action = DeathStarBenchExecutor (scenario=SocialNetwork)
[05/16/2024 15:09:44] Profile: Action = DeathStarBenchExecutor (scenario=MediaMicroservices)
[05/16/2024 15:23:30] 
Process execution failed (error/exit code=18, command=sudo docker-compose -f docker-compose.yml up -d).

StandardError: user-service Pulling 
movie-id-service Pulling 
cast-info-service Pulling 
user-review-service Pulling 
movie-review-service Pulling 
dns-media Pulling 
review-storage-service Pulling 
movie-info-service Pulling 
text-service Pulling 
unique-id-service Pulling 
plot-service Pulling 
compose-review-service Pulling 
rating-service Pulling 
34667c7e4631 Pulling fs layer 
d18d76a881a4 Pulling fs layer 
119c7358fbfc Pulling fs layer 
2aaf13f3eff0 Pulling fs layer 
557e7c725261 Pulling fs layer 
2174816b6bf4 Pulling fs layer 
61318c207cbf Pulling fs layer 
35f4dec8d6e2 Pulling fs layer 
2aaf13f3eff0 Waiting 
61318c207cbf Waiting 
557e7c725261 Waiting 
35f4dec8d6e2 Waiting 
2174816b6bf4 Waiting 
34667c7e4631 Pulling fs layer 
d18d76a881a4 Pulling fs layer 
119c7358fbfc Pulling fs layer 
2aaf13f3eff0 Pulling fs layer 
557e7c725261 Pulling fs layer 
2174816b6bf4 Pulling fs layer 
61318c207cbf Pulling fs layer 
35f4dec8d6e2 Pulling fs layer 
35f4dec8d6e2 Waiting 
2aaf13f3eff0 Waiting 
2174816b6bf4 Waiting 
557e7c725261 Waiting 
61318c207cbf Waiting 
34667c7e4631 Pulling fs layer 
d18d76a881a4 Pulling fs layer 
119c7358fbfc Pulling fs layer 
2aaf13f3eff0 Pulling fs layer 
557e7c725261 Pulling fs layer 
2174816b6bf4 Pulling fs layer 
61318c207cbf Pulling fs layer 
35f4dec8d6e2 Pulling fs layer 
35f4dec8d6e2 Waiting 
2aaf13f3eff0 Waiting 
557e7c725261 Waiting 
2174816b6bf4 Waiting 
61318c207cbf Waiting 
d18d76a881a4 Verifying Checksum 
d18d76a881a4 Download complete 
d18d76a881a4 Downloading     760B/852B
d18d76a881a4 Downloading     852B/852B
d18d76a881a4 Verifying Checksum 
d18d76a881a4 Download complete 
d18d76a881a4 Downloading     852B/852B
d18d76a881a4 Verifying Checksum 
d18d76a881a4 Download complete 
119c7358fbfc Downloading     529B/529B
119c7358fbfc Download complete 
119c7358fbfc Downloading     529B/529B
119c7358fbfc Download complete 
119c7358fbfc Download complete 
34667c7e4631 Downloading  440.6kB/43.56MB
34667c7e4631 Downloading  440.6kB/43.56MB
34667c7e4631 Downloading  440.6kB/43.56MB
34667c7e4631 Downloading  882.9kB/43.56MB
34667c7e4631 Downloading  882.9kB/43.56MB
34667c7e4631 Downloading  882.9kB/43.56MB
34667c7e4631 Downloading  1.768MB/43.56MB
34667c7e4631 Downloading  1.768MB/43.56MB
34667c7e4631 Downloading  1.768MB/43.56MB
34667c7e4631 Downloading  2.648MB/43.56MB
34667c7e4631 Downloading  2.648MB/43.56MB
34667c7e4631 Downloading  2.648MB/43.56MB
34667c7e4631 Downloading  3.537MB/43.56MB
34667c7e4631 Downloading  3.537MB/43.56MB
34667c7e4631 Downloading  3.537MB/43.56MB
34667c7e4631 Pulling fs layer 
d18d76a881a4 Pulling fs layer 
119c7358fbfc Pulling fs layer 
2aaf13f3eff0 Pulling fs layer 
557e7c725261 Pulling fs layer 
2174816b6bf4 Pulling fs layer 
61318c207cbf Pulling fs layer 
35f4dec8d6e2 Pulling fs layer 
35f4dec8d6e2 Waiting 
34667c7e4631 Downloading  3.537MB/43.56MB
d18d76a881a4 Download complete 
119c7358fbfc Download complete 
2aaf13f3eff0 Waiting 
557e7c725261 Waiting 
61318c207cbf Waiting 
2174816b6bf4 Waiting 
9fbefa337077 Pulling fs layer 
4c1e632f1ae9 Pulling fs layer 
4f4fb700ef54 Pulling fs layer 
9fbefa337077 Waiting 
4c1e632f1ae9 Waiting 
4f4fb700ef54 Waiting 
review-storage-service Error 
cast-info-service Error 
compose-review-service Error 
user-service Error 
user-review-service Error 
plot-service Error 
movie-info-service Error 
text-service Error 
Error response from daemon: Get "https://registry-1.docker.io/v2/yg397/media-microservices/manifests/sha256:deab297966f0b27475cb124b8afe0c7c653990c19254d538e3cfe7696836fddd": Get "https://auth.docker.io/token?scope=repository%3Ayg397%2Fmedia-microservices%3Apull&service=registry.docker.io": dial tcp: lookup auth.docker.io on 127.0.0.53:53: server misbehaving

   at VirtualClient.ProcessExtensions.ThrowErrored[TError](IProcessProxy process, String errorMessage, ErrorReason errorReason) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Core/ProcessExtensions.cs:line 385
   at VirtualClient.ProcessExtensions.ThrowIfErrored[TError](IProcessProxy process, IEnumerable`1 successCodes, String errorMessage, ErrorReason errorReason) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Core/ProcessExtensions.cs:line 227
   at VirtualClient.ProcessExtensions.ThrowIfErrored[TError](IProcessProxy process, String errorMessage, ErrorReason errorReason) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Core/ProcessExtensions.cs:line 208
   at VirtualClient.Actions.DeathStarBenchExecutor.<>c__DisplayClass58_1.<<ExecuteCommandAsync>b__0>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Actions/DeathStarBench/DeathStarBenchExecutor.cs:line 443
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.LogMessageAsync[TResult](ILogger logger, String message, EventContext eventContext, Func`1 body, Boolean displayErrors) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 592

[05/16/2024 15:23:30] 
Error occured at starting server in mediamicroservices

   at VirtualClient.Actions.DeathStarBenchServerExecutor.ExecuteServerAsync(EventContext telemetryContext, CancellationToken cancellationToken) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Actions/DeathStarBench/DeathStarBenchServerExecutor.cs:line 137
   at VirtualClient.Actions.DeathStarBenchServerExecutor.<>c__DisplayClass9_0.<<ExecuteAsync>b__0>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Actions/DeathStarBench/DeathStarBenchServerExecutor.cs:line 67
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.<>c__DisplayClass15_0.<<LogMessageAsync>b__0>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 566
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.LogMessageAsync[TResult](ILogger logger, String message, EventContext eventContext, Func`1 body, Boolean displayErrors) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 592

[05/16/2024 15:23:30] 
Error occured at starting server in mediamicroservices

   at VirtualClient.Actions.DeathStarBenchServerExecutor.ExecuteServerAsync(EventContext telemetryContext, CancellationToken cancellationToken) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Actions/DeathStarBench/DeathStarBenchServerExecutor.cs:line 137
   at VirtualClient.Actions.DeathStarBenchServerExecutor.<>c__DisplayClass9_0.<<ExecuteAsync>b__0>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Actions/DeathStarBench/DeathStarBenchServerExecutor.cs:line 67
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.<>c__DisplayClass15_0.<<LogMessageAsync>b__0>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 566
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.LogMessageAsync[TResult](ILogger logger, String message, EventContext eventContext, Func`1 body, Boolean displayErrors) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 592
   at VirtualClient.Contracts.VirtualClientComponent.<>c__DisplayClass132_1.<<ExecuteAsync>b__4>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientComponent.cs:line 584
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.<>c__DisplayClass15_0.<<LogMessageAsync>b__0>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 566
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.LogMessageAsync[TResult](ILogger logger, String message, EventContext eventContext, Func`1 body, Boolean displayErrors) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 592

[05/16/2024 15:23:30] 
Error occured at starting server in mediamicroservices

   at VirtualClient.Actions.DeathStarBenchServerExecutor.ExecuteServerAsync(EventContext telemetryContext, CancellationToken cancellationToken) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Actions/DeathStarBench/DeathStarBenchServerExecutor.cs:line 137
   at VirtualClient.Actions.DeathStarBenchServerExecutor.<>c__DisplayClass9_0.<<ExecuteAsync>b__0>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Actions/DeathStarBench/DeathStarBenchServerExecutor.cs:line 67
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.<>c__DisplayClass15_0.<<LogMessageAsync>b__0>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 566
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.LogMessageAsync[TResult](ILogger logger, String message, EventContext eventContext, Func`1 body, Boolean displayErrors) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 592
   at VirtualClient.Contracts.VirtualClientComponent.<>c__DisplayClass132_1.<<ExecuteAsync>b__4>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientComponent.cs:line 584
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.<>c__DisplayClass15_0.<<LogMessageAsync>b__0>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 566
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.LogMessageAsync[TResult](ILogger logger, String message, EventContext eventContext, Func`1 body, Boolean displayErrors) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 592
   at VirtualClient.Contracts.VirtualClientComponent.ExecuteAsync(CancellationToken cancellationToken) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientComponent.cs:line 575
   at VirtualClient.Actions.DeathStarBenchExecutor.ExecuteAsync(EventContext telemetryContext, CancellationToken cancellationToken) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Actions/DeathStarBench/DeathStarBenchExecutor.cs:line 214
   at VirtualClient.Contracts.VirtualClientComponent.<>c__DisplayClass132_1.<<ExecuteAsync>b__4>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientComponent.cs:line 584
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.<>c__DisplayClass15_0.<<LogMessageAsync>b__0>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 566
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.LogMessageAsync[TResult](ILogger logger, String message, EventContext eventContext, Func`1 body, Boolean displayErrors) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 592

[05/16/2024 15:23:30] 
Error occured at starting server in mediamicroservices

   at VirtualClient.Actions.DeathStarBenchServerExecutor.ExecuteServerAsync(EventContext telemetryContext, CancellationToken cancellationToken) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Actions/DeathStarBench/DeathStarBenchServerExecutor.cs:line 137
   at VirtualClient.Actions.DeathStarBenchServerExecutor.<>c__DisplayClass9_0.<<ExecuteAsync>b__0>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Actions/DeathStarBench/DeathStarBenchServerExecutor.cs:line 67
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.<>c__DisplayClass15_0.<<LogMessageAsync>b__0>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 566
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.LogMessageAsync[TResult](ILogger logger, String message, EventContext eventContext, Func`1 body, Boolean displayErrors) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 592
   at VirtualClient.Contracts.VirtualClientComponent.<>c__DisplayClass132_1.<<ExecuteAsync>b__4>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientComponent.cs:line 584
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.<>c__DisplayClass15_0.<<LogMessageAsync>b__0>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 566
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.LogMessageAsync[TResult](ILogger logger, String message, EventContext eventContext, Func`1 body, Boolean displayErrors) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 592
   at VirtualClient.Contracts.VirtualClientComponent.ExecuteAsync(CancellationToken cancellationToken) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientComponent.cs:line 575
   at VirtualClient.Actions.DeathStarBenchExecutor.ExecuteAsync(EventContext telemetryContext, CancellationToken cancellationToken) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Actions/DeathStarBench/DeathStarBenchExecutor.cs:line 214
   at VirtualClient.Contracts.VirtualClientComponent.<>c__DisplayClass132_1.<<ExecuteAsync>b__4>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientComponent.cs:line 584
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.<>c__DisplayClass15_0.<<LogMessageAsync>b__0>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 566
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.LogMessageAsync[TResult](ILogger logger, String message, EventContext eventContext, Func`1 body, Boolean displayErrors) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 592
   at VirtualClient.Contracts.VirtualClientComponent.ExecuteAsync(CancellationToken cancellationToken) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientComponent.cs:line 575
   at VirtualClient.ProfileExecutor.<>c__DisplayClass79_2.<<ExecuteActionsAsync>b__3>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Core/ProfileExecutor.cs:line 425
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.<>c__DisplayClass15_0.<<LogMessageAsync>b__0>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 566
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.LogMessageAsync[TResult](ILogger logger, String message, EventContext eventContext, Func`1 body, Boolean displayErrors) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 592

[05/16/2024 15:23:30] Profile: Wait for Exit...
[05/16/2024 15:23:30] Profile: Exited
[05/16/2024 15:23:30] 
Error occured at starting server in mediamicroservices

   at VirtualClient.Contracts.Extensions.CommonExtensions.ThrowIfErrored(Task runtimeTask) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/Extensions/CommonExtensions.cs:line 86
   at VirtualClient.ProfileExecutor.ExecuteAsync(ProfileTiming timing, CancellationToken cancellationToken) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Core/ProfileExecutor.cs:line 286
   at VirtualClient.RunProfileCommand.ExecuteProfileAsync(IEnumerable`1 profiles, IServiceCollection dependencies, CancellationTokenSource cancellationTokenSource) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Main/RunProfileCommand.cs:line 813
   at VirtualClient.RunProfileCommand.ExecuteAsync(String[] args, CancellationTokenSource cancellationTokenSource) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Main/RunProfileCommand.cs:line 157

Application is shutting down...

Then, as my focus is on the hotel reservation application, I changed the profile by keeping only the action corresponding to hotel reservation, i.e., removing the other two actions in the profile JSON file, but got the consul error:

[05/16/2024 16:39:19] Profile: Initialize
[05/16/2024 16:39:19] Profile: Install Dependencies
[05/16/2024 16:39:19] Profile: Dependency = DependencyPackageInstallation (scenario=InstallDeathStarBenchPackage)
[05/16/2024 16:39:19] Profile: Dependency = DockerInstallation (scenario=InstallDocker)
[05/16/2024 16:39:28] Profile: Dependency = LinuxPackageInstallation (scenario=InstallLinuxPackages)
[05/16/2024 16:39:38] Profile: Dependency = ApiServer (scenario=StartAPIServer)
[05/16/2024 16:39:38] Profile: Dependency = LinuxPackageInstallation (scenario=InstallEpelPackage)
[05/16/2024 16:39:38] Profile: Dependency = LinuxPackageInstallation (scenario=InstallAtop)
Hosting environment: Production
Content root path: /opt/virtualclient/
Now listening on: http://[::]:4500
Application started. Press Ctrl+C to shut down.
[05/16/2024 16:39:42] Profile: Execute Monitors
[05/16/2024 16:39:42] Profile: Monitor = PerfCounterMonitor (scenario=CaptureCounters)
[05/16/2024 16:39:42] Profile: Monitor = LspciMonitor (scenario=CaptureDeviceInformation)
[05/16/2024 16:39:42] Profile: Execute Actions
[05/16/2024 16:39:43] Profile: Action = DeathStarBenchExecutor (scenario=HotelReservation)
[05/16/2024 16:40:20] 
Process execution failed (error/exit code=18, command=sudo docker-compose -f docker-compose.yml up -d).

StandardError: user Pulling 
geo Pulling 
profile Pulling 
recommendation Pulling 
frontend Pulling 
reservation Pulling 
consul Pulling 
search Pulling 
rate Pulling 
user Error 
profile Error 
reservation Error 
rate Error 
geo Error 
search Error 
recommendation Error 
consul Error 
frontend Error 
Error response from daemon: manifest for consul:latest not found: manifest unknown: manifest unknown

   at VirtualClient.ProcessExtensions.ThrowErrored[TError](IProcessProxy process, String errorMessage, ErrorReason errorReason) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Core/ProcessExtensions.cs:line 385
   at VirtualClient.ProcessExtensions.ThrowIfErrored[TError](IProcessProxy process, IEnumerable`1 successCodes, String errorMessage, ErrorReason errorReason) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Core/ProcessExtensions.cs:line 227
   at VirtualClient.ProcessExtensions.ThrowIfErrored[TError](IProcessProxy process, String errorMessage, ErrorReason errorReason) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Core/ProcessExtensions.cs:line 208
   at VirtualClient.Actions.DeathStarBenchExecutor.<>c__DisplayClass58_1.<<ExecuteCommandAsync>b__0>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Actions/DeathStarBench/DeathStarBenchExecutor.cs:line 443
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.LogMessageAsync[TResult](ILogger logger, String message, EventContext eventContext, Func`1 body, Boolean displayErrors) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 592

[05/16/2024 16:40:20] 
Error occured at starting server in hotelreservation

   at VirtualClient.Actions.DeathStarBenchServerExecutor.ExecuteServerAsync(EventContext telemetryContext, CancellationToken cancellationToken) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Actions/DeathStarBench/DeathStarBenchServerExecutor.cs:line 137
   at VirtualClient.Actions.DeathStarBenchServerExecutor.<>c__DisplayClass9_0.<<ExecuteAsync>b__0>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Actions/DeathStarBench/DeathStarBenchServerExecutor.cs:line 67
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.<>c__DisplayClass15_0.<<LogMessageAsync>b__0>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 566
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.LogMessageAsync[TResult](ILogger logger, String message, EventContext eventContext, Func`1 body, Boolean displayErrors) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 592

[05/16/2024 16:40:20] 
Error occured at starting server in hotelreservation

   at VirtualClient.Actions.DeathStarBenchServerExecutor.ExecuteServerAsync(EventContext telemetryContext, CancellationToken cancellationToken) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Actions/DeathStarBench/DeathStarBenchServerExecutor.cs:line 137
   at VirtualClient.Actions.DeathStarBenchServerExecutor.<>c__DisplayClass9_0.<<ExecuteAsync>b__0>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Actions/DeathStarBench/DeathStarBenchServerExecutor.cs:line 67
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.<>c__DisplayClass15_0.<<LogMessageAsync>b__0>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 566
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.LogMessageAsync[TResult](ILogger logger, String message, EventContext eventContext, Func`1 body, Boolean displayErrors) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 592
   at VirtualClient.Contracts.VirtualClientComponent.<>c__DisplayClass132_1.<<ExecuteAsync>b__4>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientComponent.cs:line 584
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.<>c__DisplayClass15_0.<<LogMessageAsync>b__0>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 566
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.LogMessageAsync[TResult](ILogger logger, String message, EventContext eventContext, Func`1 body, Boolean displayErrors) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 592

[05/16/2024 16:40:20] 
Error occured at starting server in hotelreservation

   at VirtualClient.Actions.DeathStarBenchServerExecutor.ExecuteServerAsync(EventContext telemetryContext, CancellationToken cancellationToken) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Actions/DeathStarBench/DeathStarBenchServerExecutor.cs:line 137
   at VirtualClient.Actions.DeathStarBenchServerExecutor.<>c__DisplayClass9_0.<<ExecuteAsync>b__0>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Actions/DeathStarBench/DeathStarBenchServerExecutor.cs:line 67
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.<>c__DisplayClass15_0.<<LogMessageAsync>b__0>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 566
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.LogMessageAsync[TResult](ILogger logger, String message, EventContext eventContext, Func`1 body, Boolean displayErrors) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 592
   at VirtualClient.Contracts.VirtualClientComponent.<>c__DisplayClass132_1.<<ExecuteAsync>b__4>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientComponent.cs:line 584
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.<>c__DisplayClass15_0.<<LogMessageAsync>b__0>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 566
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.LogMessageAsync[TResult](ILogger logger, String message, EventContext eventContext, Func`1 body, Boolean displayErrors) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 592
   at VirtualClient.Contracts.VirtualClientComponent.ExecuteAsync(CancellationToken cancellationToken) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientComponent.cs:line 575
   at VirtualClient.Actions.DeathStarBenchExecutor.ExecuteAsync(EventContext telemetryContext, CancellationToken cancellationToken) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Actions/DeathStarBench/DeathStarBenchExecutor.cs:line 214
   at VirtualClient.Contracts.VirtualClientComponent.<>c__DisplayClass132_1.<<ExecuteAsync>b__4>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientComponent.cs:line 584
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.<>c__DisplayClass15_0.<<LogMessageAsync>b__0>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 566
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.LogMessageAsync[TResult](ILogger logger, String message, EventContext eventContext, Func`1 body, Boolean displayErrors) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 592

[05/16/2024 16:40:20] 
Error occured at starting server in hotelreservation

   at VirtualClient.Actions.DeathStarBenchServerExecutor.ExecuteServerAsync(EventContext telemetryContext, CancellationToken cancellationToken) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Actions/DeathStarBench/DeathStarBenchServerExecutor.cs:line 137
   at VirtualClient.Actions.DeathStarBenchServerExecutor.<>c__DisplayClass9_0.<<ExecuteAsync>b__0>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Actions/DeathStarBench/DeathStarBenchServerExecutor.cs:line 67
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.<>c__DisplayClass15_0.<<LogMessageAsync>b__0>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 566
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.LogMessageAsync[TResult](ILogger logger, String message, EventContext eventContext, Func`1 body, Boolean displayErrors) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 592
   at VirtualClient.Contracts.VirtualClientComponent.<>c__DisplayClass132_1.<<ExecuteAsync>b__4>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientComponent.cs:line 584
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.<>c__DisplayClass15_0.<<LogMessageAsync>b__0>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 566
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.LogMessageAsync[TResult](ILogger logger, String message, EventContext eventContext, Func`1 body, Boolean displayErrors) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 592
   at VirtualClient.Contracts.VirtualClientComponent.ExecuteAsync(CancellationToken cancellationToken) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientComponent.cs:line 575
   at VirtualClient.Actions.DeathStarBenchExecutor.ExecuteAsync(EventContext telemetryContext, CancellationToken cancellationToken) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Actions/DeathStarBench/DeathStarBenchExecutor.cs:line 214
   at VirtualClient.Contracts.VirtualClientComponent.<>c__DisplayClass132_1.<<ExecuteAsync>b__4>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientComponent.cs:line 584
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.<>c__DisplayClass15_0.<<LogMessageAsync>b__0>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 566
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.LogMessageAsync[TResult](ILogger logger, String message, EventContext eventContext, Func`1 body, Boolean displayErrors) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 592
   at VirtualClient.Contracts.VirtualClientComponent.ExecuteAsync(CancellationToken cancellationToken) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientComponent.cs:line 575
   at VirtualClient.ProfileExecutor.<>c__DisplayClass79_2.<<ExecuteActionsAsync>b__3>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Core/ProfileExecutor.cs:line 425
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.<>c__DisplayClass15_0.<<LogMessageAsync>b__0>d.MoveNext() in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 566
--- End of stack trace from previous location ---
   at VirtualClient.Contracts.VirtualClientLoggingExtensions.LogMessageAsync[TResult](ILogger logger, String message, EventContext eventContext, Func`1 body, Boolean displayErrors) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/VirtualClientLoggingExtensions.cs:line 592

[05/16/2024 16:40:20] Profile: Wait for Exit...
[05/16/2024 16:40:20] Profile: Exited
[05/16/2024 16:40:20] 
Error occured at starting server in hotelreservation

   at VirtualClient.Contracts.Extensions.CommonExtensions.ThrowIfErrored(Task runtimeTask) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Contracts/Extensions/CommonExtensions.cs:line 86
   at VirtualClient.ProfileExecutor.ExecuteAsync(ProfileTiming timing, CancellationToken cancellationToken) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Core/ProfileExecutor.cs:line 286
   at VirtualClient.RunProfileCommand.ExecuteProfileAsync(IEnumerable`1 profiles, IServiceCollection dependencies, CancellationTokenSource cancellationTokenSource) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Main/RunProfileCommand.cs:line 813
   at VirtualClient.RunProfileCommand.ExecuteAsync(String[] args, CancellationTokenSource cancellationTokenSource) in /home/vsts/work/1/s/src/VirtualClient/VirtualClient.Main/RunProfileCommand.cs:line 157

Application is shutting down...

To Reproduce Steps to reproduce the behavior:

  1. VirtualClient version: v1.6.0
  2. VC command arguments: --profile=./PERF-NETWORK-DEATHSTARBENCH.json --packages=https://virtualclient.blob.core.windows.net/packages , exactly used this DeathStarBench profile existed on the repository.
  3. System under test configuration (OS, VM size, architecture, etc): Ubuntu 22.04, RAM 16GB, CPU core-i7

Expected behavior I expected to successfully run DeathStarBench and output the metrics.

Additional question It seems the problem for hotelReservation, related to this issue. So, I am wondering if there is any way to update the DeathStartBench package used by VirtualClient. In other words, is there any way to download deathstarbench.2.0.0.zip mentioned in the profile, and after my custom modification, use it in my command instead of the default one?

yangpanMS commented 4 months ago

Acknowledged, will try to repro. Just to confirm, social network is working for you and you are able to get metrics?

re: " is there any way to download deathstarbench.2.0.0.zip mentioned in the profile" yes it's a public package you can download via link https://virtualclient.blob.core.windows.net/packages/deathstarbench.2.0.0.zip It's essentailly a snapshot of the original repo

dimoibiehg commented 4 months ago

Acknowledged, will try to repro. Just to confirm, social network is working for you and you are able to get metrics?

re: " is there any way to download deathstarbench.2.0.0.zip mentioned in the profile" yes it's a public package you can download via link https://virtualclient.blob.core.windows.net/packages/deathstarbench.2.0.0.zip It's essentailly a snapshot of the original repo

Many thanks for your consideration. re: "Just to confirm, social network is working for you and you are able to get metrics?" yes, it seems that the social network application is working (by keeping only the action related to this applicaion in the profile Json file, see the following image of the console): Screenshot from 2024-05-17 17-09-20

yangpanMS commented 4 months ago

@nmalkapuram please help add your investigation here. Thanks.

nmalkapuram commented 4 months ago

I could replicate and see we have errors with both HotelReservation and MediaMicroservices. Looking into the issue and will update here as soon as I get the fix.

nmalkapuram commented 4 months ago

Hi @dimoibiehg, We had couple of issues here.

  1. Permission issue in mediamicroservices: We had to make a file "mediamicroservices/wrk2/deps/luajit/src/luajit" as executable to resolve this issue. Then we had an other error with make command. It required changing few XCFLAGS+= -fPIC -fPIE in Makefile to resolve this.
  2. consul:latest not found in hotelReservation consul image with latest tag was not found in docker. Updated to a specific version from the docker library to fix this issue

https://github.com/microsoft/VirtualClient/pull/332 .This is the PR for the fix. And you can find the package that is updated at https://virtualclientinternal.blob.core.windows.net/packages/deathstarbench.3.0.0.zip

yangpanMS commented 2 months ago

@dimoibiehg I'm going to close this issue due to no response. Please reopen as needed.