Unable to get results of GC benchmarks

[alexcovington]

I'm currently stuck at the analyze step of running the GC benchmarks.

I can successfully run one of the benchfiles, but when running python3 . diff, I get the following error:

$ python3 . run bench/suite/normal_server.yaml --overwrite
$ python3 . diff bench/suite/normal_server.yaml
ValueError: <Microsoft.Diagnostics.Tracing.Parsers.Clr.GCType object at 0x7f6f91342240> is not a valid GCType

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "./__main__.py", line 9, in <module>
    run_command(ALL_COMMANDS)
  File "./src/commonlib/command.py", line 87, in run_command
    run_command_worker(commands, cmd_and_args)
  File "./src/commonlib/command.py", line 243, in run_command_worker
    cast(Callable[[Any], None], command.fn)(args)
  File "./src/analysis/report.py", line 507, in diff
    doc = diff_for_jupyter(
  File "./src/analysis/report.py", line 545, in diff_for_jupyter
    diffables = get_diffables(
  File "./src/analysis/diffable.py", line 160, in get_diffables
    return get_diffables_from_bench_file(
  File "./src/analysis/diffable.py", line 258, in get_diffables_from_bench_file
    all_runs: Mapping[SingleTestCombination, SingleDiffed] = make_mapping(
  File "./src/commonlib/collection_util.py", line 72, in make_mapping
    for k, v in pairs:
  File "./src/analysis/diffable.py", line 259, in <genexpr>
    _get_for_single_test(
  File "./src/analysis/diffable.py", line 131, in _get_for_single_test
    iterations: Sequence[MaybeMetricValuesForSingleIteration] = [
  File "./src/analysis/diffable.py", line 132, in <listcomp>
    traces.get_run_metrics(iteration.to_test_result(), run_metrics)
  File "./src/analysis/process_trace.py", line 338, in get_run_metrics
    self.get(test_result, need_mechanisms_and_reasons=False, need_join_info=False),
  File "./src/analysis/process_trace.py", line 314, in get
    updated = get_processed_trace(
  File "./src/analysis/process_trace.py", line 100, in get_processed_trace
    _get_processed_trace_from_process(
  File "./src/analysis/process_trace.py", line 160, in _get_processed_trace_from_process
    process_names, proc = get_process_names_and_process_info(
  File "./src/analysis/core_analysis.py", line 205, in get_process_names_and_process_info
    get_process_info_from_process(clr, p, trace_path, process, show_name),
  File "./src/analysis/core_analysis.py", line 252, in get_process_info_from_process
    return get_process_info_from_mang(p, trace_path, process, show_name, mang)
  File "./src/analysis/core_analysis.py", line 269, in get_process_info_from_mang
    gcs=_get_gcs_from_mang(mang, show_name),
  File "./src/analysis/core_analysis.py", line 241, in _get_gcs_from_mang
    return [gc for i, gc in enumerate(unfiltered_gcs) if flt(i, gc)]
  File "./src/analysis/core_analysis.py", line 241, in <listcomp>
    return [gc for i, gc in enumerate(unfiltered_gcs) if flt(i, gc)]
  File "./src/analysis/core_analysis.py", line 230, in flt
    gc_type = GCType(gc.Type)
  File "/usr/lib/python3.8/enum.py", line 304, in __call__
    return cls.__new__(cls, value)
  File "/usr/lib/python3.8/enum.py", line 595, in __new__
    raise exc
  File "/usr/lib/python3.8/enum.py", line 579, in __new__
    result = cls._missing_(value)
  File "/usr/lib/python3.8/enum.py", line 608, in _missing_
    raise ValueError("%r is not a valid %s" % (value, cls.__name__))
ValueError: <Microsoft.Diagnostics.Tracing.Parsers.Clr.GCType object at 0x7f6f91342240> is not a valid GCType
Processing /home/amd/projects/performance/src/benchmarks/gc/bench/suite/normal_server.yaml.out/defgcperfsim__a__only_config__0gb__0.nettrace (1.63 MB)

Output of dotnet --info:

$ dotnet --info
.NET SDK (reflecting any global.json):
 Version:   6.0.100-preview.4.21255.9
 Commit:    950e4949a7

Runtime Environment:
 OS Name:     ubuntu
 OS Version:  20.04
 OS Platform: Linux
 RID:         ubuntu.20.04-x64
 Base Path:   /home/amd/.dotnet/sdk/6.0.100-preview.4.21255.9/

Host (useful for support):
  Version: 6.0.0-preview.4.21253.7
  Commit:  bfd6048a60

.NET SDKs installed:
  3.1.404 [/home/amd/.dotnet/sdk]
  3.1.407 [/home/amd/.dotnet/sdk]
  5.0.101 [/home/amd/.dotnet/sdk]
  6.0.100-alpha.1.20608.2 [/home/amd/.dotnet/sdk]
  6.0.100-preview.2.21112.6 [/home/amd/.dotnet/sdk]
  6.0.100-preview.3.21153.9 [/home/amd/.dotnet/sdk]
  6.0.100-preview.3.21173.9 [/home/amd/.dotnet/sdk]
  6.0.100-preview.4.21255.9 [/home/amd/.dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 3.1.10 [/home/amd/.dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.13 [/home/amd/.dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.1 [/home/amd/.dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.0-alpha.1.20573.11 [/home/amd/.dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.0-preview.2.21111.15 [/home/amd/.dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.0-preview.3.21152.6 [/home/amd/.dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.0-preview.3.21172.8 [/home/amd/.dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.0-preview.4.21253.5 [/home/amd/.dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 3.1.10 [/home/amd/.dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.13 [/home/amd/.dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.1 [/home/amd/.dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.0-alpha.1.20561.11 [/home/amd/.dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.0-preview.2.21110.7 [/home/amd/.dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.0-preview.3.21152.1 [/home/amd/.dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.0-preview.3.21167.1 [/home/amd/.dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.0-preview.4.21253.7 [/home/amd/.dotnet/shared/Microsoft.NETCore.App]

To install additional .NET runtimes or SDKs:
  https://aka.ms/dotnet-download

[danmoseley]

@ivdiazsa looks like this is your script.

[ivdiazsa]

This stack trace looks quite strange. At simple glance, I have no idea why it would end up with an invalid type. I'll begin investigating the issue. Thanks for bringing it to my attention.

[alexcovington]

@ivdiazsa Friendly ping. Wondering if there are any updates on this?