Open wohenniubi opened 1 year ago
Here, "B0" means batch size used is 0 which is invalid.
We are supporting nvidia implementation inside CM and have currently tested on L4, T4, A100 and RTX 4090. We'll be very happy to assist you if you can test it on H100. Here, are the instructions: https://github.com/mlcommons/ck/blob/master/docs/mlperf/inference/bert/README_nvidia.md and public discord channel for any queries.
make generate_engines RUN_ARGS="--benchmarks=bert --scenarios=offline"
will lead to the errorError Code 4: Internal Error (Network has dynamic or shape inputs, but no optimization profile has been defined.)
The detailed error is as follows:
The system device id info is as follows: I have already set the system id as H100_PCIe_80GB_Custom
============= DETECTED SYSTEM ==============
SystemConfiguration: System ID (Optional Alias): H100_PCIe_80GB_Custom CPUConfiguration: 2x CPU (CPUArchitecture.x86_64): Intel(R) Xeon(R) Platinum 8480+ 56 Cores, 2 Threads/Core MemoryConfiguration: 528.08 GB (Matching Tolerance: 0.05) AcceleratorConfiguration: 2x GPU (0x233110DE): NVIDIA H100 PCIe AcceleratorType: Discrete SM Compute Capability: 90 Memory Capacity: 79.65 GiB Max Power Limit: 310.0 W NUMA Config String: &