When parsing large Summary files, the UI will display more slowly

[BofengDuke]

Environment

Hardware Environment(Ascend/GPU/CPU):

Uncomment only one /device <> line, hit enter to put that in a new line, and remove leading whitespaces from that line:

/device ascend
/device gpu
/device cpu

/device ascend /device gpu

Software Environment:

• MindSpore version (source or binary):
• Python version (e.g., Python 3.7.5):
• OS platform and distribution (e.g., Linux Ubuntu 16.04):
• GCC/Compiler version (if compiled from source):

Describe the current behavior

I have more than 30 summary_dir, and each summary_dir has a summary file with a file size of more than 800M. When I start mindinsight, I found the UI page was empty, No data is displayed in the summary list for more than 30 seconds.

Describe the expected behavior

I expect to see the summary dir list when I open Minginsight

[BofengDuke]

Analysis

I found that in MindInsight only one process was used to load and parse the data. And when parsing the summary file in the ms_data_loader.py file, MindInsight used the Protobuf tool's ParseFromString method, which would consume CPU resources and prevent other threads from processing the request if the file is large. It cause there is no data in UI page over 30 seconds.

Advice

It is recommended that when using the ParseFromString method, another process should be used so that other threads of the current process are not affected to handle other requests