Improve build time for Q# hello world application

vadym-kl commented 4 years ago

Describe the enhancement request

Currently hello world sample takes around 7 times longer for Q# project to compile than for C# project.

To Reproduce

Run the following PowerShell script to reproduce:

function time-hello {
    param( [String]$language, [String]$project )
    Write-Host "****************"
    Write-Host "Creating hello-world console project: 'dotnet new console -n $project -lang $language"
    start-process dotnet "new console", "-n $project", "-lang $language" -Wait -WindowStyle Hidden
    Set-Location "./$project"
    Write-Host "Running 'dotnet restore' for $language console project"
    start-process dotnet -argumentlist "restore" -Wait -WindowStyle Hidden
    Write-Host "Measuring 'dotnet build --no-restore' for $language console project"
    Measure-Command `
        {start-process dotnet -argumentlist "build", "--no-restore" -Wait -WindowStyle Hidden} `
        | Out-String -Stream `
        | Select-String "TotalSeconds"
    Set-Location ..
    Write-Host "Deleting $language project directory"
    Remove-Item -r "./$project"
    Write-Host " "
}

Write-Host "****************"
Write-Host "System info:" 
dotnet --info | Select-String ".NET Core" -Context 0,4
Write-Host "****************"
Write-Host "QDK info:"
dotnet new -u | Select-String Quantum -Context 0,4
time-hello "Q#" "qs-hello"
time-hello "C#" "cs-hello"

Expected behavior

Given that Q# is transpiled into C# and Q# is much simpler programming language than C#, I would expect Q# project to take at most two times longer to build.

Actual behavior

Currently hello world sample takes around 7 times longer for Q# project to compile than for C# project, which is 14 seconds on a performant machine.

System information

OS: Windows
.NET Core Version: 3.1.201
Build is run on a dedicated SSD Samsung SSD 960 EVO 500 GB, separate from the SSD on which Windows is installed
CPU: Intel(R) Core(TM) i7-7700T CPU @ 2.90GHz set to full performance
32 GB RAM

Additional context

See videos showing build time with versions: -2003 -2004.

bettinaheim commented 4 years ago

I'll take a look as part of this months bug fixing session. Eliminating the dependency on the Restore target when building should improve the user experience for small projects.

bettinaheim commented 4 years ago

This should be improved with the changes to the Sdk setup with the next release.

msoeken commented 4 years ago

I am still observing the reported behavior (factor 7) with 0.12.20070124:

****************
System info:

> .NET Core SDK (reflecting any global.json):
   Version:   3.1.400-preview-015178
   Commit:    60cb58d3b1

  Runtime Environment:
> .NET Core SDKs installed:
    3.1.300 [C:\Program Files\dotnet\sdk]
    3.1.400-preview-015178 [C:\Program Files\dotnet\sdk]

> .NET Core runtimes installed:
    Microsoft.AspNetCore.All 2.1.16 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
    Microsoft.AspNetCore.All 2.1.18 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
    Microsoft.AspNetCore.App 2.1.16 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
    Microsoft.AspNetCore.App 2.1.18 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
> To install additional .NET Core runtimes or SDKs:
    https://aka.ms/dotnet-download
****************
QDK info:
>   Microsoft.Quantum.ProjectTemplates
      Details:
>       NuGetPackageId: Microsoft.Quantum.ProjectTemplates
        Version: 0.12.20070124
        Author: Microsoft
      Templates:
        Console Application (console) Q#
>       dotnet new -u Microsoft.Quantum.ProjectTemplates

****************
Creating hello-world console project: 'dotnet new console -n qs-hello -lang Q#
Running 'dotnet restore' for Q# console project
Measuring 'dotnet build --no-restore' for Q# console project
TotalSeconds      : 13.0265699
Deleting Q# project directory

****************
Creating hello-world console project: 'dotnet new console -n cs-hello -lang C#
Running 'dotnet restore' for C# console project
Measuring 'dotnet build --no-restore' for C# console project
TotalSeconds      : 2.0177722
Deleting C# project directory

ShaunJW360 commented 4 years ago

@vadym-kl Would it be possible to contribute to assingmnet?

vadym-kl commented 4 years ago

@ShaunJW360, hopefully @bettinaheim can help answer your question. I am not working on Q# compiler.

bettinaheim commented 4 years ago

@vadym-kl Thanks for the ping. Sorry, I should have posted an update earlier. @cesarzc could you give an update and suggestions for contributions to https://github.com/microsoft/qsharp-compiler/issues/437?

cesarzc commented 4 years ago

Hello @ShaunJW360, thank you for your interest in contributing to the Q# compiler!

I am currently working on this issue. Here's a bit of background on it:

When the compiler command line tool (qsc.exe) is invoked with the --perf option, it generates a JSON file with performance data. For the Hello World application, it looks like the following:

{
"OverallCompilation": 9365,
"OverallCompilation.Build": 1049,
"OverallCompilation.OutputGeneration": 25,
"OverallCompilation.OutputGeneration.BinaryGeneration": 0,
"OverallCompilation.OutputGeneration.SyntaxTreeSerialization": 24,
"OverallCompilation.ReferenceLoading": 7449,
"OverallCompilation.RewriteSteps": 802,
"OverallCompilation.SourcesLoading": 10
}

As it can be seen, most of the time is spent doing reference loading, which is what I am currently working on.

For #437, the JSON file with performance data looks like the following:

{
  "OverallCompilation": 58464,
  "OverallCompilation.Build": 48190,
  "OverallCompilation.OutputGeneration": 2853,
  "OverallCompilation.OutputGeneration.BinaryGeneration": 17,
  "OverallCompilation.OutputGeneration.DocumentationGeneration": 561,
  "OverallCompilation.OutputGeneration.SyntaxTreeSerialization": 2274,
  "OverallCompilation.ReferenceLoading": 1925,
  "OverallCompilation.RewriteSteps": 5436,
  "OverallCompilation.SourcesLoading": 27
}

As it can be seen, in this case most of the time is spent in the build task. However, we currently don't have more details about it, so a great first step to make progress on #437 would be to extend data points gathered by the --perf option to obtain more details on the build task.

Would you be interested in working on this first step regarding #437?

msoeken commented 4 years ago

@cesarzc Thanks for the additional input. Can you clarify whether the numbers are referring to time in ms, and whether the separation by . indicate hierarchy, in the sense that, e.g., all times reported in OverallCompilation.* are included in the time reported for OverallCompilation?

cesarzc commented 4 years ago

Yes, the numbers indicate time in ms. Regarding the . symbol, it does indicate hierarchy as you described it.

billti commented 1 year ago

Still an issue? Package restore can take a while, but in the MSBuild log on my machine I'm seeing the actual build step for new Q# console application take a little under 3 seconds now.

vadym-kl commented 1 year ago

Currently I see 3.5 times slower. See the steps to reproduce below.

Running

function time-hello {
    param( [String]$language, [String]$project )
    Write-Host "****************"
    Write-Host "Creating hello-world console project: 'dotnet new console -n $project -lang $language"
    start-process dotnet "new console", "-n $project", "-lang $language" -Wait -WindowStyle Hidden
    Set-Location "./$project"
    Write-Host "Running 'dotnet restore' for $language console project"
    start-process dotnet -argumentlist "restore" -Wait -WindowStyle Hidden
    Write-Host "Measuring 'dotnet build --no-restore' for $language console project"
    Measure-Command `
        {start-process dotnet -argumentlist "build", "--no-restore" -Wait -WindowStyle Hidden} `
        | Out-String -Stream `
        | Select-String "TotalSeconds"
    Set-Location ..
    Write-Host "Deleting $language project directory"
    Remove-Item -r "./$project"
    Write-Host " "
}

Write-Host "****************"
Write-Host "System info:" 
dotnet --info | Select-String ".NET" -Context 0,4
Write-Host "****************"
Write-Host "QDK info:"
dotnet new -u | Select-String Quantum -Context 0,4
time-hello "Q#" "qs-hello"
time-hello "C#" "cs-hello"

gives me

****************
System info:

> .NET SDK (reflecting any global.json):
   Version:   6.0.110
   Commit:    ce0a42998a

  Runtime Environment:
>  Base Path:   C:\Program Files\dotnet\sdk\6.0.110\

  global.json file:
    Not found

> .NET SDKs installed:
>   3.1.424 [C:\Program Files\dotnet\sdk]
>   6.0.110 [C:\Program Files\dotnet\sdk]

> .NET runtimes installed:
>   Microsoft.AspNetCore.App 3.1.30 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
>   Microsoft.AspNetCore.App 6.0.10 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
>   Microsoft.NETCore.App 3.1.30 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
>   Microsoft.NETCore.App 6.0.10 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
>   Microsoft.WindowsDesktop.App 3.1.30 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
>   Microsoft.WindowsDesktop.App 6.0.10 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

> Download .NET:
>   https://aka.ms/dotnet-download

> Learn about .NET Runtimes and SDKs:
>   https://aka.ms/dotnet/runtimes-sdk-info
****************
QDK info:
>    Microsoft.Quantum.ProjectTemplates
        Version: 0.26.233415
        Details:
           Author: Microsoft
           NuGetSource: https://api.nuget.org/v3/index.json
>          Quantum Application Honeywell (azq-honeywell) Q#
>          Quantum Application IonQ (azq-ionq) Q#
           Class library (classlib) Q#
           xUnit Test Project (xunit) Q#
        Uninstall Command:
>          dotnet new --uninstall Microsoft.Quantum.ProjectTemplates

****************
Creating hello-world console project: 'dotnet new console -n qs-hello -lang Q#
Running 'dotnet restore' for Q# console project
Measuring 'dotnet build --no-restore' for Q# console project
TotalSeconds      : 7.0547765
Deleting Q# project directory

****************
Creating hello-world console project: 'dotnet new console -n cs-hello -lang C#
Running 'dotnet restore' for C# console project
Measuring 'dotnet build --no-restore' for C# console project
TotalSeconds      : 2.0348118
Deleting C# project directory

microsoft / qsharp-compiler