search

dotnet / source-build

A repository to track efforts to produce a source tarball of the .NET Core SDK and all its components
MIT License
265 stars 130 forks source link

Errors while building on 16.04 Xenial Xerus #95

Closed sergiusens closed 6 years ago

sergiusens commented 7 years ago

This is what I get with a fresh clone (first time I attempt a build as well) with a call to ./build.sh

"/home/sergiusens/dotnet-source-build/build.proj" (default target) (1) ->
"/home/sergiusens/dotnet-source-build/targets/repository.proj" (BuildRepositoryAndDependencies target) (2) ->
"/home/sergiusens/dotnet-source-build/targets/repository.proj" (BuildRepositoryAndDependencies target) (2:3) ->
"/home/sergiusens/dotnet-source-build/targets/repository.proj" (BuildRepositoryAndDependencies target) (2:5) ->
(Build target) -> 
  EXEC : error : Could not read profile: Unsupported format version [/home/sergiusens/dotnet-source-build/targets/repository.proj]
  /home/sergiusens/dotnet-source-build/targets/repository.proj(72,5): error MSB3073: The command "/home/sergiusens/dotnet-source-build/src/coreclr/build.sh x64 Release skiptests -PortableBuild=false " exited with code 1.

"/home/sergiusens/dotnet-source-build/build.proj" (default target) (1) ->
"/home/sergiusens/dotnet-source-build/targets/repository.proj" (BuildRepositoryAndDependencies target) (2) ->
"/home/sergiusens/dotnet-source-build/targets/repository.proj" (BuildRepositoryAndDependencies target) (2:15) ->
  /home/sergiusens/dotnet-source-build/targets/repository.proj(72,5): error MSB3073: The command "/home/sergiusens/dotnet-source-build/src/websdk/build-core.sh" exited with code 126.

    553 Warning(s)
    3 Error(s)

I was successful in the past in just building coreclr by calling ./build.sh on its own from the repo though.

omajid commented 7 years ago

Can you include more of the log? Maybe upload it somewhere? In my experience, the real cause is often hidden in the middle of thousands of lines of noise.

Also, can you building coreclr using build.sh x64 Release skiptests -PortableBuild=false? Does that lead to any errors?

sergiusens commented 7 years ago

Here's a gist for both a regular build of coreclr and one with VERBOSE=1 added to the make command.

And just in case, this is where the code is standing:

sergiusens@builder:~/dotnet-source-build/src/coreclr⟫ git log -1
commit 9679ded412d1421f3d54ee3082e8b5f900217a4a
Author: Mike McLaughlin <mikem@microsoft.com>
Date:   Wed Jun 7 07:20:55 2017 -0700

    Fix portable build sos plugin problems. (#12130)

    Removing the explicit reference to liblldb. Since the lldb program has
    already loaded this lib, our will now load regardless of the distro.

    Issue #12098.
sergiusens commented 7 years ago

I have a successful build with an unpatched coreclr on 

commit bb1c4a48f9c6d49d5c75628394b7c822f96c57be
Merge: 20de3a9 acfcd31
Author: Brian Sullivan <briansul@microsoft.com>
Date:   Tue Aug 8 18:22:34 2017 -0700

    Merge pull request #13276 from dotnet-bot/from-tfs

    Merge changes from TFS

But not with an unpatched

sergiusens@builder:~/dotnet-source-build/src/coreclr⟫ git log -1
commit 9679ded412d1421f3d54ee3082e8b5f900217a4a
Author: Mike McLaughlin <mikem@microsoft.com>
Date:   Wed Jun 7 07:20:55 2017 -0700

    Fix portable build sos plugin problems. (#12130)

    Removing the explicit reference to liblldb. Since the lldb program has
    already loaded this lib, our will now load regardless of the distro.

    Issue #12098.
sergiusens commented 7 years ago

If I take coreclr to a new commit

sergiusens@builder:~/dotnet-source-build⟫ git diff src/coreclr/
diff --git a/src/coreclr b/src/coreclr
index 9679ded..bb1c4a4 160000
--- a/src/coreclr
+++ b/src/coreclr
@@ -1 +1 @@
-Subproject commit 9679ded412d1421f3d54ee3082e8b5f900217a4a
+Subproject commit bb1c4a48f9c6d49d5c75628394b7c822f96c57be-dirty

I get a different error

crummel commented 7 years ago

Hi Sergio, For the first error you got, error: Could not read profile: Unsupported format version: Your BuildVersion-<VersionNumber>.props might be missing or malformed - I've seen this happen when there was an issue running git during the build. You could also build with /v:diag and search for CreateVersionInfoFile in the output. I think the second CMakeFiles/Makefile2:1733: recipe for target 'src/jit/standalone/CMakeFiles/clrjit.dir/all' failed error on that gist might be the same issue with some added CMake weirdness from the rebuild attempt.

The inaccessible due to its protection level errors when you update CoreCLR look like that version had some incompatible changes - that commit is a lot newer, maybe try something closer to 9679ded like 02b76a06a63221eaed1b3f7d5fd99ddd4071706b?

sergiusens commented 7 years ago

With regards to the commit being newer I was just trying to make sure my build environment was sane.

If 02b76a06a63221eaed1b3f7d5fd99ddd4071706b is the right commit to be on, let me hop on to that and build with /v:diag

sergiusens commented 7 years ago

Here's the log for the full build with coreclr on 02b76a06a63221eaed1b3f7d5fd99ddd4071706b log.txt

There is no mention of CreateVersionInfoFile in there.

sergiusens commented 7 years ago

Oh and btw, 

sergiusens@builder:~/dotnet-source-build⟫ cat src/coreclr/bin/obj/BuildVersion-20170607-01.props

produces:

<?xml version="1.0" encoding="utf-8"?>
<!-- This is a generated file.  Seed Date is 20170607-01. -->
<Project ToolsVersion="14.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <PropertyGroup>
    <VersionPropsImported>true</VersionPropsImported>
    <BuildNumberMajor Condition="'$(BuildNumberMajor)' == ''">25407</BuildNumberMajor>
    <BuildNumberMinor Condition="'$(BuildNumberMinor)' == ''">01</BuildNumberMinor>
    <LatestCommit Condition="'$(LatestCommit)' == ''">9679ded412d1421f3d54ee3082e8b5f900217a4a</LatestCommit>
    <BuiltByString Condition="'$(BuiltByString)' == ''"> built by: sergiusens-builder</BuiltByString>
  </PropertyGroup>
</Project>

My naive eyes cannot seem to find a problem.

crummel commented 7 years ago

Yeah, that looks fine to me. I'm going through your log and setting up a repro now.

crummel commented 7 years ago

There's not much useful in the log, I've got a build going now.

crummel commented 7 years ago

I got a new error: /bin/sh: 2: /tmp/tmp1d2d2cc4abda4dc1b5726983d5a750d5.exec.cmd: /home/chris/source-build/src/websdk/build-core.sh: Permission denied. Chmod'ing this and restarting the build.

sergiusens commented 7 years ago

I have a sense you got further than me as I don't have EXEC on that script either and never reached it.

sergiusens commented 7 years ago

I see this as per our conversation in some of the xmls

130 sergiusens@builder:~/dotnet-source-build/src/msbuild? git diff
diff --git a/NuGet.Config b/NuGet.Config
index 4385d98..fc4cd5f 100644
--- a/NuGet.Config
+++ b/NuGet.Config
@@ -1,4 +1,4 @@
-<?xml version="1.0" encoding="utf-8"?>
+<U+FEFF><?xml version="1.0" encoding="utf-8"?>
 <configuration>
   <packageRestore>
     <add key="enabled" value="True" />
@@ -7,6 +7,7 @@
        Should be removed if restore does not regress-->
   <packageSources>
     <clear />
+    <add key="source-built" value="/home/sergiusens/dotnet-source-build/bin/obj/x64/Release/source-built/" />
     <add key="nuget.org" value="https://api.nuget.org/v3/index.json" />
     <add key="nuget.org v2" value="https://nuget.org/api/v2/" />
     <add key="myget.org roslyn nightly" value="https://dotnet.myget.org/F/roslyn/api/v3/index.json" />
@@ -16,4 +17,4 @@
     <add key="myget.org aspnet vnext v3" value="https://www.myget.org/F/aspnetvnext/api/v3/index.json" />
     <add key="myget.org aspnet vnext" value="https://www.myget.org/F/aspnetvnext/" />
   </packageSources>
-</configuration>
+</configuration>
\ No newline at end of file
diff --git a/src/Shared/Constants.cs b/src/Shared/Constants.cs
index b58b0d4..29c8f49 100644
--- a/src/Shared/Constants.cs
+++ b/src/Shared/Constants.cs
@@ -78,7 +78,7 @@ namespace Microsoft.Build.Shared
         internal const string CurrentAssemblyVersion = Microsoft.VisualStudio.Internal.BrandNames.VSGeneralAssemblyVersion;
 #endif

-        internal const string CurrentAssemblyFileVersion = "15.3.0.0";
+        internal const string CurrentAssemblyFileVersion = "15.3.378.6360";

         /// <summary>
         /// Current version of this MSBuild Engine assembly in the form, e.g, "12.0"
crummel commented 7 years ago

Ok, the EXEC : error : Could not read profile: Unsupported format version is due to: -fprofile-instr-use=/home/chris/source-build/packages/optimization.linux-x64.pgo.coreclr/2.0.0-release-20170531-3000/data/coreclr.profdata in the compile command. Looking into this further...

sergiusens commented 7 years ago

On mié, 9 ago, 2017 at 7:46 PM, Chris Rummel notifications@github.com wrote:

Ok, the EXEC : error : Could not read profile: Unsupported format version is due to: -fprofile-instr-use=/home/chris/source-build/packages/optimization.linux-x64.pgo.coreclr/2.0.0-release-20170531-3000/data/coreclr.profdata in the compile command. Looking into this further...

Thanks for that find!

sergiusens commented 7 years ago

FWIW, I tried building coreclr at that commit with not patches applied and still get error: Could not read profile: Unsupported format version

omajid commented 7 years ago

Would it be possible to try a different version of llvm/clang? I wonder if this will fix things. I understood that PGO data is not compatible between different versions of compilers (let alone between different compilers) so I am not sure how this works, generally.

crummel commented 7 years ago

Nice call, using clang-3.9 works. Now to figure out how to get the generated Makefile to use 3.9 instead of 3.5.

sergiusens commented 7 years ago

On jue, 10 ago, 2017 at 2:27 PM, Chris Rummel notifications@github.com wrote:

Nice call, using clang-3.9 works. Now to figure out how to get the generated Makefile to use 3.9 instead of 3.5.

Seems most of the work was done in #773, we just need to add 3.9

sergiusens commented 7 years ago

I was eager to reply on that last one, its already there.

crummel commented 7 years ago

Yeah, I just added clang3.9 to https://github.com/dotnet/source-build/blob/b914e2fa16f35fe5b58fb18795872ac7c97030be/targets/coreclr.props#L5 and that seems to be working, the build is in CoreFX now

sergiusens commented 7 years ago

On jue, 10 ago, 2017 at 4:47 PM, Chris Rummel notifications@github.com wrote:

Yeah, I just added clang3.9 to https://github.com/dotnet/source-build/blob/b914e2fa16f35fe5b58fb18795872ac7c97030be/targets/coreclr.props#L5 and that seems to be working, the build is in CoreFX now

Nice, I've finally reached this now, which we saw already:

  /home/sergiusens/dotnet-source-build/targets/repository.proj(72,5): 
error MSB3073: The command 
"/home/sergiusens/dotnet-source-build/src/websdk/build-core.sh" exited 
with code 126.

By creating a patch, will link soon

sergiusens commented 7 years ago

Hm, my dumb patch breaks 14.04 builds, while I could see the logs due to permissions I did get redirected to what seems to be a 14.04 url

crummel commented 7 years ago

Yeah, that looks like the problem.

15:15:38   Commencing CoreCLR Repo build
15:15:38   Setting up directories for build
15:15:38   Checking prerequisites...
15:15:38   Please install clang-3.9 before running this script

I'm not sure that it would work without this though, that PR build hasn't passed since June 9th. I'm not sure clang 3.9 is available on 14.04 without adding a package source, I'll have to ask someone who knows about the Jenkins images to see what we can do there.

sergiusens commented 7 years ago

I don't think so either, I'll make the patch smarter and pick 3.9 as a default on 16.04 only

sergiusens commented 7 years ago

arm is failing and the PR doesn't even get close to arm as it is a different code path, can't confirm, but I am highly suspicious in being correct :stuck_out_tongue:

crummel commented 7 years ago

Yeah, that one is a very early issue with the docker image:

15:14:51 + ./arm-ci.sh arm ./build.sh /p:Platform=arm /p:Configuration=Release /p:IsJenkinsBuild=true
15:14:52 Unable to find image 'ARM_Release_prtest:/opt/code:latest' locally
15:14:52 Error response from daemon: unable to ping registry endpoint https://ARM_Release_prtest:/v0/
15:14:52 v2 ping attempt failed with error: Get https://ARM_Release_prtest:/v2/: dial tcp: lookup ARM_Release_prtest: no such host
15:14:52  v1 ping attempt failed with error: Get https://ARM_Release_prtest:/v1/_ping: dial tcp: lookup ARM_Release_prtest: no such host
sergiusens commented 7 years ago

I've also created #98 to fix that chmod issue, my local build hasn't completed to verify it works, but I think it should.

crummel commented 7 years ago

This is the fix I'm testing out for the chmod issue: https://github.com/crummel/dotnet_source-build/commit/0703b939be49c37368d170aec4c198dc8c0057f9. I think yours is changing the Windows .cmd file instead of the Linux one.

sergiusens commented 7 years ago

oh, you are indeed right! But wouldn't it be better to create the file with the right permissions instead of changing it in the build?

crummel commented 7 years ago

It would, but I think it's supposed to already going by the patch file (unless I'm misunderstanding git file modes). I'm just trying to brute-force it for now.

sergiusens commented 7 years ago

yeah, I just checked, something is reverting it during the build...

diff --git a/build-core.sh b/build-core.sh
new file mode 100755
index 0000000..4aaec8c

The 755 there is indeed what we want (octal for owner/group/others rwxr-xr-x)

omajid commented 7 years ago

I have a fix for build-core.sh and a few other fixes I needed for Fedora 26 here: https://pagure.io/fedora-dotnet/tree/f26. This is against Preview 2 version of this repo. Not sure how many patches were redundant after Preview 2.

sergiusens commented 7 years ago

Nice, well would be nice that the permission would be kept, so @crummel this will end up in the form of patches/websdk/XXXX-...patch right?

sergiusens commented 7 years ago

I updated my branch on clang, but haven't tested yet (doing that now).

crummel commented 7 years ago

If we find a websdk fix for the permissions issue that would be a patch, if we stick with just the targets file that's in the source-build repo so we can update it directly.

sergiusens commented 7 years ago

@crummel sorry again, didn't pay attention to your target fix. I'll cherry-pick that.

Byt the way, my latest update to #97 works fine on 16.04. Did the ci spit out anything different that the clang-3.9 package not being found?

crummel commented 7 years ago

Yes, now we've got:

17:07:59   mscorlib.forwards.cs(9,73): error CS0122: 'CultureAwareComparer' is inaccessible due to its protection level [/mnt/j/workspace/Private/dotnet_source-build/master/dotnet_source-build_Ubuntu14.04_Release_prtest/src/corefx/src/shims/manual/mscorlib.csproj] [/mnt/j/workspace/Private/dotnet_source-build/master/dotnet_source-build_Ubuntu14.04_Release_prtest/targets/repository.proj]
17:07:59   mscorlib.forwards.cs(10,73): error CS0122: 'OrdinalComparer' is inaccessible due to its protection level [/mnt/j/workspace/Private/dotnet_source-build/master/dotnet_source-build_Ubuntu14.04_Release_prtest/src/corefx/src/shims/manual/mscorlib.csproj] [/mnt/j/workspace/Private/dotnet_source-build/master/dotnet_source-build_Ubuntu14.04_Release_prtest/targets/repository.proj]

It looks like the versions are still mismatched. I think https://github.com/karajas/source-build/commit/c45e0f1c8398e5a54090c36980a8f91b85291c5b has all the versions set correctly but it does have an FSharp issue that Karthik is still working on, I'm trying our fixes out with those versions now.

sergiusens commented 7 years ago

I've got much further than before with the clang version change and your chmod project patch, still failing though:

Done building target "Build" in project "repository.proj".                                                     
Target "Package" skipped, due to false condition; ('$(BuildPackagesCommand)' != '') was evaluated as ('' != '').                                                                                                              
Target "CopyPackage" in project "/home/sergiusens/dotnet-source-build/targets/repository.proj" (target "BuildRepositoryAndDependencies" depends on it):                                                                       
Task "Copy"                                                                                                    
  Copying file from "/home/sergiusens/dotnet-source-build/src/cli-deps-satellites/bin/Release/CliDeps.Satellites.FSharp.4.4.1-pre-20170712-01.nupkg" to "/home/sergiusens/dotnet-source-build/bin/obj/x64/Release/source-built/CliDeps.Satellites.FSharp.4.4.1-pre-20170712-01.nupkg".                                                       
  Copying file from "/home/sergiusens/dotnet-source-build/src/cli-deps-satellites/bin/Release/CliDeps.Satellites.Roslyn.2.3.0-pre-20170712-01.nupkg" to "/home/sergiusens/dotnet-source-build/bin/obj/x64/Release/source-built/CliDeps.Satellites.Roslyn.2.3.0-pre-20170712-01.nupkg".                                                       
Done executing task "Copy".                                                                                    
Done building target "CopyPackage" in project "repository.proj".                                               
Target "WriteVersions" in project "/home/sergiusens/dotnet-source-build/targets/repository.proj" (target "BuildRepositoryAndDependencies" depends on it):                                                                     
Using "WriteVersionsFile" task from assembly "/home/sergiusens/dotnet-source-build/tasks/Microsoft.DotNet.SourceBuild.Tasks/bin/Debug/netstandard1.5/Microsoft.DotNet.SourceBuild.Tasks.dll".                                 
Task "WriteVersionsFile"                                                                                       
Done executing task "WriteVersionsFile".                                                                       
Done building target "WriteVersions" in project "repository.proj".                                             
Target "BuildRepositoryAndDependencies" in project "/home/sergiusens/dotnet-source-build/targets/repository.proj" (entry point):                                                                                              
Done building target "BuildRepositoryAndDependencies" in project "repository.proj".                            
Done Building Project "/home/sergiusens/dotnet-source-build/targets/repository.proj" (BuildRepositoryAndDependencies target(s)).                                                                                              
Done executing task "MSBuild" -- FAILED.                                                                       
Done building target "BuildRepositoryReferences" in project "repository.proj" -- FAILED.                       
Done Building Project "/home/sergiusens/dotnet-source-build/targets/repository.proj" (BuildRepositoryAndDependencies target(s)) -- FAILED.                                                                                    
Done executing task "MSBuild" -- FAILED.                                                                       
Done building target "Build" in project "build.proj" -- FAILED.                                                
Done Building Project "/home/sergiusens/dotnet-source-build/build.proj" (default targets) -- FAILED.

Going to dig through the logs for a bit and see what this failure is all about

sergiusens commented 7 years ago

Ah, this is just a side-effect of removing clang-3.5 from my system and corefx failing due to that

sergiusens commented 7 years ago

Oh, I missed that previous comment about https://github.com/karajas/source-build/commit/c45e0f1c8398e5a54090c36980a8f91b85291c5b I'll wait on that then.

crummel commented 7 years ago

https://matell.blob.core.windows.net/cli-src/dotnet-2.0.0-11.tar.gz is the tarball produced from that - it still has an F# issue but C# is working if that's enough to get you unblocked.

sergiusens commented 7 years ago

It would be a great start, yes, but given that that commit is built up on others, do you mind giving me a diff of what you ended up applying?

crummel commented 7 years ago

Here's the diff: https://gist.github.com/crummel/0b482c67ac3482e3dbd73b71f7794c21

sergiusens commented 7 years ago

Oh wait, I thought you applied changes from karajas/source-build@c45e0f1 We should be mostly good then.

sergiusens commented 7 years ago

@crummel while I wait for the build, let me ask how did you get by 

17:07:59   mscorlib.forwards.cs(9,73): error CS0122: 'CultureAwareComparer' is inaccessible due to its protection level
17:07:59   mscorlib.forwards.cs(10,73): error CS0122: 'OrdinalComparer' is inaccessible due to its protection level [/mnt/j/workspace/Private/dotnet_source-build/master/dotnet_source-build_Ubuntu14.04_Release_prtest/src/corefx/src/shims/manual/mscorlib.csproj] [/mnt/j/workspace/Private/dotnet_source-build/master/dotnet_source-build_Ubuntu14.04_Release_prtest/targets/repository.proj]

?

As of last night I had a bunch of those. Is your diff missing anything applied from karajas/source-build@c45e0f1 or did you straight out switch/checkout fixOnMattBranch?

crummel commented 7 years ago

I just checked out fixOnMattBranch (FixingFsharp should also work as they have the same versions, and F# is included in that branch but still not working). You might have to reset your submodules (git submodule sync --recursive/git submodule update --recursive) if you're still getting those mismatched errors.

sergiusens commented 7 years ago

@crummel any idea when that might get merged? Even if not perfect it should provide a better alternative that broken master, right?

crummel commented 7 years ago

I'm not sure yet, we're still hoping to have a complete fix today.

sergiusens commented 7 years ago

That is good to hear!