Open nh2 opened 6 years ago
My workaround is to just re-run it.
I'm using ghc-8.0.2 installed via stack.
@nh2 Thanks for reporting. I think I've seen this occasionally hitting CI, but I could never figure out why this happens. We use Shake resources to limit the number of parallel accesses to the package database, but presumably the error says that two processes collided when accessing package.conf.d/package.cache
.
By the way, googling for package.conf.d\package.cache: you don't have permission to modify this file
reveals quite a lot of similar reports outside of Hadrian. Could this be a GHC bug?
GHC 8.0 was plagued by this problem - GHC 8.2 is vastly better, but it still happens occasionally. There's definitely GHC-level races on this file. I thought we were using Shake to avoid them, but I guess not entirely.
@ndmitchell Thanks! Do you think there is a GHC ticket for this problem? I tried to find an existing ticket, but only found a closed one.
It's been closed since it got fixed in 8.2 :) Or at least it got mostly fixed, I still see it from time to time but it's a lot better at least.
I see :) Well, I'm tempted to close this issue then.
All: if you think we can somehow workaround this GHC bug in Hadrian, please reopen this issue.
Does every GHC compile take the ghc-pkg reader lock? I think it should.
@ndmitchell Yes. GHC takes a readPackageDb
resource, whereas ghc-pkg
takes writePackageDb
:
https://github.com/snowleopard/hadrian/blob/master/src/Rules.hs#L78-L81
This is not type-checked though, so we might have missed this somewhere. We could in principle make this more type-safe, but I'm not sure if this is worth it.
Given it still fails we probably did miss something somewhere.
Hmm, I'll double check, but didn't you say above that it's a GHC bug? That is it got better in 8.2 but is still a bug that you've come across?
It's long known issue in ghc-pkg that if you read/write concurrently to the package db it fails with the above error. See https://github.com/commercialhaskell/stack/issues/2617. GHC 8.2 made it a lot better, but I still don't think it's perfect. Hadrian works around it with reader/writer resources, but clearly the workaround isn't perfect either.
Can’t we fix this in ghc-pkg?
It was fixed in ghc-pkg in 8.2. However, it still occurs occasionally for me so I suspect the fix is incomplete.
OK, I've reopened this issue. Let's check that our workaround this ghc-pkg
issue covers all cases.
I would far prefer to not workaround bugs in ghc-pkg
and rather just fix them. Working around them just means they will remain hidden until some unfortunate user tries to use the compiler. @ndmitchell, can you open a ticket describing precisely what you have observed? ghc-pkg
now (#13194) locks the database prior to modification so either the our locking logic is wrong or there is an unrelated bug.
When detected it was reported, and fixed. The Hadrian workaround is required until 8.2 is the GHC boot version.
Since then I’ve had it some times, but never reproducibly or reliably, most often on a machine with aggressive anti virus. If I ever have it happen reliably I’ll ticket it.
Just now I got this interesting error on Windows (on top of ghc/ghc@a1950e6dc03b560105903ad44050b1570f3bc24f):
ls
suggests I have permissions:Maybe it's something about some
ghc-pkg
invocations running in parallel on Windows, and that's not OK?