search

fpco / stackage-curator

DEPRECATED Tools for curating Stackage package sets and building reusable package databases
MIT License
10 stars 11 forks source link

stackage-security-lock error #66

Closed UnkindPartition closed 6 years ago

UnkindPartition commented 6 years ago

Hi,

I sometimes catch a locking error when running make-bundle:

user error (hTryLock: lock already exists: /home/circleci/.stack/indices/Hackage/hackage-security-lock)

It seems clear where it's coming from — stackage-curator runs many copies of stack, stack calls to hackage-security, and hackage-security doesn't handle an already taken lock gracefully. The solution would be to add a retry somewhere in hackage-security.

I just wanted to check whether you also observe this issue or you have some workaround, or you don't run stackage-curator with many jobs? (We run with just 9.)

snoyberg commented 6 years ago

I don't think that's where the error comes from, we only update with Stack once. I believe you probably terminated the application preemptively once, and are running into a hackage-security bug around using directories for locking, see https://github.com/haskell/hackage-security/pull/203. Can you try building again with a newer hackage-security and if that works sending a PR for the stack.yaml?

snoyberg commented 6 years ago

And no, I've never observed this issue myself.

UnkindPartition commented 6 years ago

Ah, that makes sense—thanks for the info.

It didn't look to me that it happened during the update, but I could be wrong. I'll see if I can reproduce it again and provide more details.

UnkindPartition commented 6 years ago

Ok, I caught it again—it happens not during the update but during stack unpack, which is run once per package.

> /tmp/stackage-build6795$ stack unpack data-accessor-0.2.2.8@sha256:012bf967934ec049d2e74b65ea3aa51a78ec0b71a621ceec4d27a9c23a581cf6
Updating package index Hackage (mirrored at https://s3.amazonaws.com/hackage.fpcomplete.com/) ...
Selected mirror https://s3.amazonaws.com/hackage.fpcomplete.com/
Downloading timestamp
Exception user error (hTryLock: lock already exists: /home/circleci/.stack/indices/Hackage/hackage-security-lock) when using mirror https://s3.amazonaws.com/hackage.fpcomplete.com/
Selected mirror http://hackage.fpcomplete.com/
Downloading timestamp
Exception user error (hTryLock: lock already exists: /home/circleci/.stack/indices/Hackage/hackage-security-lock) when using mirror http://hackage.fpcomplete.com/
Selected mirror http://objects-us-east-1.dream.io/hackage-mirror/
Downloading timestamp
user error (hTryLock: lock already exists: /home/circleci/.stack/indices/Hackage/hackage-security-lock)
UnkindPartition commented 6 years ago

I haven't tried to update hackage-security yet, but if you think this could be related to file vs directory locking, I will.

snoyberg commented 6 years ago

I'd still like to see an attempt at an update there. Also, what version of Stack are you using? Does it include the hackage-security update?

UnkindPartition commented 6 years ago

This was with:

Version 1.9.1, Git revision f9d0042c141660e1d38f797e1d426be4a99b2a3c (6168 commits) x86_64 hpack-0.31.0

I'll update hackage-security to master.

UnkindPartition commented 6 years ago

Actually, looking at stack's stack.yaml as of the v1.9.1 tag, it should've been built with hackage-security-0.5.3.0@rev:2, which is more or less the latest hackage-security. (It already contains https://github.com/haskell/hackage-security/commit/2a1a6cc33331179ac6e131ee6783978dbcc8882c, which replaces your earlier patch.)

snoyberg commented 6 years ago

If you want to send a PR to address the problem, I'll be happy to review. But we're not running into this on the Stackage build server.

On a related tangent: we've got plans in place to totally revamp the stackage-curator tool based on the recent pantry work I did in Stack. I'm guessing that will affect you significantly, and likely make it easier in many cases to achieve what you're trying to do. For example, the new version will be based around pantry locations, which can point to arbitrary URLs for tarballs, allowing hackage.head.

Are you interested in collaborating on getting this new version operational?

UnkindPartition commented 6 years ago

If you want to send a PR to address the problem, I'll be happy to review. But we're not running into this on the Stackage build server.

Got it. I think the fix will be on the hackage-security side though.

On a related tangent: we've got plans in place to totally revamp the stackage-curator tool based on the recent pantry work I did in Stack. I'm guessing that will affect you significantly, and likely make it easier in many cases to achieve what you're trying to do. For example, the new version will be based around pantry locations, which can point to arbitrary URLs for tarballs, allowing hackage.head.

That sounds interesting, although it seems we're pretty close to what we need with the current setup. (Also, incidentally, just today I ran into a pantry issue when trying out stack head—it was https://github.com/commercialhaskell/stack/issues/4292.) Let me think about it.

How far do you think the new stackage-curator is from being operational?

snoyberg commented 6 years ago

The new curator tool is currently living in the stack repo, but it will get moved out once pantry is ready to move to its own repo. The major steps we still need are:

It's currently doing the building by shelling out to Stack, though I was recently discussing the possibility of using a different approach to building, potentially even Nix (to ensure that the stack2nix-style integration is working correctly).

UnkindPartition commented 6 years ago

I think I figured it out — stack unpack does an index update only when the index is absent (or maybe out-of-date?). So a simple stack update before running stackage-curator did the trick for me, and I bet you are already doing that. I already have a patch for hackage-security though, so I guess I'll submit it anyway.

As to your collaboration offer, I don't think I'm going to have the bandwidth for it, but I'll definitely keep it in mind.

DanBurton commented 6 years ago

So a simple stack update before running stackage-curator did the trick for me, and I bet you are already doing that.

Correct, curators typically know to do stack update before using stackage-curator. Perhaps we could document this better, or just build it into stackage-curator.