yesodweb / persistent

Persistence interface for Haskell allowing multiple storage methods.
MIT License
467 stars 296 forks source link

MongoDB: get blocks forever? #207

Closed alrunner4 closed 9 years ago

alrunner4 commented 10 years ago

In my Yesod application, I've got the following implementation for YesodAuth's onLogin:

onLogin = do
    mAuth <- maybeAuthId
    case mAuth of
        Nothing -> error "logged in, but no AuthId!"
        Just authId -> do
            maybeUserId <- runDB $ selectKeys [
                Authenticators `multiEq` authId ] [] $$ await
            case maybeUserId of
                Nothing -> liftIO $ putStrLn "a new user needs to be created"
                Just userid -> do
                    setSession "userid" ( toPathPiece userid )
                    liftIO $ putStrLn $ "about to lookup user " ++ show userid
                    mUser <- runDB $ get userid
                    liftIO $ putStrLn $ "user logged in = " ++ show mUser

The relevant model in this case being:

User
    displayName      Text
    authenticators   [AppAuth]
    deriving         Read Show

And I'm using type AppAuth = Text since persistFileWith doesn't like it when I try to use AuthId App, which is also Text.

I'm not 100% sure this is a bug, but the selectKeys query does work when my MongoDB database is populated with some test data and I login with Google's OpenAuth (authGoogleEmail), but even though I know with 100% certainty that userid is a valid Key in my database (it just came back from the selectKeys query, after all), the get always blocks indefinitely. The last line of log the server generates is: about to lookup user Key {unKey = PersistObjectId "R\232\SO@\ESC\237\ESC9\192\&0_\228"} confirming that we're somewhere in runDB when the handler hangs.

My runDB is nothing special, either:

runDB action = do
    App{..} <- getYesod
    runMongoDBPoolDef action dbConnectionPool

Any ideas? Am I doing something stupid? persistent-mongoDB does do get, right? My mongod is version 2.4.9.

Thanks, Alex

alrunner4 commented 10 years ago

Okay. Playing around some more with this, I tried wiping my User collection and starting fresh: I think the problem may have been in the marshalling from the MongoDB document to my User data type. Is it expected behavior for queries to block indefinitely if a MongoDB document is malformed? I noticed that the User document I had been loading was missing a (required) field I added to the model after it was created, and deleting that field from other documents in the database causes the problem to occur. Documents matching the Yesod model schema load correctly without blocking.

gregwebs commented 10 years ago

expected behavior is an exception, and that is what I have always seen. The MongoDB driver is slow, but I have never seen a query block forever. We can add a test case to persistent that this should throw an exception. Perhaps there is an issue with the particular fields of this model. Can you see if this is reproducible for any instance of a missing field or is it just for the particular fields you are dealing with.

alrunner4 commented 10 years ago

Is it possible that the exception is being thrown but not caught by Yesod for some reason? I had also considered the possibility of an exception, but my browser never got a response like it does with error exceptions, instead just spinning the waiting-for-response indicator forever. I'll add a catchall to see if it's happening.

What kind of cases are you thinking of for "any instance of a missing field?" As in including Maybe fields? Or fields (not) involved in the query? Or fields of a particular type? Some examples should help me get the coverage you're looking for.

gregwebs commented 10 years ago

I didn't have a specific thought on the field type, just that perhaps your user field is for example a Maybe Int and there is a specific problem with that which we can test in isolation.

You can try using the docToEntityEither function and the docToEntityThrow function and see what happens.

gregwebs commented 10 years ago

are you on windows using yesod devel?

alrunner4 commented 10 years ago

Windows, yes, but I'm not using yesod devel. I didn't start from the scaffolding, and though I converted the cabal build to produce both library and program in an attempt to get yesod devel working, I never got it all the way there.

I've been busy with other things since I discovered that I just needed to correct some of my MongoDB documents, but hopefully I'll have some time this weekend to produce a few minimal test cases.

gregwebs commented 10 years ago

Ok, so it seems like there is a Windows specific issue with MongoDB hanging.

gregwebs commented 10 years ago

Perhaps this was an issue with how exceptions are propogated up to runDB or with a pure exception popping up at the wrong time. This behavior may change a little in persistent2, can you try against persistent2?

snoyberg commented 9 years ago

Closing out old issues, please reopen if still relevant