Using mongeez to ensure indexes fails

[stoffer]

I'm trying to setup and use mongeez, as it seems like just the tool to use for mongodb data migration management.

And inserting data also works like a charm, but I would like to ensure indexes on certain keys in some of my collections. Mongeez adds a new document in the mogeez collection saying the changelog has been run, but now index is added to the collection.

If I run the mongo script for ensuring index directly in the mongo shell it works fine.

Any idea or comments ?

Thanks Christoffer Magnussen

[epishkin]

Can you post an example please?

On Oct 23, 2012, at 7:19 AM, Christoffer Magnussen notifications@github.com wrote:

I'm trying to setup and use mongeez, as it seems like just the tool to use for mongodb data migration management.

And inserting data also works like a charm, but I would like to ensure indexes on certain keys in some of my collections. Mongeez adds a new document in the mogeez collection saying the changelog has been run, but now index is added to the collection.

If I run the mongo script for ensuring index directly in the mongo shell it works fine.

Any idea or comments ?

Thanks Christoffer Magnussen

— Reply to this email directly or view it on GitHub.

[stoffer]

Yes. I think it is regarding mongeez usage of eval when executing scripts. But lets find out: I'm trying to apply the following index script:

<changeSet changeId="ensureOwnerIndexesMaterialCollection" author="cma" runAlways="true">
    <script>
      db.materialMetadata.ensureIndex({"_o" : 1}, {background : true });
    </script>
</changeSet>

It executes and adds a document in the mongeez collection, but no index is created.

When executing it directly in the mongo shell it seems to work:

db.materialMetadata.ensureIndex({"_o" : 1}, {background : true}); db.materialMetadata.getIndexes(); [ { "v" : 1, "key" : { "_id" : 1 }, "ns" : "metadata.materialMetadata", "name" : "id" }, { "v" : 1, "key" : { "_o" : 1 }, "ns" : "metadata.materialMetadata", "name" : "_o_1", "background" : true } ]

But as fas as I can see you are using the eval functionality of the mongoDB driver, which would end up end up in a query like this: db.materialMetadata.getIndexes(); [ { "v" : 1, "key" : { "_id" : 1 }, "ns" : "metadata.materialMetadata", "name" : "id" } ]

db.eval(function() {db.materialMetadata.ensureIndex({"_o" : 1}, {background : true})}); null db.materialMetadata.getIndexes(); [ { "v" : 1, "key" : { "_id" : 1 }, "ns" : "metadata.materialMetadata", "name" : "id" } ] So no effect either.

Removing the {background: true} will however make it work !

db.eval(function() {db.materialMetadata.ensureIndex({"_o" : 1})}); null db.materialMetadata.getIndexes(); [ { "v" : 1, "key" : { "_id" : 1 }, "ns" : "metadata.materialMetadata", "name" : "id" }, { "v" : 1, "key" : { "_o" : 1 }, "ns" : "metadata.materialMetadata", "name" : "_o_1" } ]

So I guess a workaround is not having {unique : true} or {background : true} for now, but it would be nice to have at somepoint.

Hope you can use this information ?

[epishkin]

which version of mongo driver do you use?

[stoffer]

spring mongo 1.0.2 looks like that is using the mongo-java-driver-2.7.1

[matthurne]

We have observed this issue as well. Has any progress been made in determining the cause and a potential fix? As others have described, ensuring indexes with no options succeeds, but including options such as { background: true } results in indexes not being created despite mongeez indicating that the changeset was executed successfully.

[notdang]

Happens for me too. Apparently db.eval acquires the write/read global locks on the DB and the background:true tries to re-index in background, which is not possible at this stage.

[pamelaluis]

Yes, I've observed this issue too. The problem I've found is that mongeez uses db.eval(script) which results in a global write lock and somehow this conflicts with creating indexes in the background.

The error you would usually get if you were to run the following command on the mongo shell: db.eval("db..ensureIndex({fields:1},{background:true});")

error: "can't start bg index b/c in recursive lock (db.eval?)"

A fix for this would probably be to use db.command() and passing the unlock:true option. Sample code that appears to resolve this problem is below:

       DBObject command = BasicDBObjectBuilder.start()
            .add("$eval", code)
            .add("nolock", true)
            .get();
        CommandResult retVal = db.command(command);
        retVal.throwOnError();
        Object commandReturnValue = retVal.get("retval");

        if (commandReturnValue != null && DBObject.class
            .isAssignableFrom(commandReturnValue.getClass())) {
            DBObject dbObjectRetVal = (DBObject) commandReturnValue;
            Object errorMessage = dbObjectRetVal.get("errmsg");
            if (errorMessage != null) {
                throw new CommandFailureException(retVal);
            } else {
                LOG.info("Command: " + code);
                LOG.info(commandReturnValue.toString());
            }
        }

Just calling db.eval() seems to eat up the exception with using background indexes.

[WeipingGuo]

any update on this issue? i would like to create an index on existing big data set and would like to do background indexing. thanks.