ReplicaSet, Insert - Githubissues

0xmilk commented 9 years ago

I try to make an insert with Mongo.jl and following uri. With simple uri (someurl:27015) there is no issue. If the uri includes ReplicaSet it stuck.

mongodb://<user>:<password>@c47.cockney.0.mongolayer.com:10047,c47.cockney.1.mongolayer.com:10047/shopco_ml?replicaSet=set-542aa5fee8270e9654005f6a

I use this code:

mongoUser = ... # user 
mongoPass= ... # password
serverUrlBest = "mongodb://$(mongoUser):$(mongoPass)@c47.cockney.0.mongolayer.com:10047,c47.cockney.1.mongolayer.com:10047/shopco_ml?replicaSet=set-542aa5fee8270e9654005f6a"

client = MongoClient(serverUrlBest)

# Get a handle to collection named "try_coll" in database "try_db"
collection = MongoCollection(client, "try_db", "try_coll")

2015/02/20 14:59:32.0173: [36443]:    DEBUG:      cluster: Client initialized in replica set mode.

bsonObject = BSONObject({"ping" => "pong"})
# Insert a document
insert(collection, bsonObject)

2015/02/20 14:59:32.0739: [36443]:    DEBUG:      cluster: Reconnecting to replica set.
2015/02/20 14:59:32.0861: [36443]:    DEBUG:      cluster: Registering potential peer: c47.cockney.0.mongolayer.com:10047
2015/02/20 14:59:32.0861: [36443]:    DEBUG:      cluster: Registering potential peer: c47.cockney.1.mongolayer.com:10047
2015/02/20 14:59:32.0861: [36443]:     INFO:      cluster: c47.cockney.0.mongolayer.com:10047: Got replicaSet "set-542aa5fee8270e4354005f6a" expected "set-542aa5fee8270e4354005f6a:27017/".
2015/02/20 14:59:32.0976: [36443]:    DEBUG:      cluster: Registering potential peer: c47.cockney.1.mongolayer.com:10047
2015/02/20 14:59:32.0976: [36443]:    DEBUG:      cluster: Registering potential peer: c47.cockney.0.mongolayer.com:10047
2015/02/20 14:59:32.0976: [36443]:     INFO:      cluster: c47.cockney.1.mongolayer.com:10047: Got replicaSet "set-542aa5fee8270e4354005f6a" expected "set-542aa5fee8270e4354005f6a:27017/".
2015/02/20 14:59:33.0061: [36443]:    DEBUG:      cluster: Registering potential peer: c47.cockney.0.mongolayer.com:10047
2015/02/20 14:59:33.0061: [36443]:    DEBUG:      cluster: Registering potential peer: c47.cockney.1.mongolayer.com:10047
2015/02/20 14:59:33.0061: [36443]:     INFO:      cluster: c47.cockney.0.mongolayer.com:10047: Got replicaSet "set-542aa5fee8270e4354005f6a" expected "set-542aa5fee8270e4354005f6a:27017/".
insert: No acceptable peer could be found.
while loading In[4], in expression starting on line 3

 in error at error.jl:21
 in insert at /Users/amr5hgku30unjdfk44e3/.julia/v0.3/Mongo/src/MongoCollection.jl:45

ghost commented 9 years ago

Strange. What happens if you add :27017/ to the end of the replicaSet=set-542aa5fee8270e9654005f6a part of the URI, ie. change the URI to:

serverUrlBest = "mongodb://$(mongoUser):$(mongoPass)@c47.cockney.0.mongolayer.com:10047,c47.cockney.1.mongolayer.com:10047/shopco_ml?replicaSet=set-542aa5fee8270e9654005f6a:27017/"

?

0xmilk commented 9 years ago

Here the results with :27017

2015/02/20 15:57:09.0203: [36443]:    DEBUG:      cluster: Reconnecting to replica set.
2015/02/20 15:57:09.0288: [36443]:    DEBUG:      cluster: Registering potential peer: c47.cockney.0.mongolayer.com:10047
2015/02/20 15:57:09.0288: [36443]:    DEBUG:      cluster: Registering potential peer: c47.cockney.1.mongolayer.com:10047
2015/02/20 15:57:09.0288: [36443]:     INFO:      cluster: c47.cockney.0.mongolayer.com:10047: Got replicaSet "set-542aa5fee8270e9654005f6a" expected "set-542aa5fee8270e9654005f6a:27017 :27017/".
2015/02/20 15:57:09.0380: [36443]:    DEBUG:      cluster: Registering potential peer: c47.cockney.1.mongolayer.com:10047
2015/02/20 15:57:09.0380: [36443]:    DEBUG:      cluster: Registering potential peer: c47.cockney.0.mongolayer.com:10047
2015/02/20 15:57:09.0380: [36443]:     INFO:      cluster: c47.cockney.1.mongolayer.com:10047: Got replicaSet "set-542aa5fee8270e9654005f6a" expected "set-542aa5fee8270e9654005f6a:27017 :27017/".
2015/02/20 15:57:09.0474: [36443]:    DEBUG:      cluster: Registering potential peer: c47.cockney.0.mongolayer.com:10047
2015/02/20 15:57:09.0474: [36443]:    DEBUG:      cluster: Registering potential peer: c47.cockney.1.mongolayer.com:10047
2015/02/20 15:57:09.0474: [36443]:     INFO:      cluster: c47.cockney.0.mongolayer.com:10047: Got replicaSet "set-542aa5fee8270e9654005f6a" expected "set-542aa5fee8270e9654005f6a:27017 :27017/".
insert: No acceptable peer could be found.
while loading In[20], in expression starting on line 4

 in error at error.jl:21
 in insert at /Users/amr5hgku30unjdfk44e3/.julia/v0.3/Mongo/src/MongoCollection.jl:45

The same with :27017/

2015/02/20 15:57:20.0032: [36443]:    DEBUG:      cluster: Reconnecting to replica set.
2015/02/20 15:57:20.0218: [36443]:    DEBUG:      cluster: Registering potential peer: c47.cockney.0.mongolayer.com:10047
2015/02/20 15:57:20.0218: [36443]:    DEBUG:      cluster: Registering potential peer: c47.cockney.1.mongolayer.com:10047
2015/02/20 15:57:20.0218: [36443]:     INFO:      cluster: c47.cockney.0.mongolayer.com:10047: Got replicaSet "set-542aa5fee8270e9654005f6a" expected "set-542aa5fee8270e9654005f6a:27017/ :27017/".
2015/02/20 15:57:20.0312: [36443]:    DEBUG:      cluster: Registering potential peer: c47.cockney.1.mongolayer.com:10047
2015/02/20 15:57:20.0312: [36443]:    DEBUG:      cluster: Registering potential peer: c47.cockney.0.mongolayer.com:10047
2015/02/20 15:57:20.0312: [36443]:     INFO:      cluster: c47.cockney.1.mongolayer.com:10047: Got replicaSet "set-542aa5fee8270e9654005f6a" expected "set-542aa5fee8270e9654005f6a:27017/ :27017/".
2015/02/20 15:57:20.0408: [36443]:    DEBUG:      cluster: Registering potential peer: c47.cockney.0.mongolayer.com:10047
2015/02/20 15:57:20.0408: [36443]:    DEBUG:      cluster: Registering potential peer: c47.cockney.1.mongolayer.com:10047
2015/02/20 15:57:20.0408: [36443]:     INFO:      cluster: c47.cockney.0.mongolayer.com:10047: Got replicaSet "set-542aa5fee8270e9654005f6a" expected "set-542aa5fee8270e9654005f6a:27017/ :27017/".
insert: No acceptable peer could be found.
while loading In[22], in expression starting on line 4

 in error at error.jl:21
 in insert at /Users/milk_/.julia/v0.3/Mongo/src/MongoCollection.jl:45

ghost commented 9 years ago

Hmmm, I'm not sure. Does the URI work from the standard MongoDB command-line client?

For the record, I don't know much about MongoDB replication. But, the URIs are passed verbatim to the libmongoc library and from what I can tell from the documentation you should not need to do anything special with the library to use replication, so it's possibly just a matter of formatting the URI correctly.

0xmilk commented 9 years ago

Right, it's the C driver itself!

#include <bcon.h>
#include <bson.h>
#include <mongoc.h>
#include <stdio.h>
#include <stdlib.h>

int
main (int argc,
      char *argv[])
{
   mongoc_collection_t *collection;
   mongoc_client_t *client;
   mongoc_cursor_t *cursor;
   const bson_t *item;
   bson_error_t error;
   bson_oid_t oid;
   bson_t *query;
   bson_t *doc;
   char *str;
   bool r;

   mongoc_init ();

   /* get a handle to our collection */
   client = mongoc_client_new ("mongodb://<user>:<password>@c47.cockney.0.mongolayer.com:10047,c47.cockney.1.mongolayer.com:10047/shopco_ml?replicaSet=set-542aa5fee8270e9654005f6a:27017/");
   collection = mongoc_client_get_collection (client, "shopco_ml", "test");

   /* insert a document */
   bson_oid_init (&oid, NULL);
   doc = BCON_NEW ("_id", BCON_OID (&oid),
                   "hello", BCON_UTF8 ("world!"));
   r = mongoc_collection_insert (collection, MONGOC_INSERT_NONE, doc, NULL, &error);
   if (!r) {
      fprintf (stderr, "%s\n", error.message);
      return EXIT_FAILURE;
   }

   /* build a query to execute */
   query = BCON_NEW ("_id", BCON_OID (&oid));

   /* execute the query and iterate the results */
   cursor = mongoc_collection_find (collection, MONGOC_QUERY_NONE, 0, 0, 0, query, NULL, NULL);
   while (mongoc_cursor_next (cursor, &item)) {
      str = bson_as_json (item, NULL);
      printf ("%s\n", str);
      bson_free (str);
   }

   /* release everything */
   mongoc_cursor_destroy (cursor);
   mongoc_collection_destroy (collection);
   mongoc_client_destroy (client);
   bson_destroy (query);
   bson_destroy (doc);

   return 0;
}

2015/02/20 16:06:46.0053: [36777]:    DEBUG:      cluster: Client initialized in replica set mode.
2015/02/20 16:06:46.0055: [36777]:    DEBUG:      cluster: Reconnecting to replica set.
2015/02/20 16:06:46.0140: [36777]:    DEBUG:      cluster: Registering potential peer: c47.cockney.0.mongolayer.com:10047
2015/02/20 16:06:46.0140: [36777]:    DEBUG:      cluster: Registering potential peer: c47.cockney.1.mongolayer.com:10047
2015/02/20 16:06:46.0140: [36777]:     INFO:      cluster: c47.cockney.0.mongolayer.com:10047: Got replicaSet "set-542aa5fee8270e9654005f6a" expected "set-542aa5fee8270e9654005f6a:27017/".
2015/02/20 16:06:46.0278: [36777]:    DEBUG:      cluster: Registering potential peer: c47.cockney.1.mongolayer.com:10047
2015/02/20 16:06:46.0278: [36777]:    DEBUG:      cluster: Registering potential peer: c47.cockney.0.mongolayer.com:10047
2015/02/20 16:06:46.0278: [36777]:     INFO:      cluster: c47.cockney.1.mongolayer.com:10047: Got replicaSet "set-542aa5fee8270e9654005f6a" expected "set-542aa5fee8270e9654005f6a:27017/".
2015/02/20 16:06:46.0376: [36777]:    DEBUG:      cluster: Registering potential peer: c47.cockney.0.mongolayer.com:10047
2015/02/20 16:06:46.0376: [36777]:    DEBUG:      cluster: Registering potential peer: c47.cockney.1.mongolayer.com:10047
2015/02/20 16:06:46.0376: [36777]:     INFO:      cluster: c47.cockney.0.mongolayer.com:10047: Got replicaSet "set-542aa5fee8270e9654005f6a" expected "set-542aa5fee8270e9654005f6a:27017/".
No acceptable peer could be found.

0xmilk commented 9 years ago

Thank you Peter! :)

ghost commented 9 years ago

OK great! If there is anything I need to be doing differently on the Julia side let me know; otherwise, I will close this issue for now.

0xmilk commented 9 years ago

There is no issue with the format of the uri or the C-driver. The C works with the uri, but it doesn't if you add the :27017 at the end. So I did, after you asked what happen if I do and forgot to remove it. My mistake: Copy Pasted the uri with the 'wrong' :27017 The same what between the C-driver and the ccall happen. It seem to be a problem in the MongoClient.jl. It wraps or add the :27017 at the end to the uri.

Following code works:

#include <bcon.h>
#include <bson.h>
#include <mongoc.h>
#include <stdio.h>
#include <stdlib.h>

int
main (int argc,
      char *argv[])
{
   mongoc_collection_t *collection;
   mongoc_client_t *client;
   mongoc_cursor_t *cursor;
   const bson_t *item;
   bson_error_t error;
   bson_oid_t oid;
   bson_t *query;
   bson_t *doc;
   char *str;
   bool r;

   mongoc_init ();

   /* get a handle to our collection */
   client = mongoc_client_new ("mongodb://<user>:<password>@c47.cockney.0.mongolayer.com:10047,c47.cockney.1.mongolayer.com:10047/shopco_ml?replicaSet=set-542aa5fee8270e9654005f6a");
   collection = mongoc_client_get_collection (client, "shopco_ml", "test");

   /* insert a document */
   bson_oid_init (&oid, NULL);
   doc = BCON_NEW ("_id", BCON_OID (&oid),
                   "hello", BCON_UTF8 ("world from c!"));
   r = mongoc_collection_insert (collection, MONGOC_INSERT_NONE, doc, NULL, &error);
   if (!r) {
      fprintf (stderr, "%s\n", error.message);
      return EXIT_FAILURE;
   }

   /* build a query to execute */
   query = BCON_NEW ("_id", BCON_OID (&oid));

   /* execute the query and iterate the results */
   cursor = mongoc_collection_find (collection, MONGOC_QUERY_NONE, 0, 0, 0, query, NULL, NULL);
   while (mongoc_cursor_next (cursor, &item)) {
      str = bson_as_json (item, NULL);
      printf ("%s\n", str);
      bson_free (str);
   }

   /* release everything */
   mongoc_cursor_destroy (cursor);
   mongoc_collection_destroy (collection);
   mongoc_client_destroy (client);
   bson_destroy (query);
   bson_destroy (doc);

   return 0;
}

ghost commented 9 years ago

OK thanks, I will try to take a look this weekend to figure out what is going on!

0xmilk commented 9 years ago

http://pkg.julialang.org links to version 0.1.0 - Pkg.add("Mongo").

With your master branch Pkg.checkout("Mongo","master") everything works.

This MongoClient is used within Pkg.add("Mongo")

  MongoClient(host::String = "localhost", port::Int = 27017) = begin
        uri = "mongodb://$host:$port/"
        uriCStr = bytestring(uri)
        client = new(
            uri,
            ccall(
                (:mongoc_client_new, libmongoc),
                Ptr{Void}, (Ptr{Uint8}, ),
                uriCStr
                )
            )
        finalizer(client, destroy)
        return client
    end

ghost commented 9 years ago

Ah OK, I will update the pkg version. That’s easy!

On Feb 23, 2015, at 6:00 AM, 0xmilk notifications@github.com wrote:

http://pkg.julialang.org http://pkg.julialang.org/ links to version 0.1.0 - Pkg.add("Mongo").

With your master branch Pkg.checkout("Mongo","master") everything works.

MongoClient(host::String = "localhost", port::Int = 27017) = begin uri = "mongodb://$host:$port/" uriCStr = bytestring(uri) client = new( uri, ccall( (:mongoc_client_new, libmongoc), Ptr{Void}, (Ptr{Uint8}, ), uriCStr ) ) finalizer(client, destroy) return client end — Reply to this email directly or view it on GitHub https://github.com/pzion/Mongo.jl/issues/2#issuecomment-75523052.

ghost commented 9 years ago

Just an FYI that what is in master is now the official version of Mongo.jl. Thanks for your help tracking this down!

ScottPJones / Mongo.jl

ReplicaSet, Insert #2