Testing MongoDB xojo driver

[ghost]

I am interested in your MongoDB xojo driver project. Building up a simple test application, I am raising a NilObjectException on your sample code 3rd statement.

The first one - client.GetDB('test') - seems to be working fine as it has an 'Address', its 'isConnected' flag was set to True and 'LastErrorCode' to 0.

Its port was set wrong to me: 49701 - mongodb default port being 27017.

On next statement - db.GetCollection("data"), nothing is happening: the coll object remains Nil - therefore the 3rd statement is raising a NilObjectException...

I am using mongodb 2.4.8 (from macports.org) on Mavericks and xojo 2013 R 3.2.

This MongoDB server is working fine from mongo shell or nodejs code: this is obviously a problem with your xojo driver.

I am interested in your project and keen to help you if I could...

[alwyn1024]

The collection "data" probably doesn't exist in your MongoDB? The driver deliberately returns a Nil object with db.GetCollection() if the collection doesn't exist.

I'll update the driver to implicitly create the collection if it doesn't exist yet. Thanks for pointing this out.

Would you like to become a Collaborator?

[ghost]

OK - I went a step forward to find out db.GetCollection("data") returns Nil because there is no "data" collection in "test" database.

This raises a few issues:

• In MongoDB when you start using a database or a database collection that doesn't exist you never need to create them. In this xojo implementation you'll crash if they didn't exist yet. Does it mean we had to go to mongo shell first in order to create them adding some dummy document in a collection?

After doing this I could insert documents quite nicely from xojo - this is very nice:

db.data.find() { "_id" : ObjectId("527c920ac7bc14aa935bcd31"), "test" : 0 } { "_id" : ObjectId("527c9275f5e8fddacaeb8d68"), "mydoc" : 123 } { "_id" : ObjectId("527c9275f5e8fddacaeb8d69"), "a" : 1 }

• At least the driver should exit cleanly if the requested resource didn't exist yet on this server
• At best it would behave like mongo does and creates the required resources on the fly if they didn't exists yet

I'll be glad to help - just let me know what you need.

[ghost]

OOPs - I didn't see your reply before posting my 2nd comment. I could certainly share time on this project: I'll be glad to become a "Collaborator". Feel free to enroll me...

[alwyn1024]

Ok, it turned out your idea improved the code a lot. I was able to replace two blocks of code with two single lines of code. Database and Collections are now implicitly created if they don't already exist.

I think the most important thing that I need help with at this point is testing, testing and more testing, by simply using the driver and discovering any teething problems there might be.

I'm also very new to MongoDB so advice on how the driver should operate (as you described with this issue) will be a massive help to get the driver to a full spec implementation.

You're now added as a collaborator to the mongodb-xojodriver project.

[alwyn1024]

The port will be different once you've connected. The driver connects to the server on port 27017, and the server then accepts the connection on a differnt port (e.g. 49701), since it needs to keep port 27017 open for other clients.

On 2013/11/08 08:15 AM, nodeleaf wrote:

Its port was set wrong to me: 49701 - mongodb default port being 27017.

[ghost]

I confirm the missing collection was created on the fly after I dropped the existing one: this is probably the fastest fix I ever seen in a 30 years long carreer as as developer: weel done...

What I could do to help at this stage is to build a new driver test xojo application where one could set db and collection names from a UI, define the document she wants to insert, setup update and remove queries - and display the resulting output into a listbox.

If you’ll agree with that I should be able to push this new app by the end of the week-end.

[alwyn1024]

If you could develop a proper test harness it will be super valuable. I would really appreciate such a contribution a lot.

I have already created the folder /test in the root folder that contains a project named drivertest.xojo_binary_project.

Perhaps you can use this project as a starting point and expand it how you see fit. Currently it doesn't have anything in it... simply a Test button with the code in the README.md file.

Thanks for your interest Patrick. I think having a MongoDB driver for Xojo would benefit a lot of people.

On 2013/11/08 10:39 AM, Patrick Perroud wrote:

Hello,

I confirm the missing collection was created on the fly after I dropped the existing one: this is probably the fastest fix I ever seen in a 30 years long carreer as as developer: weel done...

What I could do to help at this stage is to build a new driver test xojo application where one could set db and collection names from a UI, define the document she wants to insert, setup update and remove queries - and display the resulting output into a listbox.

If you’ll agree with that I should be able to push this new app by the end of the week-end.

Best regards

Patrick Perroud

On 8 Nov 2013, at 09:18 , Alwyn Bester notifications@github.com wrote:

Ok, it turned out your idea improved the code a lot. I was able to replace two blocks of code with two single lines of code. Database and Collections are now implicitly created if they don't already suggest.

I think the most important thing that I need help with at this point is testing, testing and more testing, by simply using the driver and discovering any teething problems there might be.

I'm also very new to MongoDB so advice on how the driver should operate (as you described with this issue) will be a massive help to get the driver to a full spec implementation.

You're now added as a collaborator to the mongodb-xojodriver project.

— Reply to this email directly or view it on GitHub.

---

https://nodeleaf.net

— Reply to this email directly or view it on GitHub https://github.com/alwyn1024/mongodb-xojodriver/issues/1#issuecomment-28046863.

[ghost]

This MongoDB driver is of great interest to me at least. I'll complete your test application with proper UI etc. Its embedded driver would need an update every time you'll commit a new update indeed...

[alwyn1024]

Thanks, looking forward to seeing the test app. I'll make sure to update the embedded driver with each new commit. I'm hoping Xojo will eventually add the ability to make the modules external to the project, so that a new update isn't required each time.

[ghost]

Actually I am running into a couple of connection issues - one minor but the other one looks more critical.

I've done a small banner at window's top just to check connections - setting host, port and db values from UI but also to check a connection is possible on this machine - in case the server was dead, missing, etc.

When these values are set correctly, the driver connects just fine.

If I put a database name that doesn't exist then it connects as well - which is less good since it means a new database will be created on next insert - even if the 'new' database name was a typo...

This is a minor issue but it would be nice if we could notify end user it's going to play with a new database.

More critical is a connection failure: let's say I put 2701 in the port field - when I try to connect it hangs on the connection definitively. No timeout apparently. Since we can't expect zero mistake from end users a clean exist on connection failures seems to be a major issue. Typically - the connection failure message box is never showing up in this case since the connection thread never returns. Do you want me to fill a bug about this one?

[ghost]

Something you could try to exit on connection failure is to add an Error event in your MongoClient.

As it inherits from the TCPSocket class, you could test something like this in this event:

dim err as Integer = me.LastErrorCode

Typically - I am getting an error 22 when I try to connect with an invalid port.

In other words - when this event is called you could read this value then exit gracefully from a connection failure.

[alwyn1024]

I really like where you're heading with the test application. The banner at the top is a great idea and will make it very easy to connect to databases.

I've revised the connection code in the driver to fix the problems that you found.

A 5 second timeout is added, so if a connection cannot be established the driver no longer gets stuck in an infinite loop.

The connection routine now also checks the LastErrorCode property and immediately exits the connection loop if the TCPSocket reports an error in this property.

In the future, please feel free to file as many bug reports as needed. If each issue is logged as a separate issue it just makes things easier.

Same goes for new feature requests. If there is any specific feature that you need in the driver, please log a report for it.

If all the issues and feature requests are logged as separate tickets, it is easier to keep track of what needs to be done, and we can prioritize issues and feature requests according to importance.

By the way, thanks for your help. The driver already improved a lot with all your feedback.

[ghost]

Excellent - look at this below - no more blocking connection errors:

I believe error #22 is a posix error: it means a bad parameter was passed to a method. Not very useful in this case but it won't hurt to report such errors in a test application like this one...

[ghost]

The connection routine now also checks the LastErrorCode property and immediately exits the connection loop if the TCPSocket reports an error in this property.

I didn't find this in the last driver I pulled this morning: is this done or something you will do?

[alwyn1024]

The test interface looks good.

The LastErrorCode check is already implemented in the private MongoClient.connectMongo() method:

start = Microseconds do Poll() App.DoEvents() loop until IsConnected or ((Microseconds - start) > 5000 * 1000) or (LastErrorCode <> 0)

The last statement "(LastErrorCode <> 0)" ensures that the connection loop will exit if LastErrorCode contains an error code.