add mongo smart cell

[Rovel]

Hi I'm trying to add a mongo db smart cell but I have a few questions:

• How can I test it on livebook locally like #31 did in that screenshot? if I just add in a livebook sketch the kinodb from my repo/branch it works or I need to build a kino bundled with my kinodb repo/branch and then build livebook with it?

• I thought that would be good to use the form/config from PG and MSQL since they follow the same structure is there any issue around it since mongo is a a NOSQL?

• I would love to help build more docs around adding connectors and testing it since it could be a good starting point for future contributions, is there any around that I didn't find?

[josevalim]

How can I test it on livebook locally like https://github.com/livebook-dev/kino_db/pull/31 did in that screenshot?

Yes, add your fork of kino_db as a git dependency in your notebook!

I thought that would be good to use the form/config from PG and MSQL since they follow the same structure is there any issue around it since mongo is a a NOSQL?

It should be fine as long as you can connect to Mongo. However, notice one of the main features of connecting to a database is to be able to emit queries. If you can't send SQL queries to Mongo, then this won't be really useful.

[Rovel]

Thanks, I found some places where people found useful to convert sql to mongo queries to integrate BI/notebooks solutions with mongodb, so think I have 2 options (if there is others please let me know), 1 - a mongodb querie smart cell 2 - sql to mongo lib translation like:

• https://www.mongodb.com/docs/manual/reference/sql-comparison/
• https://github.com/vincentrussell/sql-to-mongo-db-query-converter
• https://www.npmjs.com/package/mongo-sql
• https://www.site24x7.com/tools/sql-to-mongodb.html
• http://www.querymongo.com/

[josevalim]

Got it! In this case:

1. a mongodb queries smart cell - in this case, I suggest a kino_mongodb project that has its own connection cell and query cell, since those will require different UI prompts anyway

2. sql to mongo lib translation - this should not be part of kino_db itself but if this functionality is implemented as a separate library, then we can depend on it as an optional dependency and make it part of this project

[Rovel]

I made some progress in the branch yesterday but I got some blocks here is the resume:

• changed mongo lib (it works as a standalone livebook dep now)
• moved mongo to a exclusive form/params (url)
• added exclusive JS form (based on SQLite)

a mongodb queries smart cell - in this case, I suggest a kino_mongodb project that has its own connection cell and query cell, since those will require different UI prompts anyway

yesterday I made the smart cell db connect sort to work on mongo but I was not able to invoke Kino.start_child for the mongo lib.

I tried to read and understand the kino.start_child/1 but nothing I gave as opts to start the process made it work, some help here would be appreciated. please see last commit 5095338

{:ok, unquote(quoted_var(attrs["variable"]))} = Kino.start_child({Mongo, opts}) #target implementation
{:ok, unquote(quoted_var(attrs["variable"]))} = Mongo.start_link(url: unquote(attrs["URL"])) #working implementation

I would love some clarification on Kino.start_child/1 since Postgrex.start_link looks much the same as Mongo.start_link and I'm no Elixir expert.... 😢

Looking at kino_db if I just duplicate sql_query_cell and modify a couple of things it will probably be ok, if I do create a kino_mongodb, there will be a lot of duplicated code since it will be a copy of kino_db with mongo_stuff instead other dbs, but the handling, selection and transforming functions will be mostly the same.

sql to mongo lib translation - this should not be part of kino_db itself but if this functionality is implemented as a separate library, then we can depend on it as an optional dependency and make it part of this project

The Mongo query smart cell seems a better approach for v1 in my opinion since there is no need for all the mapping from SQL to Mongo to have a working example.

To finish I would like to ask if its ok to keep this PR open or close until I get those things sorted out since I read the hacktoberfest discussion 👀 .

[josevalim]

Mongo.start_link(opts) should be equivalent to Kino.start_child({Mongo, opts}). The only difference is that you start it as part of the Kino supervision tree.

Looking at kino_db if I just duplicate sql_query_cell and modify a couple of things it will probably be ok, if I do create a kino_mongodb, there will be a lot of duplicated code since it will be a copy of kino_db with mongo_stuff instead other dbs, but the handling, selection and transforming functions will be mostly the same.

It all depends on how to actually query Mongo without writing SQL. Are we expecting users to write JSON? If that's the case, then we can support it in the query builder by automatically swapping from SQL to JSON if a Mongo connection is given. Anything else is likely out of scope for this project.

[Rovel]

The query interface is pretty simple but nowhere I found saying clearly that its json, but is pretty like json, nested key values.

In this driver they are a map examples and in mongo db docs looks jsonish

I zipped a livebook testing this branch: mongo_smart_cell.zip

if you spin up a docker compose like this to serve a mongo instance you can connect and test file above

version: "3"
services:
  mongo:
    container_name: mongo
    image: mongo:4.0
    ports:
      - 27017:27017

by the way, can a dockerfile and docker compose with the databases like this would be helpful to testing the project? I can open another pr with those since I'm using docker to develop

[josevalim]

The query cell is configured to be SQL and perform syntax highlighting and so on. So we would need to at least change it to JSON for Mongo connections.

[Rovel]

I finally found in the docs, they are actually BSON, Mongo Document does still make sense? the elixir driver still have something looking like this: https://github.com/elixir-mongo/mongodb/blob/master/lib/mongo/query.ex

[josevalim]

Thank you!

I was reading this document and it seems that, even using JSON/BSON, querying MongoDB requires multiple programmatic APIs. For example, find, distinct, insert, etc. So it seems the best way to query MongoDB is via regular code and using a regular Elixir code cell.

So I think our two possible suggestions are:

1. Have an example notebook that shows how to connect and query using regular code cells
2. Have a SQL to Mongo library

I would suggest starting with 1. Glad to discuss adding it to KinoDB if 2 ever becomes a thing :)

[Rovel]

Have an example notebook that shows how to connect and query using regular code cells

I have something like that already done, including using the mongodb smart cell connector created in this branch, does make sens have a connector and not having smart cell for querie?

[josevalim]

Unfortunately not as part of this repo, because I would say the expectation here would be that they are all equally queryable. But the connection bits should be easy to extract out. :)