Track appname in MongoClient

[ncclementi]

It'll be useful for the Atlas folks to see how many people are connecting to Atlas Clusters via our dask-mongo connector. To do that we need to include the appname when creating the client.

I believe it would look something like

app_info = f"dask-mongo/{dask_mongo.__version__}"

with pymongo.MongoClient(**connection_kwargs, appname=app_info) as mongo_client:
       #do things

We should include this in write_mongo, fetch_mongo, and read_mongo in https://github.com/coiled/dask-mongo/blob/main/dask_mongo/core.py

For reference, read appname in https://pymongo.readthedocs.io/en/stable/api/pymongo/mongo_client.html

cc: @jrbourbeau @rrpelgrim @baf509

[jrbourbeau]

Sounds reasonable to me. Are there restrictions on what characters can be in appname? Is app_info = f"dask-mongo/{dask_mongo.__version__}" a valid name?

[ncclementi]

It looks like appname takes strings, according to the docs.

appname: (string or None) The name of the application that created this MongoClient instance. The server will log this value upon establishing each connection. It is also recorded in the slow query log and profile collections.

[ShaneHarvey]

Are there restrictions on what characters can be in appname?

No restrictions, f"dask-mongo/{dask_mongo.__version__}" makes sense to me.

[jrbourbeau]

Thanks for clarifying @ShaneHarvey. Are you able to confirm if the appname added in https://github.com/coiled/dask-mongo/pull/19 is showing up on the mongo side?

[ShaneHarvey]

You can verify by inspecting the server logs. The server will log the appname on each new connection like this:

{"t":{"$date":"2022-05-24T18:57:35.254-07:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"127.0.0.1:49719","uuid":"6182ae63-66e5-40c6-8284-511b3363fd94","connectionId":27,"connectionCount":6}}
{"t":{"$date":"2022-05-24T18:57:35.255-07:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn27","msg":"client metadata","attr":{"remote":"127.0.0.1:49719","client":"conn27","doc":{"driver":{"name":"PyMongo","version":"4.2.0.dev1"},"os":{"type":"Darwin","name":"Darwin","architecture":"x86_64","version":"10.16"},"platform":"CPython 3.7.9.final.0","application":{"name":"dask-mongo/1.1.1"}}}}

[ncclementi]

@ShaneHarvey Can you tell me more about how can I access the logs? I'm currently using the M0 free tier instances, and I can't seem to find the logs on Atlas. Is there a way to access the logs through some pymongo command?

[baf509]

I don't think you can download the logs for the M0, at M10 or higher there should be a button under the "Three Dot Menu" on the Cluster card which says "Download Logs".

I'm guessing Shane was using an M10 or maybe just doing the testing with a local MongoDB where you can access the logs locally.

On Wed, May 25, 2022 at 10:04 AM Naty Clementi @.***> wrote:

@ShaneHarvey https://github.com/ShaneHarvey Can you tell me more about how can I access the logs? I'm currently using the M0 free tier instances, and I can't seem to find the logs on Atlas. Is there a way to access the logs through some pymongo command?

— Reply to this email directly, view it on GitHub https://github.com/coiled/dask-mongo/issues/18#issuecomment-1137289860, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAXQ7NBAJLKLQBXKAVPVAHDVLYXPFANCNFSM5W2NK6OQ . You are receiving this because you were mentioned.Message ID: @.***>

-- Benjamin Flast 908-249-3044

[ncclementi]

Thanks @baf509 I was able to test it locally by printing the mongo_client.server_info().

{"t":{"$date":"2022-05-25T12:08:24.475-04:00"},"s":"I",  "c":"COMMAND",  "id":51803,   "ctx":"conn11","msg":"Slow query","attr":{"type":"command","ns":"test-db.test-collection",
"appName":"dask-mongo/2021.7.3","command":{"insert":"test-collection","ordered":true,"lsid":{"id":{"$uuid":"667d444d-3034-445d-bbf4-f812a75b6c56"}},"$db":"test-db"},"ninserted":2,"keysInserted":2,"numYields":0,"reslen":45,"locks":{"ParallelBatchWriterMode":{"acquireCount":{"r":4}},"ReplicationStateTransition":{"acquireCount":{"w":4}},"Global":{"acquireCount":{"r":1,"w":3}},"Database":{"acquireCount":{"r":1,"w":3}},"Collection":{"acquireCount":{"r":1,"w":3}},"Mutex":
{"acquireCount":{"r":5}}},"flowControl":{"acquireCount":3,"timeAcquiringMicros":10},"storage":{},"remote":"127.0.0.1:53735","protocol":"op_msg","durationMillis":252}}

I will go ahead and do a release, pypi should be fast, we will have to wait a bit for conda-forge until the bots pick it up.

@jrbourbeau Do I have a green light to release?

[jrbourbeau]

Releasing sounds great 👍

[ncclementi]

@baf509 The new release of dask-mongo that includes the tracking of the appname is up on PyPI and should be in conda-forge in 1h or so, we've just merged the conda-forge feedstock PR.

[bafMdb]

That's great, thanks @ncclementi!