mongo adapter starts with many blank lines

[llimllib]

When I run a mongo query, my preview window looks like this:

readme@adama > 
readme@adama > 
readme@adama > 
readme@adama > 
readme@adama > 
readme@adama > 
readme@adama > 
readme@adama > 
readme@adama > 
readme@adama > 
readme@adama > 
readme@adama > 
readme@adama > 
readme@adama > 
readme@adama > 
readme@adama > 
readme@adama > 
readme@adama > 
readme@adama > 
readme@adama > [
  [ 'Awesome Fresh Chair', ObjectId("644a7fd89f2af185ac856ec1") ],
  [ 'Unbranded Metal Shoes', ObjectId("644a7ff9f797ad86ec568966") ],
  [ 'Intelligent Metal Bacon', ObjectId("644a7ffdf797ad86ec56898a") ],
  [ 'Handmade Rubber Car', ObjectId("644a8001f797ad86ec5689ae") ],
  [ 'Licensed Concrete Fish', ObjectId("644a8813178bc88ea5e99c03") ],
  [
    'Handcrafted Concrete Bacon',
    ObjectId("644a8817178bc88ea5e99c27")
  ],
  [ 'Generic Rubber Sausages', ObjectId("644a881b178bc88ea5e99c4b") ],
  [ 'bananas', ObjectId("644a88a98c6a4b4cd9e5749d") ]
]
readme@adama > 

I have mongosh installed, and running the query this way (which, as best as I can tell, matches the way the mongo adapter is running it?) doesn't show any of the readme@adama business:

$ mongosh mongodb://localhost/readme --quiet --eval 'db.projects.find().map(p => [p.name, p._id]);'
[
  [ 'Awesome Fresh Chair', ObjectId("644a7fd89f2af185ac856ec1") ],
  [ 'Unbranded Metal Shoes', ObjectId("644a7ff9f797ad86ec568966") ],
  [ 'Intelligent Metal Bacon', ObjectId("644a7ffdf797ad86ec56898a") ],
  [ 'Handmade Rubber Car', ObjectId("644a8001f797ad86ec5689ae") ],
  [ 'Licensed Concrete Fish', ObjectId("644a8813178bc88ea5e99c03") ],
  [
    'Handcrafted Concrete Bacon',
    ObjectId("644a8817178bc88ea5e99c27")
  ],
  [ 'Generic Rubber Sausages', ObjectId("644a881b178bc88ea5e99c4b") ],
  [ 'bananas', ObjectId("644a88a98c6a4b4cd9e5749d") ]
]

Is there any way to suppress the leading empty lines?

[kezhenxu94]

I got the same issue and found it due to

https://github.com/tpope/vim-dadbod/blob/fb30422b7bee7e2fa4205a4d226f01477f4fc593/autoload/db.vim#L511

it adds the same number blank lines before the real script so MongoDB responded the same lines of prompt, @tpope can you kindly share why is needed to repeat the blank lines before the actual script?

[tpope]

I thought --quiet got rid of the prompt. Is this a behavior change in mongosh compared to mongo? Is there another flag we could use?

[llimllib]

It seems like mongosh treats redirected files as interactive input, while it treats named .js files as executable (and will not echo the output by default):

# with a redirected file, mongosh processes it as interactive input, including the leading `\n\n\n` 
$ mongosh $DATABASE_URL --quiet <<< $'\n\n\ndb.projects.find().map(p => [p.name, p._id]);'
readme> 

readme> 

readme> 

readme> db.projects.find().map(p => [p.name, p._id]);
[
  [ 'testing', ObjectId('6605a863b3cee3d7239712be') ],
  [ 'testing-no-superhub', ObjectId('660af5bb1765ca4e3fb0a183') ],
  [ 'test', ObjectId('6628f802f8e230315e58b684') ]
]

# note that I had to add `console.log` here, otherwise it executes but doesn't print
$ echo $'\n\n\nconsole.log(db.projects.find().map(p => [p.name, p._id]));' > /tmp/f.js
$ mongosh $DATABASE_URL --quiet /tmp/f.js                                                 
[
  [ 'testing', ObjectId('6605a863b3cee3d7239712be') ],
  [ 'testing-no-superhub', ObjectId('660af5bb1765ca4e3fb0a183') ],
  [ 'test', ObjectId('6628f802f8e230315e58b684') ]
]

--quiet saves you the whole connection info stanza, but doesn't silence the prompt when mongo thinks it's doing interactive input. WIthout the newlines, you still get the prompt:

$ mongosh $DATABASE_URL --quiet <<< $'db.projects.find().map(p => [p.name, p._id]);'      
readme> db.projects.find().map(p => [p.name, p._id]);
[
  [ 'testing', ObjectId('6605a863b3cee3d7239712be') ],
  [ 'testing-no-superhub', ObjectId('660af5bb1765ca4e3fb0a183') ],
  [ 'test', ObjectId('6628f802f8e230315e58b684') ]
]
readme> 

If you use --eval rather than a file redirection, mongosh does not show the prompt:

$ mongosh $DATABASE_URL --quiet --eval $'\n\n\ndb.projects.find().map(p => [p.name, p._id]);'
[
  [ 'testing', ObjectId('6605a863b3cee3d7239712be') ],
  [ 'testing-no-superhub', ObjectId('660af5bb1765ca4e3fb0a183') ],
  [ 'test', ObjectId('6628f802f8e230315e58b684') ]
]

[tpope]

We can switch it to pass a filename. Please try this patch and confirm it fixes the issue:

diff --git i/autoload/db/adapter/mongodb.vim w/autoload/db/adapter/mongodb.vim
index a54fd6f..2ae9a6b 100644
--- i/autoload/db/adapter/mongodb.vim
+++ w/autoload/db/adapter/mongodb.vim
@@ -30,8 +30,8 @@ function! db#adapter#mongodb#interactive(url) abort
         \ db#url#as_argv(url, '--host ', '--port ', '', '-u ', '-p ', '')
 endfunction

-function! db#adapter#mongodb#filter(url) abort
-  return db#adapter#mongodb#interactive(a:url) + ['--quiet']
+function! db#adapter#mongodb#input(url, in) abort
+  return db#adapter#mongodb#interactive(a:url) + ['--quiet', a:in]
 endfunction

 function! db#adapter#mongodb#complete_opaque(url) abort

[llimllib]

With that change, the blank lines are gone, but then I have to add console.log(...) to all my queries to get any output - it would be a lot nicer to have the --eval behavior of printing the output automatically.

(I understand that mongo doesn't make this easy, and appreciate your time looking into it)

[tpope]

Unfortunately it may be a while before I have enough time to install MongoDB and investigate myself. What about something like --query 'eval(<read from stdin>)'? Can someone figure out the <read from stdin> part?

[llimllib]

From the shell, you can use $(cat <file>):

$ printf "\n\n\ndb.projects.find().map(p => [p.name, p._id]);" >| query.js            
$ mongosh $DATABASE_URL --quiet --eval "$(cat query.js)"                  
[
  [ 'testing', ObjectId('6605a863b3cee3d7239712be') ],
  [ 'testing-no-superhub', ObjectId('660af5bb1765ca4e3fb0a183') ],
  [ 'test', ObjectId('6628f802f8e230315e58b684') ]
]

I don't follow the execution of the process in vimscript, so I'm not sure if that's possible here?

[tpope]

We don't use the shell, but we could effectively the same thing by calling readfile() and passing it as an argument. Not a great solution, command line arguments have constraints like size limits and then you can end up with a whole dang file in ps aux.

[llimllib]

Agreed. As far as I can tell, mongosh is completely impervious to sensible stdin handling though

[llimllib]

I really wish the usual "- means read from stdin" behavior worked, but no joy:

$ cat query.js| mongosh $DATABASE_URL --quiet --eval -
SyntaxError: Unexpected token (1:1)

> 1 | -
    |  ^

I looked in their github repo, but they have issues disabled and appear to use jira so I don't know where I'd file an issue with an enhancement request

[tpope]

You could maybe abuse massage() for this. A lot of adapters use it to add trailing semicolons, but maybe wrapping with console.log() could be sufficient? I guess it wouldn't cover cases with internal semicolons.

[twiclo]

I'm probably missing context but I'm not seeing why we can't do something like this

$ echo 'printjson(db.runCommand({ ping: 1 }))' | xargs -I {} mongosh --eval {}
{
  ok: 1
}

[tpope]

I'm probably missing context but I'm not seeing why we can't do something like this

$ echo 'printjson(db.runCommand({ ping: 1 }))' | xargs -I {} mongosh --eval {}
{
  ok: 1
}

All xargs does here is pass the input as an argument. It's just mongosh --eval 'printjson(db.runCommand({ ping: 1 }))' with extra steps. For doing this in Vim, see my comment about readfile above.

[twiclo]

I think wrapping it in a printjson() is the most elegant solution:

$ cat -p tmp.js 
printjson(db.runCommand({ ping: 1 }))
$ mongosh --host localhost nm-uda --quiet ./tmp.js 
{
  ok: 1
}

A console.log would look like this

$ mongosh --host localhost nm-uda --quiet ./tmp.js 
{ ok: 1 }

From what I can tell wrapping the query in printjson() needs to happen before the query is written to a file. Do we have a way of knowing that this is a mongo query before we write it to file?

[tpope]

Change console.log to printjson and this still applies:

You could maybe abuse massage() for this. A lot of adapters use it to add trailing semicolons, but maybe wrapping with console.log() could be sufficient? I guess it wouldn't cover cases with internal semicolons.

[twiclo]

My point wasn't that printjson is better for semicolons than console.log. Just that wrapping the query in either one is the way to go. I also just tested this with a semicolon and it works:

$ cat -p tmp.js 
console.log(db.runCommand({ ping: 1 }));
$ mongosh --host localhost nm-uda --quiet ./tmp.js 
{ ok: 1 }

I think the next step is modifying the query before it's written to /tmp but I'm not sure if it's possible to know if it's a mongo type query before the query is written. We don't want to wrap a psql query in printjson

[tpope]

massage(), you want massage(), massage() is the thing you are looking for. It's per-adapter and can change the query before writing it to disk.

[twiclo]

That PR above works for me on Linux but I had a coworker on Mac give it a shot. He gets this error:

MongoshInvalidInputError: [COMMON-10001] Invalid URI: /var/folders/66/t36g34pn17x8fnjqttkxnnqw0000gn/T/nvim.spencerlundgren/9pDmj2/0

When he switched off of my fork he was able to run queries just fine. I'm assuming that's happening because the #filter was changed to an #input but I don't know how that can lead to an error like this.

Edit: My fork also stops dadbod-ui from working. Running a query produces no output. Not sure if that's something that will need to be fixed with them or the lack of a #filter function is causing issues with my fork.