search

euberdeveloper / mongoback

The most powerful npm module to export and make backups of your MongoDB
https://mongoback.euber.dev
GNU General Public License v3.0
8 stars 1 forks source link

Mongoexport progressbar is hidden or printed at once on screen #6

Closed raza91 closed 3 years ago

raza91 commented 3 years ago

mongoexport progress bar is printed at once on screen 

`NODE_ENV=db_hosts node index.js
{ host: 'localhost',
  port: 27017,
  dbName: 'test',
  collection: 'user' }

FAF Query prepared {"$or":[]}
[COLLECTIONS TO EXPORT]
{
  "test": [
    "user"
  ]
}
[COMMAND] mongoexport --host=localhost --port=27017 --db=test --collection=user -vvvvv --fields="name,emails" --type=csv --out=exported/test/user.csv
[SUCCESS]
2021-01-29T18:00:28.656+0500    will listen for SIGTERM, SIGINT, and SIGKILL
2021-01-29T18:00:28.660+0500    connected to: mongodb://localhost:27017/
2021-01-29T18:00:28.706+0500    Getting estimated count for test.user
2021-01-29T18:00:29.699+0500    [........................]  test.user  120000/10000005  (1.2%)
2021-01-29T18:00:30.692+0500    [........................]  test.user  248000/10000005  (2.5%)
2021-01-29T18:00:31.702+0500    [........................]  test.user  376000/10000005  (3.8%)
2021-01-29T18:00:32.692+0500    [#.......................]  test.user  504000/10000005  (5.0%)
2021-01-29T18:00:33.692+0500    [#.......................]  test.user  632000/10000005  (6.3%)
2021-01-29T18:00:34.711+0500    [#.......................]  test.user  760000/10000005  (7.6%)
2021-01-29T18:00:35.705+0500    [##......................]  test.user  840000/10000005  (8.4%)
2021-01-29T18:00:36.702+0500    [##......................]  test.user  928000/10000005  (9.3%)
2021-01-29T18:00:37.682+0500    [##......................]  test.user  1016000/10000005  (10.2%)
2021-01-29T18:00:38.712+0500    [##......................]  test.user  1104000/10000005  (11.0%)
2021-01-29T18:00:39.689+0500    [##......................]  test.user  1184000/10000005  (11.8%)
2021-01-29T18:00:40.702+0500    [###.....................]  test.user  1280000/10000005  (12.8%)
2021-01-29T18:00:41.694+0500    [###.....................]  test.user  1368000/10000005  (13.7%)
2021-01-29T18:00:42.693+0500    [###.....................]  test.user  1424000/10000005  (14.2%)
2021-01-29T18:00:43.703+0500    [###.....................]  test.user  1504000/10000005  (15.0%)
2021-01-29T18:00:44.713+0500    [###.....................]  test.user  1584000/10000005  (15.8%)
2021-01-29T18:00:45.683+0500    [###.....................]  test.user  1640000/10000005  (16.4%)
2021-01-29T18:00:46.693+0500    [####....................]  test.user  1696000/10000005  (17.0%)
2021-01-29T18:00:47.713+0500    [####....................]  test.user  1792000/10000005  (17.9%)
2021-01-29T18:00:48.703+0500    [####....................]  test.user  1896000/10000005  (19.0%)
2021-01-29T18:00:49.683+0500    [####....................]  test.user  1992000/10000005  (19.9%)
2021-01-29T18:00:50.713+0500    [#####...................]  test.user  2088000/10000005  (20.9%)
2021-01-29T18:00:51.703+0500    [#####...................]  test.user  2176000/10000005  (21.8%)
2021-01-29T18:00:52.703+0500    [#####...................]  test.user  2256000/10000005  (22.6%)
2021-01-29T18:00:53.684+0500    [#####...................]  test.user  2344000/10000005  (23.4%)
2021-01-29T18:00:54.714+0500    [#####...................]  test.user  2456000/10000005  (24.6%)
2021-01-29T18:00:55.683+0500    [######..................]  test.user  2560000/10000005  (25.6%)
2021-01-29T18:00:56.684+0500    [######..................]  test.user  2672000/10000005  (26.7%)
2021-01-29T18:00:57.683+0500    [######..................]  test.user  2792000/10000005  (27.9%)
2021-01-29T18:00:58.713+0500    [######..................]  test.user  2904000/10000005  (29.0%)
2021-01-29T18:00:59.696+0500    [#######.................]  test.user  2992000/10000005  (29.9%)
2021-01-29T18:01:00.694+0500    [#######.................]  test.user  3096000/10000005  (31.0%)
2021-01-29T18:01:01.714+0500    [#######.................]  test.user  3200000/10000005  (32.0%)
2021-01-29T18:01:02.694+0500    [#######.................]  test.user  3304000/10000005  (33.0%)
2021-01-29T18:01:03.694+0500    [########................]  test.user  3408000/10000005  (34.1%)
2021-01-29T18:01:04.694+0500    [########................]  test.user  3512000/10000005  (35.1%)
2021-01-29T18:01:05.684+0500    [########................]  test.user  3600000/10000005  (36.0%)
2021-01-29T18:01:06.684+0500    [########................]  test.user  3704000/10000005  (37.0%)
2021-01-29T18:01:07.714+0500    [#########...............]  test.user  3816000/10000005  (38.2%)
2021-01-29T18:01:08.715+0500    [#########...............]  test.user  3936000/10000005  (39.4%)
2021-01-29T18:01:09.694+0500    [#########...............]  test.user  4016000/10000005  (40.2%)
2021-01-29T18:01:10.694+0500    [#########...............]  test.user  4096000/10000005  (41.0%)
2021-01-29T18:01:11.714+0500    [##########..............]  test.user  4192000/10000005  (41.9%)
2021-01-29T18:01:12.714+0500    [##########..............]  test.user  4256000/10000005  (42.6%)
2021-01-29T18:01:13.704+0500    [##########..............]  test.user  4328000/10000005  (43.3%)
2021-01-29T18:01:14.684+0500    [##########..............]  test.user  4424000/10000005  (44.2%)
2021-01-29T18:01:15.714+0500    [##########..............]  test.user  4512000/10000005  (45.1%)
2021-01-29T18:01:16.754+0500    [###########.............]  test.user  4600000/10000005  (46.0%)
2021-01-29T18:01:17.695+0500    [###########.............]  test.user  4680000/10000005  (46.8%)
2021-01-29T18:01:18.695+0500    [###########.............]  test.user  4760000/10000005  (47.6%)
2021-01-29T18:01:19.715+0500    [###########.............]  test.user  4832000/10000005  (48.3%)
2021-01-29T18:01:20.715+0500    [###########.............]  test.user  4928000/10000005  (49.3%)
2021-01-29T18:01:21.685+0500    [###########.............]  test.user  5000000/10000005  (50.0%)
2021-01-29T18:01:22.685+0500    [############............]  test.user  5088000/10000005  (50.9%)
2021-01-29T18:01:23.695+0500    [############............]  test.user  5200000/10000005  (52.0%)
2021-01-29T18:01:24.715+0500    [############............]  test.user  5288000/10000005  (52.9%)
2021-01-29T18:01:25.705+0500    [############............]  test.user  5352000/10000005  (53.5%)
2021-01-29T18:01:26.695+0500    [#############...........]  test.user  5464000/10000005  (54.6%)
2021-01-29T18:01:27.715+0500    [#############...........]  test.user  5552000/10000005  (55.5%)
2021-01-29T18:01:28.685+0500    [#############...........]  test.user  5640000/10000005  (56.4%)
2021-01-29T18:01:29.705+0500    [#############...........]  test.user  5720000/10000005  (57.2%)
2021-01-29T18:01:30.715+0500    [#############...........]  test.user  5808000/10000005  (58.1%)
2021-01-29T18:01:31.715+0500    [##############..........]  test.user  5904000/10000005  (59.0%)
2021-01-29T18:01:32.715+0500    [##############..........]  test.user  6008000/10000005  (60.1%)
2021-01-29T18:01:33.688+0500    [##############..........]  test.user  6104000/10000005  (61.0%)
2021-01-29T18:01:34.695+0500    [##############..........]  test.user  6216000/10000005  (62.2%)
2021-01-29T18:01:35.695+0500    [###############.........]  test.user  6304000/10000005  (63.0%)
2021-01-29T18:01:36.697+0500    [###############.........]  test.user  6368000/10000005  (63.7%)
2021-01-29T18:01:37.696+0500    [###############.........]  test.user  6464000/10000005  (64.6%)
2021-01-29T18:01:38.696+0500    [###############.........]  test.user  6568000/10000005  (65.7%)
2021-01-29T18:01:39.716+0500    [################........]  test.user  6680000/10000005  (66.8%)
2021-01-29T18:01:40.716+0500    [################........]  test.user  6776000/10000005  (67.8%)
2021-01-29T18:01:41.716+0500    [################........]  test.user  6872000/10000005  (68.7%)
2021-01-29T18:01:42.696+0500    [################........]  test.user  6976000/10000005  (69.8%)
2021-01-29T18:01:43.706+0500    [################........]  test.user  7056000/10000005  (70.6%)
2021-01-29T18:01:44.716+0500    [#################.......]  test.user  7136000/10000005  (71.4%)
2021-01-29T18:01:45.696+0500    [#################.......]  test.user  7216000/10000005  (72.2%)
2021-01-29T18:01:46.716+0500    [#################.......]  test.user  7288000/10000005  (72.9%)
2021-01-29T18:01:47.696+0500    [#################.......]  test.user  7384000/10000005  (73.8%)
2021-01-29T18:01:48.706+0500    [#################.......]  test.user  7480000/10000005  (74.8%)
2021-01-29T18:01:49.716+0500    [##################......]  test.user  7576000/10000005  (75.8%)
2021-01-29T18:01:50.706+0500    [##################......]  test.user  7680000/10000005  (76.8%)
2021-01-29T18:01:51.697+0500    [##################......]  test.user  7776000/10000005  (77.8%)
2021-01-29T18:01:52.727+0500    [##################......]  test.user  7888000/10000005  (78.9%)
2021-01-29T18:01:53.697+0500    [###################.....]  test.user  7992000/10000005  (79.9%)
2021-01-29T18:01:54.697+0500    [###################.....]  test.user  8088000/10000005  (80.9%)
2021-01-29T18:01:55.717+0500    [###################.....]  test.user  8200000/10000005  (82.0%)
2021-01-29T18:01:56.697+0500    [###################.....]  test.user  8304000/10000005  (83.0%)
2021-01-29T18:01:57.717+0500    [####################....]  test.user  8400000/10000005  (84.0%)
2021-01-29T18:01:58.709+0500    [####################....]  test.user  8520000/10000005  (85.2%)
2021-01-29T18:01:59.697+0500    [####################....]  test.user  8600000/10000005  (86.0%)
2021-01-29T18:02:00.717+0500    [####################....]  test.user  8712000/10000005  (87.1%)
2021-01-29T18:02:01.697+0500    [#####################...]  test.user  8808000/10000005  (88.1%)
2021-01-29T18:02:02.717+0500    [#####################...]  test.user  8920000/10000005  (89.2%)
2021-01-29T18:02:03.737+0500    [#####################...]  test.user  9016000/10000005  (90.2%)
2021-01-29T18:02:04.698+0500    [#####################...]  test.user  9112000/10000005  (91.1%)
2021-01-29T18:02:05.697+0500    [######################..]  test.user  9216000/10000005  (92.2%)
2021-01-29T18:02:06.717+0500    [######################..]  test.user  9312000/10000005  (93.1%)
2021-01-29T18:02:07.697+0500    [######################..]  test.user  9408000/10000005  (94.1%)
2021-01-29T18:02:08.708+0500    [######################..]  test.user  9504000/10000005  (95.0%)
2021-01-29T18:02:09.688+0500    [######################..]  test.user  9568000/10000005  (95.7%)
2021-01-29T18:02:10.688+0500    [#######################.]  test.user  9664000/10000005  (96.6%)
2021-01-29T18:02:11.758+0500    [#######################.]  test.user  9776000/10000005  (97.8%)
2021-01-29T18:02:12.688+0500    [#######################.]  test.user  9832000/10000005  (98.3%)
2021-01-29T18:02:13.698+0500    [#######################.]  test.user  9928000/10000005  (99.3%)
2021-01-29T18:02:14.358+0500    [########################]  test.user  10000005/10000005  (100.0%)

After [SUCCESS] then every thing is printed at once that is not correct progressbar is necessary to see progress as mongoexport do by default

`mongoexport --host=localhost --port=27017 --db=test --collection=user -vvvvv --fields="name,emails" --type=csv --out=~/test.csv 2021-01-29T17:52:45.696+0500 will listen for SIGTERM, SIGINT, and SIGKILL 2021-01-29T17:52:45.702+0500 connected to: mongodb://localhost:27017/ 2021-01-29T17:52:45.705+0500 Getting estimated count for test.user 2021-01-29T17:52:46.738+0500 [........................] test.user 104000/10000005 (1.0%) 2021-01-29T17:52:47.738+0500 [........................] test.user 208000/10000005 (2.1%) 2021-01-29T17:52:48.728+0500 [........................] test.user 312000/10000005 (3.1%) 2021-01-29T17:52:49.728+0500 [........................] test.user 416000/10000005 (4.2%) 2021-01-29T17:52:50.728+0500 [#.......................] test.user 520000/10000005 (5.2%) 2021-01-29T17:52:51.718+0500 [#.......................] test.user 632000/10000005 (6.3%) 2021-01-29T17:52:52.728+0500 [#.......................] test.user 736000/10000005 (7.4%) 2021-01-29T17:52:53.738+0500 [##......................] test.user 840000/10000005 (8.4%) 2021-01-29T17:52:54.738+0500 [##......................] test.user 936000/10000005 (9.4%) 2021-01-29T17:52:55.718+0500 [##......................] test.user 1032000/10000005 (10.3%) 2021-01-29T17:52:56.715+0500 [##......................] test.user 1144000/10000005 (11.4%) ^C2021-01-29T17:52:57.028+0500 signal 'interrupt' received; forcefully terminating

`

@euberdeveloper

euberdeveloper commented 3 years ago

You are right, the problem is that the library uses shelljs under the hood, which returns the stdout and the stderr only after the command finishes. It is expected behaviour for now. What you want could be achieved by using the event listeners on the child process, so I will mark this issue as "enhancement"

raza91 commented 3 years ago

Any idea when it can be released? @euberdeveloper

euberdeveloper commented 3 years ago

It is something that could be done in a day, but I have some exams and a short job now, so I think that I would do that in something like a week. I will write here when a new version gets released

raza91 commented 3 years ago

@euberdeveloper if you can please release it a bit earlier like in 2-3 days that will be much appreciated my work is dependent (stuck) on that enhancement

euberdeveloper commented 3 years ago

Yes, I think that I could do it

euberdeveloper commented 3 years ago

Turned out to be easier than expected. You should update to the new published version and read the realtimeLog option here: https://mongoback.euber.dev/interfaces/logoptions.html. (if you want, add a :star:)

raza91 commented 3 years ago

@euberdeveloper Excellent , Thanks