RD17 / ambar

:mag: Ambar: Document Search Engine
https://ambar.cloud/
MIT License
1.95k stars 376 forks source link

Downloading a file from Ambar Search & view files from mounting location (Crawler) #193

Closed itnoreplymax closed 5 years ago

itnoreplymax commented 6 years ago

Hi Team, Great piece of software Application. Congratulations. My Ambar setup is working fine and I can search any document I submit with the portal. But, my {dataPath} still remains empty. I was expecting to find here Ambar data.. I tried local volume & S3 bucket but no luck to store my { data path}.

1) Is it possible to download or export a file after it has been indexed and uploaded? 2) I have checked with local volume & s3 bucket in crawler portion, Once file uploaded i can to check my path the file was not present. How can to view a file in storage path.

Here attached my setup environment.

Set 2-Crawling setting for s3 bucket .docx

Set.1-Craweling.for.local.storage.1.docx docker logs.txt

**Note: server /etc/fstab information about S3 bucket

testbucket /mnt/aws-s3-bucket fuse.s3fs _netdev,allow_other 0 0**

itnoreplymax commented 6 years ago

HI Please find the belog logs frm excuting the ambar

root@ip-172-31-7-167:~/Ambar/ambar-master# docker-compose logs Attaching to ambarmaster_frontend_1, ambarmaster_pipeline0_1, ambarmaster_webapi_1, ambarmaster_myfiles_1, ambarmaster_serviceapi_1, ambarmaster_redis_1, ambarmaster_db_1, ambarmaster_rabbit_1, ambarmaster_es_1 webapi_1 | Started on :::8080 frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:44:08 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 49.207.183.30 - - [23/Oct/2018:10:44:10 +0000] "GET /?query=ambar HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Windows NT 6.1; rv:62.0) Gecko/20100101 Firefox/62.0" "-" frontend_1 | 49.207.183.30 - - [23/Oct/2018:10:44:10 +0000] "GET /vendor.eb8ecdba4b04debacfc8.js HTTP/1.1" 304 0 "http://18.224.211.18/?query=ambar" "Mozilla/5.0 (Windows NT 6.1; rv:62.0) Gecko/20100101 Firefox/62.0" "-" frontend_1 | 49.207.183.30 - - [23/Oct/2018:10:44:10 +0000] "GET /app.efa6cfc157288899b0ae.js HTTP/1.1" 304 0 "http://18.224.211.18/?query=ambar" "Mozilla/5.0 (Windows NT 6.1; rv:62.0) Gecko/20100101 Firefox/62.0" "-" frontend_1 | 49.207.183.30 - - [23/Oct/2018:10:44:10 +0000] "GET /app.bab44ae248798bbdab6f0943f400f63d.css HTTP/1.1" 304 0 "http://18.224.211.18/?query=ambar" "Mozilla/5.0 (Windows NT 6.1; rv:62.0) Gecko/20100101 Firefox/62.0" "-" frontend_1 | 49.207.183.30 - - [23/Oct/2018:10:44:10 +0000] "GET /0.searchPage.2fefb58efd3bc7904f8a.js HTTP/1.1" 304 0 "http://18.224.211.18/?query=ambar" "Mozilla/5.0 (Windows NT 6.1; rv:62.0) Gecko/20100101 Firefox/62.0" "-" frontend_1 | 49.207.183.30 - - [23/Oct/2018:10:44:11 +0000] "GET /apiUrl.txt HTTP/1.1" 200 26 "http://18.224.211.18/?query=ambar" "Mozilla/5.0 (Windows NT 6.1; rv:62.0) Gecko/20100101 Firefox/62.0" "-" frontend_1 | 49.207.183.30 - - [23/Oct/2018:10:44:11 +0000] "GET /localizations.json HTTP/1.1" 200 19940 "http://18.224.211.18/?query=ambar" "Mozilla/5.0 (Windows NT 6.1; rv:62.0) Gecko/20100101 Firefox/62.0" "-" frontend_1 | 49.207.183.30 - - [23/Oct/2018:10:44:12 +0000] "GET /owl.svg HTTP/1.1" 200 1183 "http://18.224.211.18/?query=ambar" "Mozilla/5.0 (Windows NT 6.1; rv:62.0) Gecko/20100101 Firefox/62.0" "-" frontend_1 | 49.207.183.30 - - [23/Oct/2018:10:44:13 +0000] "GET / HTTP/1.1" 200 969 "-" "Mozilla/5.0 (Windows NT 6.1; rv:62.0) Gecko/20100101 Firefox/62.0" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:44:14 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 49.207.183.30 - - [23/Oct/2018:10:44:14 +0000] "GET /apiUrl.txt HTTP/1.1" 200 26 "http://18.224.211.18/" "Mozilla/5.0 (Windows NT 6.1; rv:62.0) Gecko/20100101 Firefox/62.0" "-" frontend_1 | 49.207.183.30 - - [23/Oct/2018:10:44:14 +0000] "GET /localizations.json HTTP/1.1" 200 19940 "http://18.224.211.18/" "Mozilla/5.0 (Windows NT 6.1; rv:62.0) Gecko/20100101 Firefox/62.0" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:44:19 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:44:24 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:44:30 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:44:35 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:44:40 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:44:45 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:44:51 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:44:56 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:45:01 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:45:07 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:45:12 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:45:17 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:45:22 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:45:28 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:45:33 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:45:38 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:45:43 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:45:49 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:45:54 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:45:59 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:46:04 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:46:10 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:46:15 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:46:20 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:46:25 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:46:31 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:46:36 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:46:41 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:46:47 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:46:52 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:46:57 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:47:02 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:47:08 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:47:13 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:47:18 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:47:23 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:47:29 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:47:34 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" pipeline0_1 | Oct 23, 2018 10:43:58 AM org.apache.tika.config.InitializableProblemHandler$3 handleInitializableProblem frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:47:39 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" pipeline0_1 | WARNING: J2KImageReader not loaded. JPEG2000 files will not be processed. frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:47:44 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:47:50 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:47:55 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" pipeline0_1 | See https://pdfbox.apache.org/2.0/dependencies.html#jai-image-io pipeline0_1 | for optional dependencies. frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:48:00 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:48:06 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:48:11 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" pipeline0_1 | frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:48:16 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:48:21 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:48:27 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:48:32 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:48:37 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:48:42 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:48:48 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:48:53 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" myfiles_1 | [info] API runs on master thread myfiles_1 | [info] Creating fork for the file-watcher process myfiles_1 | Started API on :::8082 myfiles_1 | { RequestError: Error: connect ECONNREFUSED 172.18.0.6:8080 pipeline0_1 | Oct 23, 2018 10:43:58 AM org.apache.tika.config.InitializableProblemHandler$3 handleInitializableProblem frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:48:58 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:49:03 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:49:09 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" myfiles_1 | at new RequestError (/node_modules/request-promise-core/lib/errors.js:14:15) myfiles_1 | at Request.plumbing.callback (/node_modules/request-promise-core/lib/plumbing.js:87:29) myfiles_1 | at Request.RP$callback [as _callback] (/node_modules/request-promise-core/lib/plumbing.js:46:31) myfiles_1 | at self.callback (/node_modules/request/request.js:186:22) frontend_1 | 127.0.0.1 - - [23/Oct/2018:10:49:14 +0000] "GET / HTTP/1.1" 200 969 "-" "curl/7.52.1" "-" myfiles_1 | at emitOne (events.js:116:13) myfiles_1 | at Request.emit (events.js:211:7) myfiles_1 | at Request.onRequestError (/node_modules/request/request.js:878:8) myfiles_1 | at emitOne (events.js:116:13) myfiles_1 | at ClientRequest.emit (events.js:211:7) myfiles_1 | at Socket.socketErrorListener (_http_client.js:387:9) myfiles_1 | at emitOne (events.js:116:13) myfiles_1 | at Socket.emit (events.js:211:7) myfiles_1 | at emitErrorNT (internal/streams/destroy.js:64:8) myfiles_1 | at _combinedTickCallback (internal/process/next_tick.js:138:11) myfiles_1 | at process._tickCallback (internal/process/next_tick.js:180:9) myfiles_1 | name: 'RequestError', myfiles_1 | message: 'Error: connect ECONNREFUSED 172.18.0.6:8080', pipeline0_1 | WARNING: Tesseract OCR is installed and will be automatically applied to image files unless pipeline0_1 | you've excluded the TesseractOCRParser from the default parser. myfiles_1 | cause: myfiles_1 | { Error: connect ECONNREFUSED 172.18.0.6:8080 pipeline0_1 | Tesseract may dramatically slow down content extraction (TIKA-2359). pipeline0_1 | As of Tika 1.15 (and prior versions), Tesseract is automatically called. pipeline0_1 | In future versions of Tika, users may need to turn the TesseractOCRParser on via TikaConfig. pipeline0_1 | Oct 23, 2018 10:43:58 AM org.apache.tika.config.InitializableProblemHandler$3 handleInitializableProblem pipeline0_1 | WARNING: org.xerial's sqlite-jdbc is not loaded. pipeline0_1 | Please provide the jar on your classpath to parse sqlite files. pipeline0_1 | See tika-parsers/pom.xml for the correct version. myfiles_1 | at Object._errnoException (util.js:1022:11) serviceapi_1 | Started on :::8081 db_1 | 2018-10-23T10:43:34.715+0000 I CONTROL [initandlisten] MongoDB starting : pid=1 port=27017 dbpath=/data/db 64-bit host=5c964449184a myfiles_1 | at _exceptionWithHostPort (util.js:1044:20) db_1 | 2018-10-23T10:43:34.716+0000 I CONTROL [initandlisten] db version v3.4.2 myfiles_1 | at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1198:14) myfiles_1 | code: 'ECONNREFUSED', db_1 | 2018-10-23T10:43:34.716+0000 I CONTROL [initandlisten] git version: 3f76e40c105fc223b3e5aac3e20dcd026b83b38b db_1 | 2018-10-23T10:43:34.716+0000 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.1t 3 May 2016 redis_1 | 1:C 23 Oct 10:43:33.792 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo db_1 | 2018-10-23T10:43:34.716+0000 I CONTROL [initandlisten] allocator: tcmalloc myfiles_1 | errno: 'ECONNREFUSED', myfiles_1 | syscall: 'connect', myfiles_1 | address: '172.18.0.6', db_1 | 2018-10-23T10:43:34.716+0000 I CONTROL [initandlisten] modules: none myfiles_1 | port: 8080 }, myfiles_1 | error: myfiles_1 | { Error: connect ECONNREFUSED 172.18.0.6:8080 db_1 | 2018-10-23T10:43:34.716+0000 I CONTROL [initandlisten] build environment: redis_1 | 1:C 23 Oct 10:43:33.792 # Redis version=4.0.2, bits=64, commit=00000000, modified=0, pid=1, just started myfiles_1 | at Object._errnoException (util.js:1022:11) db_1 | 2018-10-23T10:43:34.716+0000 I CONTROL [initandlisten] distmod: debian81 redis_1 | 1:C 23 Oct 10:43:33.792 # Configuration loaded rabbit_1 | myfiles_1 | at _exceptionWithHostPort (util.js:1044:20) db_1 | 2018-10-23T10:43:34.716+0000 I CONTROL [initandlisten] distarch: x86_64 es_1 | [2018-10-23T10:43:34,810][INFO ][o.e.n.Node ] [] initializing ... redis1 | ._ rabbit_1 | =INFO REPORT==== 23-Oct-2018::10:43:39 === myfiles_1 | at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1198:14) db_1 | 2018-10-23T10:43:34.716+0000 I CONTROL [initandlisten] target_arch: x86_64 es_1 | [2018-10-23T10:43:35,024][INFO ][o.e.e.NodeEnvironment ] [RpjheBm] using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/xvda1)]], net usable_space [41.6gb], net total_space [48.4gb], spins? [possibly], types [ext4] redis1 | .-__ ''-._ rabbit_1 | Starting RabbitMQ 3.6.16 on Erlang 19.2.1 myfiles_1 | code: 'ECONNREFUSED', db_1 | 2018-10-23T10:43:34.716+0000 I CONTROL [initandlisten] options: { config: "/etc/mongod.conf", storage: { wiredTiger: { engineConfig: { cacheSizeGB: 2.0 } } } } es_1 | [2018-10-23T10:43:35,035][INFO ][o.e.e.NodeEnvironment ] [RpjheBm] heap size [1.9gb], compressed ordinary object pointers [true] redis_1 | _.- .. ''-. Redis 4.0.2 (00000000/0) 64 bit rabbit_1 | Copyright (C) 2007-2018 Pivotal Software, Inc. myfiles_1 | errno: 'ECONNREFUSED', db_1 | 2018-10-23T10:43:34.721+0000 I - [initandlisten] Detected data files in /data/db created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'. es_1 | [2018-10-23T10:43:35,099][INFO ][o.e.n.Node ] node name [RpjheBm] derived from node ID [RpjheBmcTzq0OkbftZhCaw]; set [node.name] to override redis1 | .-`` .-.\/ ., ''-. rabbit_1 | Licensed under the MPL. See http://www.rabbitmq.com/ myfiles_1 | syscall: 'connect', db_1 | 2018-10-23T10:43:34.721+0000 I STORAGE [initandlisten] es_1 | [2018-10-23T10:43:35,099][INFO ][o.e.n.Node ] version[5.6.3], pid[1], build[1a2f265/2017-10-06T20:33:39.012Z], OS[Linux/4.15.0-1023-aws/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_151/25.151-b12] redis_1 | ( ' , .-|, ) Running in standalone mode rabbit_1 | myfiles_1 | address: '172.18.0.6', db_1 | 2018-10-23T10:43:34.721+0000 I STORAGE [initandlisten] WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine es_1 | [2018-10-23T10:43:35,100][INFO ][o.e.n.Node ] JVM arguments [-Xms2g, -Xmx2g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djdk.io.permissionsUseCanonicalPath=true, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j.skipJansi=true, -XX:+HeapDumpOnOutOfMemoryError, -Xms2g, -Xmx2g, -Des.path.home=/usr/share/elasticsearch] redis1 | |`-.-...- _...-.``-.|'` _.-'| Port: 6379 rabbit_1 | RabbitMQ 3.6.16. Copyright (C) 2007-2018 Pivotal Software, Inc. myfiles_1 | port: 8080 }, db_1 | 2018-10-23T10:43:34.721+0000 I STORAGE [initandlisten] See http://dochub.mongodb.org/core/prodnotes-filesystem es_1 | [2018-10-23T10:43:38,449][INFO ][o.e.p.PluginsService ] [RpjheBm] loaded module [aggs-matrix-stats] redis1 | | `-. ._ / _.-' | PID: 1 rabbit_1 | ## ## Licensed under the MPL. See http://www.rabbitmq.com/ myfiles_1 | options: db_1 | 2018-10-23T10:43:34.721+0000 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=2048M,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0), redis_1 |-. `-. -./ _.-' _.-' es_1 | [2018-10-23T10:43:38,449][INFO ][o.e.p.PluginsService ] [RpjheBm] loaded module [ingest-common] myfiles_1 | { uri: 'http://serviceapi:8080/api/logs', myfiles_1 | method: 'POST', myfiles_1 | body: '{"source_id":"myfiles","type":"info","message":"API runs on master thread","created_datetime":"2018-10-23 10:43:56.466"}', rabbit_1 | ## ## es_1 | [2018-10-23T10:43:38,449][INFO ][o.e.p.PluginsService ] [RpjheBm] loaded module [lang-expression] es_1 | [2018-10-23T10:43:38,450][INFO ][o.e.p.PluginsService ] [RpjheBm] loaded module [lang-groovy] es_1 | [2018-10-23T10:43:38,450][INFO ][o.e.p.PluginsService ] [RpjheBm] loaded module [lang-mustache] myfiles_1 | headers: { 'Content-Type': 'application/json' }, db_1 | 2018-10-23T10:43:35.203+0000 I CONTROL [initandlisten] redis_1 | |-.`-. -.__.-' _.-'_.-'| es_1 | [2018-10-23T10:43:38,450][INFO ][o.e.p.PluginsService ] [RpjheBm] loaded module [lang-painless] myfiles_1 | callback: [Function: RP$callback], rabbit_1 | ########## Logs: tty db_1 | 2018-10-23T10:43:35.203+0000 I CONTROL [initandlisten] ** WARNING: Access control is not enabled for the database. redis_1 | |-.`-. .-'.-' | http://redis.io es_1 | [2018-10-23T10:43:38,450][INFO ][o.e.p.PluginsService ] [RpjheBm] loaded module [parent-join] myfiles_1 | transform: undefined, rabbit_1 | ###### ## tty db_1 | 2018-10-23T10:43:35.203+0000 I CONTROL [initandlisten] Read and write access to data and configuration is unrestricted. redis1 | `-. -._-._.-'.-' _.-' es_1 | [2018-10-23T10:43:38,450][INFO ][o.e.p.PluginsService ] [RpjheBm] loaded module [percolator] myfiles_1 | simple: true, rabbit_1 | ########## db_1 | 2018-10-23T10:43:35.203+0000 I CONTROL [initandlisten] WARNING: You are running this process as the root user, which is not recommended. redis1 | |`-.-._-..-' .-'.-'| es_1 | [2018-10-23T10:43:38,450][INFO ][o.e.p.PluginsService ] [RpjheBm] loaded module [reindex] myfiles_1 | resolveWithFullResponse: false, rabbit_1 | Starting broker... db_1 | 2018-10-23T10:43:35.203+0000 I CONTROL [initandlisten] redis1 | | `-.-._ _.-'_.-' | es_1 | [2018-10-23T10:43:38,450][INFO ][o.e.p.PluginsService ] [RpjheBm] loaded module [transport-netty3] myfiles_1 | transform2xxOnly: false }, rabbit_1 | db_1 | 2018-10-23T10:43:35.231+0000 I FTDC [initandlisten] Initializing full-time diagnostic data capture with directory '/data/db/diagnostic.data' redis_1 |-. `-.`-..-'.-' .-' myfiles_1 | response: undefined } es_1 | [2018-10-23T10:43:38,450][INFO ][o.e.p.PluginsService ] [RpjheBm] loaded module [transport-netty4] rabbit_1 | =INFO REPORT==== 23-Oct-2018::10:43:39 === db_1 | 2018-10-23T10:43:35.232+0000 I NETWORK [thread1] waiting for connections on port 27017 redis1 | `-. -.__.-' _.-' myfiles_1 | (node:6) UnhandledPromiseRejectionWarning: RequestError: Error: connect ECONNREFUSED 172.18.0.6:8080 es_1 | [2018-10-23T10:43:38,451][INFO ][o.e.p.PluginsService ] [RpjheBm] loaded plugin [analysis-morphology] rabbit_1 | node : rabbit@rabbit db_1 | 2018-10-23T10:43:39.697+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:55902 #1 (1 connection now open) redis_1 |-. .-' myfiles_1 | at new RequestError (/node_modules/request-promise-core/lib/errors.js:14:15) es_1 | [2018-10-23T10:43:38,451][INFO ][o.e.p.PluginsService ] [RpjheBm] loaded plugin [analysis-smartcn] rabbit_1 | home dir : /var/lib/rabbitmq db_1 | 2018-10-23T10:43:39.698+0000 I - [conn1] end connection 127.0.0.1:55902 (1 connection now open) redis_1 | `-.__.-' myfiles_1 | at Request.plumbing.callback (/node_modules/request-promise-core/lib/plumbing.js:87:29) es_1 | [2018-10-23T10:43:38,451][INFO ][o.e.p.PluginsService ] [RpjheBm] loaded plugin [analysis-stempel] rabbit_1 | config file(s) : /etc/rabbitmq/rabbitmq.config redis_1 | es_1 | [2018-10-23T10:43:41,703][INFO ][o.e.d.DiscoveryModule ] [RpjheBm] using discovery type [zen] db_1 | 2018-10-23T10:43:45.089+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:55936 #2 (1 connection now open) myfiles_1 | at Request.RP$callback [as _callback] (/node_modules/request-promise-core/lib/plumbing.js:46:31) rabbit_1 | cookie hash : f7hhhrZQak8y0pQHsRQWOQ== redis_1 | 1:M 23 Oct 10:43:33.796 # Server initialized es_1 | [2018-10-23T10:43:42,446][INFO ][o.e.n.Node ] initialized es_1 | [2018-10-23T10:43:42,447][INFO ][o.e.n.Node ] [RpjheBm] starting ... myfiles_1 | at self.callback (/node_modules/request/request.js:186:22) rabbit_1 | log : tty db_1 | 2018-10-23T10:43:45.090+0000 I - [conn2] end connection 127.0.0.1:55936 (1 connection now open) es_1 | [2018-10-23T10:43:42,616][INFO ][o.e.t.TransportService ] [RpjheBm] publish_address {172.18.0.2:9300}, bound_addresses {0.0.0.0:9300} myfiles_1 | at emitOne (events.js:116:13) rabbit_1 | sasl log : tty redis_1 | 1:M 23 Oct 10:43:33.796 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled. es_1 | [2018-10-23T10:43:42,627][INFO ][o.e.b.BootstrapChecks ] [RpjheBm] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks es_1 | [2018-10-23T10:43:45,686][INFO ][o.e.c.s.ClusterService ] [RpjheBm] new_master {RpjheBm}{RpjheBmcTzq0OkbftZhCaw}{VsTrihWZT-Sm6nFVAPM7tA}{172.18.0.2}{172.18.0.2:9300}, reason: zen-disco-elected-as-master ([0] nodes joined) rabbit_1 | database dir : /var/lib/rabbitmq/mnesia/rabbit@rabbit db_1 | 2018-10-23T10:43:50.261+0000 I NETWORK [thread1] connection accepted from 172.18.0.6:55742 #3 (1 connection now open) redis_1 | 1:M 23 Oct 10:43:33.796 * Ready to accept connections myfiles_1 | at Request.emit (events.js:211:7) es_1 | [2018-10-23T10:43:45,732][INFO ][o.e.h.n.Netty4HttpServerTransport] [RpjheBm] publish_address {172.18.0.2:9200}, bound_addresses {0.0.0.0:9200} rabbit_1 | rabbit_1 | =INFO REPORT==== 23-Oct-2018::10:43:41 === myfiles_1 | at Request.onRequestError (/node_modules/request/request.js:878:8) redis_1 | 1:M 23 Oct 10:43:39.066 # Possible SECURITY ATTACK detected. It looks like somebody is sending POST or Host: commands to Redis. This is likely due to an attacker attempting to use Cross Protocol Scripting to compromise your Redis instance. Connection aborted. redis_1 | 1:M 23 Oct 10:44:42.981 # Possible SECURITY ATTACK detected. It looks like somebody is sending POST or Host: commands to Redis. This is likely due to an attacker attempting to use Cross Protocol Scripting to compromise your Redis instance. Connection aborted. rabbit_1 | Memory high watermark set to 3190 MiB (3345181900 bytes) of 7975 MiB (8362954752 bytes) total db_1 | 2018-10-23T10:43:50.270+0000 I NETWORK [conn3] received client metadata from 172.18.0.6:55742 conn3: { driver: { name: "nodejs", version: "2.2.12" }, os: { type: "Linux", name: "linux", architecture: "x64", version: "4.15.0-1023-aws" }, platform: "Node.js v8.10.0, LE, mongodb-core: 2.0.14" } redis_1 | 1:M 23 Oct 10:45:46.730 # Possible SECURITY ATTACK detected. It looks like somebody is sending POST or Host: commands to Redis. This is likely due to an attacker attempting to use Cross Protocol Scripting to compromise your Redis instance. Connection aborted. es_1 | [2018-10-23T10:43:45,732][INFO ][o.e.n.Node ] [RpjheBm] started myfiles_1 | at emitOne (events.js:116:13) rabbit_1 | db_1 | 2018-10-23T10:43:50.379+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:55952 #4 (2 connections now open) redis_1 | 1:M 23 Oct 10:46:50.503 # Possible SECURITY ATTACK detected. It looks like somebody is sending POST or Host: commands to Redis. This is likely due to an attacker attempting to use Cross Protocol Scripting to compromise your Redis instance. Connection aborted. redis_1 | 1:M 23 Oct 10:47:54.270 # Possible SECURITY ATTACK detected. It looks like somebody is sending POST or Host: commands to Redis. This is likely due to an attacker attempting to use Cross Protocol Scripting to compromise your Redis instance. Connection aborted. myfiles_1 | at ClientRequest.emit (events.js:211:7) rabbit_1 | =INFO REPORT==== 23-Oct-2018::10:43:41 === redis_1 | 1:M 23 Oct 10:48:58.037 # Possible SECURITY ATTACK detected. It looks like somebody is sending POST or Host: commands to Redis. This is likely due to an attacker attempting to use Cross Protocol Scripting to compromise your Redis instance. Connection aborted. es_1 | [2018-10-23T10:43:45,988][INFO ][o.e.g.GatewayService ] [RpjheBm] recovered [2] indices into cluster_state db_1 | 2018-10-23T10:43:50.383+0000 I - [conn4] end connection 127.0.0.1:55952 (2 connections now open) rabbit_1 | Enabling free disk space monitoring myfiles_1 | at Socket.socketErrorListener (_http_client.js:387:9) es_1 | [2018-10-23T10:43:46,859][INFO ][o.e.c.r.a.AllocationService] [RpjheBm] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[ambar_log_record_data][5]] ...]). rabbit_1 | myfiles_1 | at emitOne (events.js:116:13) db_1 | 2018-10-23T10:43:55.663+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:55962 #5 (2 connections now open) es_1 | [2018-10-23T10:44:13,453][WARN ][o.e.d.r.RestController ] The Content-Type [application/x-ldjson] has been superseded by [application/x-ndjson] in the specification and should be used instead. es_1 | [2018-10-23T10:44:51,197][WARN ][o.e.d.r.RestController ] The Content-Type [application/x-ldjson] has been superseded by [application/x-ndjson] in the specification and should be used instead. myfiles_1 | at Socket.emit (events.js:211:7) db_1 | 2018-10-23T10:43:55.665+0000 I - [conn5] end connection 127.0.0.1:55962 (2 connections now open) es_1 | [2018-10-23T10:44:53,467][WARN ][o.e.d.r.RestController ] The Content-Type [application/x-ldjson] has been superseded by [application/x-ndjson] in the specification and should be used instead. es_1 | [2018-10-23T10:48:16,678][WARN ][o.e.d.r.RestController ] The Content-Type [application/x-ldjson] has been superseded by [application/x-ndjson] in the specification and should be used instead. myfiles_1 | at emitErrorNT (internal/streams/destroy.js:64:8) myfiles_1 | at _combinedTickCallback (internal/process/next_tick.js:138:11) myfiles_1 | at process._tickCallback (internal/process/next_tick.js:180:9) rabbit_1 | =INFO REPORT==== 23-Oct-2018::10:43:41 === db_1 | 2018-10-23T10:43:58.786+0000 I NETWORK [thread1] connection accepted from 172.18.0.9:59434 #6 (2 connections now open) db_1 | 2018-10-23T10:43:58.857+0000 I NETWORK [conn6] received client metadata from 172.18.0.9:59434 conn6: { driver: { name: "nodejs", version: "2.2.12" }, os: { type: "Linux", name: "linux", architecture: "x64", version: "4.15.0-1023-aws" }, platform: "Node.js v8.10.0, LE, mongodb-core: 2.0.14" } rabbit_1 | Disk free limit set to 50MB rabbit_1 | rabbit_1 | =INFO REPORT==== 23-Oct-2018::10:43:41 === rabbit_1 | Limiting to approx 1048476 file handles (943626 sockets) rabbit_1 | rabbit_1 | =INFO REPORT==== 23-Oct-2018::10:43:41 === rabbit_1 | FHC read buffering: OFF rabbit_1 | FHC write buffering: ON myfiles_1 | (node:6) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1) myfiles_1 | (node:6) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code. myfiles_1 | { RequestError: Error: connect ECONNREFUSED 172.18.0.6:8080 db_1 | 2018-10-23T10:44:00.931+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56006 #7 (3 connections now open) rabbit_1 | myfiles_1 | at new RequestError (/node_modules/request-promise-core/lib/errors.js:14:15) myfiles_1 | at Request.plumbing.callback (/node_modules/request-promise-core/lib/plumbing.js:87:29) db_1 | 2018-10-23T10:44:00.933+0000 I - [conn7] end connection 127.0.0.1:56006 (3 connections now open) myfiles_1 | at Request.RP$callback [as _callback] (/node_modules/request-promise-core/lib/plumbing.js:46:31) rabbit_1 | =INFO REPORT==== 23-Oct-2018::10:43:42 === db_1 | 2018-10-23T10:44:06.197+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56020 #8 (3 connections now open) myfiles_1 | at self.callback (/node_modules/request/request.js:186:22) rabbit_1 | Waiting for Mnesia tables for 30000 ms, 9 retries left db_1 | 2018-10-23T10:44:06.198+0000 I - [conn8] end connection 127.0.0.1:56020 (3 connections now open) myfiles_1 | at emitOne (events.js:116:13) rabbit_1 | db_1 | 2018-10-23T10:44:11.468+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56036 #9 (3 connections now open) myfiles_1 | at Request.emit (events.js:211:7) rabbit_1 | =INFO REPORT==== 23-Oct-2018::10:43:42 === myfiles_1 | at Request.onRequestError (/node_modules/request/request.js:878:8) rabbit_1 | Waiting for Mnesia tables for 30000 ms, 9 retries left db_1 | 2018-10-23T10:44:11.469+0000 I - [conn9] end connection 127.0.0.1:56036 (3 connections now open) myfiles_1 | at emitOne (events.js:116:13) rabbit_1 | db_1 | 2018-10-23T10:44:16.733+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56052 #10 (3 connections now open) myfiles_1 | at ClientRequest.emit (events.js:211:7) db_1 | 2018-10-23T10:44:16.733+0000 I - [conn10] end connection 127.0.0.1:56052 (3 connections now open) myfiles_1 | at Socket.socketErrorListener (_http_client.js:387:9) rabbit_1 | =INFO REPORT==== 23-Oct-2018::10:43:42 === db_1 | 2018-10-23T10:44:21.996+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56068 #11 (3 connections now open) myfiles_1 | at emitOne (events.js:116:13) rabbit_1 | Priority queues enabled, real BQ is rabbit_variable_queue rabbit_1 | rabbit_1 | =INFO REPORT==== 23-Oct-2018::10:43:42 === db_1 | 2018-10-23T10:44:21.996+0000 I - [conn11] end connection 127.0.0.1:56068 (3 connections now open) myfiles_1 | at Socket.emit (events.js:211:7) db_1 | 2018-10-23T10:44:27.263+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56084 #12 (3 connections now open) myfiles_1 | at emitErrorNT (internal/streams/destroy.js:64:8) rabbit_1 | Starting rabbit_node_monitor db_1 | 2018-10-23T10:44:27.264+0000 I - [conn12] end connection 127.0.0.1:56084 (3 connections now open) myfiles_1 | at _combinedTickCallback (internal/process/next_tick.js:138:11) rabbit_1 | db_1 | 2018-10-23T10:44:32.527+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56100 #13 (3 connections now open) myfiles_1 | at process._tickCallback (internal/process/next_tick.js:180:9) rabbit_1 | =INFO REPORT==== 23-Oct-2018::10:43:42 === db_1 | 2018-10-23T10:44:32.528+0000 I - [conn13] end connection 127.0.0.1:56100 (3 connections now open) myfiles_1 | name: 'RequestError', rabbit_1 | Management plugin: using rates mode 'basic' db_1 | 2018-10-23T10:44:37.792+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56116 #14 (3 connections now open) myfiles_1 | message: 'Error: connect ECONNREFUSED 172.18.0.6:8080', rabbit_1 | rabbit_1 | =INFO REPORT==== 23-Oct-2018::10:43:42 === myfiles_1 | cause: db_1 | 2018-10-23T10:44:37.793+0000 I - [conn14] end connection 127.0.0.1:56116 (3 connections now open) rabbit_1 | msg_store_transient: using rabbit_msg_store_ets_index to provide index db_1 | 2018-10-23T10:44:43.067+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56132 #15 (3 connections now open) myfiles_1 | { Error: connect ECONNREFUSED 172.18.0.6:8080 rabbit_1 | db_1 | 2018-10-23T10:44:43.067+0000 I - [conn15] end connection 127.0.0.1:56132 (3 connections now open) myfiles_1 | at Object._errnoException (util.js:1022:11) rabbit_1 | =INFO REPORT==== 23-Oct-2018::10:43:42 === myfiles_1 | at _exceptionWithHostPort (util.js:1044:20) db_1 | 2018-10-23T10:44:48.332+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56148 #16 (3 connections now open) rabbit_1 | msg_store_persistent: using rabbit_msg_store_ets_index to provide index rabbit_1 | myfiles_1 | at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1198:14) myfiles_1 | code: 'ECONNREFUSED', db_1 | 2018-10-23T10:44:48.333+0000 I - [conn16] end connection 127.0.0.1:56148 (3 connections now open) db_1 | 2018-10-23T10:44:53.609+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56164 #17 (3 connections now open) myfiles_1 | errno: 'ECONNREFUSED', myfiles_1 | syscall: 'connect', myfiles_1 | address: '172.18.0.6', myfiles_1 | port: 8080 }, myfiles_1 | error: db_1 | 2018-10-23T10:44:53.609+0000 I - [conn17] end connection 127.0.0.1:56164 (3 connections now open) db_1 | 2018-10-23T10:44:58.872+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56178 #18 (3 connections now open) rabbit_1 | =INFO REPORT==== 23-Oct-2018::10:43:42 === db_1 | 2018-10-23T10:44:58.873+0000 I - [conn18] end connection 127.0.0.1:56178 (3 connections now open) db_1 | 2018-10-23T10:45:04.136+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56194 #19 (3 connections now open) myfiles_1 | { Error: connect ECONNREFUSED 172.18.0.6:8080 myfiles_1 | at Object._errnoException (util.js:1022:11) myfiles_1 | at _exceptionWithHostPort (util.js:1044:20) myfiles_1 | at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1198:14) myfiles_1 | code: 'ECONNREFUSED', rabbit_1 | started TCP Listener on [::]:5672 db_1 | 2018-10-23T10:45:04.137+0000 I - [conn19] end connection 127.0.0.1:56194 (3 connections now open) myfiles_1 | errno: 'ECONNREFUSED', rabbit_1 | db_1 | 2018-10-23T10:45:09.401+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56210 #20 (3 connections now open) rabbit_1 | =INFO REPORT==== 23-Oct-2018::10:43:42 === myfiles_1 | syscall: 'connect', rabbit_1 | Management plugin started. Port: 15672 myfiles_1 | address: '172.18.0.6', rabbit_1 | db_1 | 2018-10-23T10:45:09.402+0000 I - [conn20] end connection 127.0.0.1:56210 (3 connections now open) myfiles_1 | port: 8080 }, rabbit_1 | =INFO REPORT==== 23-Oct-2018::10:43:42 === rabbit_1 | Statistics database started. rabbit_1 | completed with 6 plugins. db_1 | 2018-10-23T10:45:14.670+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56226 #21 (3 connections now open) myfiles_1 | options: rabbit_1 | rabbit_1 | =INFO REPORT==== 23-Oct-2018::10:43:42 === myfiles_1 | { uri: 'http://serviceapi:8080/api/logs', rabbit_1 | Server startup complete; 6 plugins started. rabbit_1 | rabbitmq_management db_1 | 2018-10-23T10:45:14.671+0000 I - [conn21] end connection 127.0.0.1:56226 (3 connections now open) myfiles_1 | method: 'POST', rabbit_1 | rabbitmq_web_dispatch db_1 | 2018-10-23T10:45:19.932+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56242 #22 (3 connections now open) myfiles_1 | body: '{"source_id":"myfiles","type":"info","message":"Creating fork for the file-watcher process","created_datetime":"2018-10-23 10:43:56.473"}', rabbit_1 | rabbitmq_management_agent db_1 | 2018-10-23T10:45:19.933+0000 I - [conn22] end connection 127.0.0.1:56242 (3 connections now open) db_1 | 2018-10-23T10:45:25.200+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56258 #23 (3 connections now open) myfiles_1 | headers: { 'Content-Type': 'application/json' }, db_1 | 2018-10-23T10:45:25.200+0000 I - [conn23] end connection 127.0.0.1:56258 (3 connections now open) rabbit_1 | amqp_client rabbit_1 | cowboy rabbit_1 | cowlib rabbit_1 | myfiles_1 | callback: [Function: RP$callback], myfiles_1 | transform: undefined, db_1 | 2018-10-23T10:45:30.464+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56274 #24 (3 connections now open) db_1 | 2018-10-23T10:45:30.465+0000 I - [conn24] end connection 127.0.0.1:56274 (3 connections now open) db_1 | 2018-10-23T10:45:35.725+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56290 #25 (3 connections now open) db_1 | 2018-10-23T10:45:35.726+0000 I - [conn25] end connection 127.0.0.1:56290 (3 connections now open) db_1 | 2018-10-23T10:45:40.991+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56306 #26 (3 connections now open) db_1 | 2018-10-23T10:45:40.992+0000 I - [conn26] end connection 127.0.0.1:56306 (3 connections now open) db_1 | 2018-10-23T10:45:46.263+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56320 #27 (3 connections now open) myfiles_1 | simple: true, myfiles_1 | resolveWithFullResponse: false, db_1 | 2018-10-23T10:45:46.263+0000 I - [conn27] end connection 127.0.0.1:56320 (3 connections now open) myfiles_1 | transform2xxOnly: false }, myfiles_1 | response: undefined } db_1 | 2018-10-23T10:45:51.538+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56336 #28 (3 connections now open) rabbit_1 | =INFO REPORT==== 23-Oct-2018::10:43:50 === rabbit_1 | accepting AMQP connection <0.461.0> (172.18.0.6:48822 -> 172.18.0.3:5672) myfiles_1 | (node:6) UnhandledPromiseRejectionWarning: RequestError: Error: connect ECONNREFUSED 172.18.0.6:8080 myfiles_1 | at new RequestError (/node_modules/request-promise-core/lib/errors.js:14:15) myfiles_1 | at Request.plumbing.callback (/node_modules/request-promise-core/lib/plumbing.js:87:29) myfiles_1 | at Request.RP$callback [as _callback] (/node_modules/request-promise-core/lib/plumbing.js:46:31) rabbit_1 | db_1 | 2018-10-23T10:45:51.539+0000 I - [conn28] end connection 127.0.0.1:56336 (3 connections now open) rabbit_1 | =INFO REPORT==== 23-Oct-2018::10:43:50 === rabbit_1 | connection <0.461.0> (172.18.0.6:48822 -> 172.18.0.3:5672): user 'guest' authenticated and granted access to vhost '/' rabbit_1 | rabbit_1 | =INFO REPORT==== 23-Oct-2018::10:43:57 === rabbit_1 | accepting AMQP connection <0.487.0> (172.18.0.7:36496 -> 172.18.0.3:5672) rabbit_1 | myfiles_1 | at self.callback (/node_modules/request/request.js:186:22) db_1 | 2018-10-23T10:45:56.807+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56352 #29 (3 connections now open) rabbit_1 | =INFO REPORT==== 23-Oct-2018::10:43:57 === db_1 | 2018-10-23T10:45:56.807+0000 I - [conn29] end connection 127.0.0.1:56352 (3 connections now open) myfiles_1 | at emitOne (events.js:116:13) rabbit_1 | connection <0.487.0> (172.18.0.7:36496 -> 172.18.0.3:5672): user 'guest' authenticated and granted access to vhost '/' myfiles_1 | at Request.emit (events.js:211:7) db_1 | 2018-10-23T10:46:02.077+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56368 #30 (3 connections now open) db_1 | 2018-10-23T10:46:02.078+0000 I - [conn30] end connection 127.0.0.1:56368 (3 connections now open) myfiles_1 | at Request.onRequestError (/node_modules/request/request.js:878:8) db_1 | 2018-10-23T10:46:07.337+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56384 #31 (3 connections now open) rabbit_1 | myfiles_1 | at emitOne (events.js:116:13) db_1 | 2018-10-23T10:46:07.338+0000 I - [conn31] end connection 127.0.0.1:56384 (3 connections now open) rabbit_1 | =INFO REPORT==== 23-Oct-2018::10:43:58 === myfiles_1 | at ClientRequest.emit (events.js:211:7) db_1 | 2018-10-23T10:46:12.604+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56400 #32 (3 connections now open) rabbit_1 | accepting AMQP connection <0.499.0> (172.18.0.8:55994 -> 172.18.0.3:5672) rabbit_1 | myfiles_1 | at Socket.socketErrorListener (_http_client.js:387:9) rabbit_1 | =INFO REPORT==== 23-Oct-2018::10:43:58 === db_1 | 2018-10-23T10:46:12.607+0000 I - [conn32] end connection 127.0.0.1:56400 (3 connections now open) myfiles_1 | at emitOne (events.js:116:13) rabbit_1 | connection <0.499.0> (172.18.0.8:55994 -> 172.18.0.3:5672): user 'guest' authenticated and granted access to vhost '/' db_1 | 2018-10-23T10:46:17.873+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56416 #33 (3 connections now open) myfiles_1 | at Socket.emit (events.js:211:7) rabbit_1 | myfiles_1 | at emitErrorNT (internal/streams/destroy.js:64:8) db_1 | 2018-10-23T10:46:17.874+0000 I - [conn33] end connection 127.0.0.1:56416 (3 connections now open) rabbit_1 | =INFO REPORT==== 23-Oct-2018::10:43:58 === myfiles_1 | at _combinedTickCallback (internal/process/next_tick.js:138:11) db_1 | 2018-10-23T10:46:23.140+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56432 #34 (3 connections now open) rabbit_1 | accepting AMQP connection <0.496.0> (172.18.0.9:34296 -> 172.18.0.3:5672) myfiles_1 | at process._tickCallback (internal/process/next_tick.js:180:9) db_1 | 2018-10-23T10:46:23.141+0000 I - [conn34] end connection 127.0.0.1:56432 (3 connections now open) rabbit_1 | myfiles_1 | (node:6) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 2) db_1 | 2018-10-23T10:46:28.407+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56448 #35 (3 connections now open) rabbit_1 | =INFO REPORT==== 23-Oct-2018::10:43:58 === rabbit_1 | connection <0.496.0> (172.18.0.9:34296 -> 172.18.0.3:5672): user 'guest' authenticated and granted access to vhost '/' myfiles_1 | [info] File-watcher runs on worker thread myfiles_1 | { RequestError: Error: connect ECONNREFUSED 172.18.0.6:8080 myfiles_1 | at new RequestError (/node_modules/request-promise-core/lib/errors.js:14:15) myfiles_1 | at Request.plumbing.callback (/node_modules/request-promise-core/lib/plumbing.js:87:29) myfiles_1 | at Request.RP$callback [as _callback] (/node_modules/request-promise-core/lib/plumbing.js:46:31) myfiles_1 | at self.callback (/node_modules/request/request.js:186:22) myfiles_1 | at emitOne (events.js:116:13) myfiles_1 | at Request.emit (events.js:211:7) myfiles_1 | at Request.onRequestError (/node_modules/request/request.js:878:8) db_1 | 2018-10-23T10:46:28.408+0000 I - [conn35] end connection 127.0.0.1:56448 (3 connections now open) myfiles_1 | at emitOne (events.js:116:13) db_1 | 2018-10-23T10:46:33.670+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56464 #36 (3 connections now open) db_1 | 2018-10-23T10:46:33.673+0000 I - [conn36] end connection 127.0.0.1:56464 (3 connections now open) db_1 | 2018-10-23T10:46:38.933+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56480 #37 (3 connections now open) db_1 | 2018-10-23T10:46:38.933+0000 I - [conn37] end connection 127.0.0.1:56480 (3 connections now open) db_1 | 2018-10-23T10:46:44.202+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56496 #38 (3 connections now open) db_1 | 2018-10-23T10:46:44.203+0000 I - [conn38] end connection 127.0.0.1:56496 (3 connections now open) myfiles_1 | at ClientRequest.emit (events.js:211:7) myfiles_1 | at Socket.socketErrorListener (_http_client.js:387:9) myfiles_1 | at emitOne (events.js:116:13) db_1 | 2018-10-23T10:46:49.464+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56512 #39 (3 connections now open) myfiles_1 | at Socket.emit (events.js:211:7) myfiles_1 | at emitErrorNT (internal/streams/destroy.js:64:8) myfiles_1 | at _combinedTickCallback (internal/process/next_tick.js:138:11) myfiles_1 | at process._tickCallback (internal/process/next_tick.js:180:9) myfiles_1 | name: 'RequestError', myfiles_1 | message: 'Error: connect ECONNREFUSED 172.18.0.6:8080', db_1 | 2018-10-23T10:46:49.465+0000 I - [conn39] end connection 127.0.0.1:56512 (3 connections now open) db_1 | 2018-10-23T10:46:54.726+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56528 #40 (3 connections now open) db_1 | 2018-10-23T10:46:54.727+0000 I - [conn40] end connection 127.0.0.1:56528 (3 connections now open) myfiles_1 | cause: myfiles_1 | { Error: connect ECONNREFUSED 172.18.0.6:8080 myfiles_1 | at Object._errnoException (util.js:1022:11) myfiles_1 | at _exceptionWithHostPort (util.js:1044:20) db_1 | 2018-10-23T10:46:59.995+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56544 #41 (3 connections now open) db_1 | 2018-10-23T10:46:59.999+0000 I - [conn41] end connection 127.0.0.1:56544 (3 connections now open) db_1 | 2018-10-23T10:47:05.261+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56560 #42 (3 connections now open) myfiles_1 | at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1198:14) db_1 | 2018-10-23T10:47:05.261+0000 I - [conn42] end connection 127.0.0.1:56560 (3 connections now open) myfiles_1 | code: 'ECONNREFUSED', db_1 | 2018-10-23T10:47:10.524+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56576 #43 (3 connections now open) myfiles_1 | errno: 'ECONNREFUSED', myfiles_1 | syscall: 'connect', myfiles_1 | address: '172.18.0.6', myfiles_1 | port: 8080 }, myfiles_1 | error: myfiles_1 | { Error: connect ECONNREFUSED 172.18.0.6:8080 myfiles_1 | at Object._errnoException (util.js:1022:11) myfiles_1 | at _exceptionWithHostPort (util.js:1044:20) db_1 | 2018-10-23T10:47:10.526+0000 I - [conn43] end connection 127.0.0.1:56576 (3 connections now open) db_1 | 2018-10-23T10:47:15.788+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56592 #44 (3 connections now open) db_1 | 2018-10-23T10:47:15.788+0000 I - [conn44] end connection 127.0.0.1:56592 (3 connections now open) db_1 | 2018-10-23T10:47:21.054+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56608 #45 (3 connections now open) db_1 | 2018-10-23T10:47:21.054+0000 I - [conn45] end connection 127.0.0.1:56608 (3 connections now open) db_1 | 2018-10-23T10:47:26.316+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56624 #46 (3 connections now open) db_1 | 2018-10-23T10:47:26.316+0000 I - [conn46] end connection 127.0.0.1:56624 (3 connections now open) db_1 | 2018-10-23T10:47:31.581+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56676 #47 (3 connections now open) db_1 | 2018-10-23T10:47:31.583+0000 I - [conn47] end connection 127.0.0.1:56676 (3 connections now open) myfiles_1 | at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1198:14) myfiles_1 | code: 'ECONNREFUSED', myfiles_1 | errno: 'ECONNREFUSED', myfiles_1 | syscall: 'connect', db_1 | 2018-10-23T10:47:36.845+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56692 #48 (3 connections now open) db_1 | 2018-10-23T10:47:36.846+0000 I - [conn48] end connection 127.0.0.1:56692 (3 connections now open) db_1 | 2018-10-23T10:47:42.108+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56708 #49 (3 connections now open) db_1 | 2018-10-23T10:47:42.109+0000 I - [conn49] end connection 127.0.0.1:56708 (3 connections now open) db_1 | 2018-10-23T10:47:47.376+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56724 #50 (3 connections now open) myfiles_1 | address: '172.18.0.6', myfiles_1 | port: 8080 }, myfiles_1 | options: db_1 | 2018-10-23T10:47:47.377+0000 I - [conn50] end connection 127.0.0.1:56724 (3 connections now open) myfiles_1 | { uri: 'http://serviceapi:8080/api/logs', myfiles_1 | method: 'POST', db_1 | 2018-10-23T10:47:52.642+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56740 #51 (3 connections now open) db_1 | 2018-10-23T10:47:52.643+0000 I - [conn51] end connection 127.0.0.1:56740 (3 connections now open) db_1 | 2018-10-23T10:47:57.913+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56756 #52 (3 connections now open) myfiles_1 | body: '{"source_id":"myfiles","type":"info","message":"File-watcher runs on worker thread","created_datetime":"2018-10-23 10:43:57.444"}', db_1 | 2018-10-23T10:47:57.913+0000 I - [conn52] end connection 127.0.0.1:56756 (3 connections now open) myfiles_1 | headers: { 'Content-Type': 'application/json' }, db_1 | 2018-10-23T10:48:03.179+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56772 #53 (3 connections now open) myfiles_1 | callback: [Function: RP$callback], myfiles_1 | transform: undefined, db_1 | 2018-10-23T10:48:03.179+0000 I - [conn53] end connection 127.0.0.1:56772 (3 connections now open) myfiles_1 | simple: true, myfiles_1 | resolveWithFullResponse: false, myfiles_1 | transform2xxOnly: false }, myfiles_1 | response: undefined } myfiles_1 | (node:12) UnhandledPromiseRejectionWarning: RequestError: Error: connect ECONNREFUSED 172.18.0.6:8080 myfiles_1 | at new RequestError (/node_modules/request-promise-core/lib/errors.js:14:15) db_1 | 2018-10-23T10:48:08.443+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56786 #54 (3 connections now open) db_1 | 2018-10-23T10:48:08.445+0000 I - [conn54] end connection 127.0.0.1:56786 (3 connections now open) db_1 | 2018-10-23T10:48:13.716+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56802 #55 (3 connections now open) db_1 | 2018-10-23T10:48:13.716+0000 I - [conn55] end connection 127.0.0.1:56802 (3 connections now open) db_1 | 2018-10-23T10:48:18.977+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56818 #56 (3 connections now open) db_1 | 2018-10-23T10:48:18.977+0000 I - [conn56] end connection 127.0.0.1:56818 (3 connections now open) myfiles_1 | at Request.plumbing.callback (/node_modules/request-promise-core/lib/plumbing.js:87:29) myfiles_1 | at Request.RP$callback [as _callback] (/node_modules/request-promise-core/lib/plumbing.js:46:31) db_1 | 2018-10-23T10:48:24.244+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56834 #57 (3 connections now open) db_1 | 2018-10-23T10:48:24.244+0000 I - [conn57] end connection 127.0.0.1:56834 (3 connections now open) myfiles_1 | at self.callback (/node_modules/request/request.js:186:22) db_1 | 2018-10-23T10:48:29.511+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56850 #58 (3 connections now open) myfiles_1 | at emitOne (events.js:116:13) myfiles_1 | at Request.emit (events.js:211:7) myfiles_1 | at Request.onRequestError (/node_modules/request/request.js:878:8) db_1 | 2018-10-23T10:48:29.512+0000 I - [conn58] end connection 127.0.0.1:56850 (3 connections now open) db_1 | 2018-10-23T10:48:34.773+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56866 #59 (3 connections now open) db_1 | 2018-10-23T10:48:34.776+0000 I - [conn59] end connection 127.0.0.1:56866 (3 connections now open) db_1 | 2018-10-23T10:48:40.038+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56882 #60 (3 connections now open) myfiles_1 | at emitOne (events.js:116:13) db_1 | 2018-10-23T10:48:40.038+0000 I - [conn60] end connection 127.0.0.1:56882 (3 connections now open) db_1 | 2018-10-23T10:48:45.307+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56898 #61 (3 connections now open) myfiles_1 | at ClientRequest.emit (events.js:211:7) myfiles_1 | at Socket.socketErrorListener (_http_client.js:387:9) myfiles_1 | at emitOne (events.js:116:13) db_1 | 2018-10-23T10:48:45.307+0000 I - [conn61] end connection 127.0.0.1:56898 (3 connections now open) db_1 | 2018-10-23T10:48:50.569+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56914 #62 (3 connections now open) db_1 | 2018-10-23T10:48:50.570+0000 I - [conn62] end connection 127.0.0.1:56914 (3 connections now open) myfiles_1 | at Socket.emit (events.js:211:7) myfiles_1 | at emitErrorNT (internal/streams/destroy.js:64:8) myfiles_1 | at _combinedTickCallback (internal/process/next_tick.js:138:11) db_1 | 2018-10-23T10:48:55.832+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56930 #63 (3 connections now open) db_1 | 2018-10-23T10:48:55.833+0000 I - [conn63] end connection 127.0.0.1:56930 (3 connections now open) db_1 | 2018-10-23T10:49:01.095+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56946 #64 (3 connections now open) db_1 | 2018-10-23T10:49:01.097+0000 I - [conn64] end connection 127.0.0.1:56946 (3 connections now open) db_1 | 2018-10-23T10:49:06.365+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56962 #65 (3 connections now open) db_1 | 2018-10-23T10:49:06.366+0000 I - [conn65] end connection 127.0.0.1:56962 (3 connections now open) myfiles_1 | at process._tickCallback (internal/process/next_tick.js:180:9) db_1 | 2018-10-23T10:49:11.632+0000 I NETWORK [thread1] connection accepted from 127.0.0.1:56978 #66 (3 connections now open) myfiles_1 | (node:12) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1) myfiles_1 | (node:12) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code. db_1 | 2018-10-23T10:49:11.632+0000 I - [conn66] end connection 127.0.0.1:56978 (3 connections now open)

stale[bot] commented 6 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

jerome83136 commented 5 years ago

Hi, +1 for file woanloading feature addition :-)

I agree: nice piece of software !!

stale[bot] commented 5 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.