search

usc-isi-i2 / dig-etl-engine

Download DIG to run on your laptop or server.
http://usc-isi-i2.github.io/dig/
MIT License
101 stars 39 forks source link

connection refused error while running engine.sh for the first time #259

Closed sobalgi closed 5 years ago

sobalgi commented 5 years ago

I have just installed the dig-etl-engine. I tried running engine.sh as mentioned in the Installation guide to start myDIG. Every time I try to run this I keep getting this error

image

Any help in this regard is very much appreciated.

Many Thanks, Sourabh Balgi IISc, Bangalore

saggu commented 5 years ago

Hi @sobalgi, please try using the development branch. What is your operating system, there are additional instructions for unix systems, take a look there as well. Also, please list specifications of your machine you are trying to run myDIG on.

sobalgi commented 5 years ago

Hello Mr. Singh,

Thanks a lot for your quick response. Really appreciate it.

I'm running Ubuntu/Linux on a high performance cluster.

Please find the details you requested for in the transcript below.

I believe the requirements were mentioned as atleast 8GB memory and nothing related to architectures of the machine.

Please let me know if you need any additional info. I'll be happy to share it with you.

bash-4.4$ ssh sourabh@10.192.30.14 sourabh@10.192.30.14's password: Welcome to Ubuntu 16.04.3 LTS (GNU/Linux 4.4.0-127-generic x86_64)

334 packages can be updated. 0 updates are security updates.

New release '18.04.1 LTS' available. Run 'do-release-upgrade' to upgrade to it.

System restart required Last login: Sat Aug 25 22:27:07 2018 from 10.3.1.235 sourabh@mirzakhani:~$ uname -a Linux mirzakhani 4.4.0-127-generic #153-Ubuntu SMP Sat May 19 10:58:46 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

sourabh@mirzakhani:~$ lshw -short WARNING: you should run this program as super-user. H/W path Device Class Description

                    system         Computer

/0 bus Motherboard /0/0 memory 31GiB System memory /0/1 processor Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz /0/2 processor Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz /0/100 bridge Xeon E7 v3/Xeon E5 v3/Core i7 DMI2 /0/100/1 bridge Xeon E7 v3/Xeon E5 v3/Core i7 PCI Express Ro /0/100/1/0 storage SAS3008 PCI-Express Fusion-MPT SAS-3 /0/100/1.1 bridge Xeon E7 v3/Xeon E5 v3/Core i7 PCI Express Ro /0/100/2 bridge Xeon E7 v3/Xeon E5 v3/Core i7 PCI Express Ro /0/100/2/0 display NVIDIA Corporation /0/100/2/0.1 multimedia NVIDIA Corporation /0/100/3 bridge Xeon E7 v3/Xeon E5 v3/Core i7 PCI Express Ro /0/100/3/0 display NVIDIA Corporation /0/100/3/0.1 multimedia NVIDIA Corporation /0/100/5 generic Xeon E7 v3/Xeon E5 v3/Core i7 Address Map, V /0/100/5.1 generic Xeon E7 v3/Xeon E5 v3/Core i7 Hot Plug /0/100/5.2 generic Xeon E7 v3/Xeon E5 v3/Core i7 RAS, Control S /0/100/5.4 generic Xeon E7 v3/Xeon E5 v3/Core i7 I/O APIC /0/100/11 generic C610/X99 series chipset SPSR /0/100/14 bus C610/X99 series chipset USB xHCI Host Contro /0/100/16 communication C610/X99 series chipset MEI Controller #1 /0/100/19 enp0s25 network Ethernet Connection I217-LM /0/100/1a bus C610/X99 series chipset USB Enhanced Host Co /0/100/1b multimedia C610/X99 series chipset HD Audio Controller /0/100/1c bridge C610/X99 series chipset PCI Express Root Por /0/100/1c.1 bridge C610/X99 series chipset PCI Express Root Por /0/100/1c.1/0 enp6s0 network I210 Gigabit Network Connection /0/100/1c.3 bridge C610/X99 series chipset PCI Express Root Por /0/100/1c.3/0 bridge XIO2001 PCI Express-to-PCI Bridge /0/100/1d bus C610/X99 series chipset USB Enhanced Host Co /0/100/1f bridge C610/X99 series chipset LPC Controller /0/100/1f.2 storage C610/X99 series chipset 6-Port SATA Controll /0/100/1f.3 bus C610/X99 series chipset SMBus Controller /0/3 generic Xeon E7 v3/Xeon E5 v3/Core i7 QPI Link 0 /0/4 generic Xeon E7 v3/Xeon E5 v3/Core i7 QPI Link 0 /0/6 generic Xeon E7 v3/Xeon E5 v3/Core i7 QPI Link 0 /0/7 generic Xeon E7 v3/Xeon E5 v3/Core i7 QPI Link 1 /0/8 generic Xeon E7 v3/Xeon E5 v3/Core i7 QPI Link 1 /0/9 generic Xeon E7 v3/Xeon E5 v3/Core i7 QPI Link 1 /0/a generic Xeon E7 v3/Xeon E5 v3/Core i7 R3 QPI Link 0 /0/b generic Xeon E7 v3/Xeon E5 v3/Core i7 R3 QPI Link 0 /0/c generic Xeon E7 v3/Xeon E5 v3/Core i7 R3 QPI Link 0 /0/d generic Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Regist /0/e generic Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Regist /0/f generic Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Regist /0/10 generic Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Regist /0/11 generic Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Regist /0/12 generic Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Regist /0/13 generic Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Regist /0/14 generic Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Regist /0/15 generic Xeon E7 v3/Xeon E5 v3/Core i7 Buffered Ring /0/16 generic Xeon E7 v3/Xeon E5 v3/Core i7 Buffered Ring /0/17 generic Xeon E7 v3/Xeon E5 v3/Core i7 System Address /0/18 generic Xeon E7 v3/Xeon E5 v3/Core i7 System Address /0/19 generic Xeon E7 v3/Xeon E5 v3/Core i7 System Address /0/1a generic Xeon E7 v3/Xeon E5 v3/Core i7 PCIe Ring Inte /0/1b generic Xeon E7 v3/Xeon E5 v3/Core i7 PCIe Ring Inte /0/1c generic Xeon E7 v3/Xeon E5 v3/Core i7 Scratchpad & S /0/1d generic Xeon E7 v3/Xeon E5 v3/Core i7 Scratchpad & S /0/1e generic Xeon E7 v3/Xeon E5 v3/Core i7 Scratchpad & S /0/1f generic Xeon E7 v3/Xeon E5 v3/Core i7 Home Agent 0 /0/20 generic Xeon E7 v3/Xeon E5 v3/Core i7 Home Agent 0 /0/21 generic Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Mem /0/22 generic Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Mem /0/23 generic Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Mem /0/24 generic Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Mem /0/25 generic Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Mem /0/26 generic Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Mem /0/27 generic Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO Channel /0/28 generic Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO Global B /0/29 generic Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Mem /0/2a generic Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Mem /0/2b generic Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Mem /0/2c generic Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Mem /0/2d generic Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO (VMSE) 0 /0/2e generic Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO (VMSE) 0 /0/2f generic Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO (VMSE) 0 /0/30 generic Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO (VMSE) 0 /0/31 generic Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Mem /0/32 generic Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Mem /0/33 generic Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Mem /0/34 generic Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Mem /0/35 generic Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Mem /0/36 generic Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO Channel /0/37 generic Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO Global B /0/38 generic Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Mem /0/39 generic Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO (VMSE) 2 /0/3a generic Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO (VMSE) 2 /0/3b generic Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO (VMSE) 2 /0/3c generic Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO (VMSE) 2 /0/3d generic Xeon E7 v3/Xeon E5 v3/Core i7 Power Control /0/3e generic Xeon E7 v3/Xeon E5 v3/Core i7 Power Control /0/3f generic Xeon E7 v3/Xeon E5 v3/Core i7 Power Control /0/40 generic Xeon E7 v3/Xeon E5 v3/Core i7 Power Control /0/41 generic Xeon E7 v3/Xeon E5 v3/Core i7 Power Control /0/42 generic Xeon E7 v3/Xeon E5 v3/Core i7 VCU /0/43 generic Xeon E7 v3/Xeon E5 v3/Core i7 VCU /0/5 generic Xeon E7 v3/Xeon E5 v3/Core i7 Address Map, V /0/5.1 generic Xeon E7 v3/Xeon E5 v3/Core i7 Hot Plug /0/5.2 generic Xeon E7 v3/Xeon E5 v3/Core i7 RAS, Control S /0/5.4 generic Xeon E7 v3/Xeon E5 v3/Core i7 I/O APIC /0/44 generic Xeon E7 v3/Xeon E5 v3/Core i7 QPI Link 0 /0/45 generic Xeon E7 v3/Xeon E5 v3/Core i7 QPI Link 0 /0/46 generic Xeon E7 v3/Xeon E5 v3/Core i7 QPI Link 0 /0/47 generic Xeon E7 v3/Xeon E5 v3/Core i7 QPI Link 1 /0/48 generic Xeon E7 v3/Xeon E5 v3/Core i7 QPI Link 1 /0/49 generic Xeon E7 v3/Xeon E5 v3/Core i7 QPI Link 1 /0/4a generic Xeon E7 v3/Xeon E5 v3/Core i7 R3 QPI Link 0 /0/4b generic Xeon E7 v3/Xeon E5 v3/Core i7 R3 QPI Link 0 /0/4c generic Xeon E7 v3/Xeon E5 v3/Core i7 R3 QPI Link 0 /0/4d generic Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Regist /0/4e generic Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Regist /0/4f generic Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Regist /0/50 generic Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Regist /0/51 generic Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Regist /0/52 generic Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Regist /0/53 generic Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Regist /0/54 generic Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Regist /0/55 generic Xeon E7 v3/Xeon E5 v3/Core i7 Buffered Ring /0/56 generic Xeon E7 v3/Xeon E5 v3/Core i7 Buffered Ring /0/57 generic Xeon E7 v3/Xeon E5 v3/Core i7 System Address /0/58 generic Xeon E7 v3/Xeon E5 v3/Core i7 System Address /0/59 generic Xeon E7 v3/Xeon E5 v3/Core i7 System Address /0/5a generic Xeon E7 v3/Xeon E5 v3/Core i7 PCIe Ring Inte /0/5b generic Xeon E7 v3/Xeon E5 v3/Core i7 PCIe Ring Inte /0/5c generic Xeon E7 v3/Xeon E5 v3/Core i7 Scratchpad & S /0/5d generic Xeon E7 v3/Xeon E5 v3/Core i7 Scratchpad & S /0/5e generic Xeon E7 v3/Xeon E5 v3/Core i7 Scratchpad & S /0/5f generic Xeon E7 v3/Xeon E5 v3/Core i7 Home Agent 0 /0/60 generic Xeon E7 v3/Xeon E5 v3/Core i7 Home Agent 0 /0/61 generic Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Mem /0/62 generic Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Mem /0/63 generic Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Mem /0/64 generic Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Mem /0/65 generic Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Mem /0/66 generic Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Mem /0/67 generic Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO Channel /0/68 generic Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO Global B /0/69 generic Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Mem /0/6a generic Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Mem /0/6b generic Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Mem /0/6c generic Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Mem /0/6d generic Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO (VMSE) 0 /0/6e generic Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO (VMSE) 0 /0/6f generic Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO (VMSE) 0 /0/70 generic Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO (VMSE) 0 /0/71 generic Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Mem /0/72 generic Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Mem /0/73 generic Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Mem /0/74 generic Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Mem /0/75 generic Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Mem /0/76 generic Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO Channel /0/77 generic Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO Global B /0/78 generic Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Mem /0/79 generic Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO (VMSE) 2 /0/7a generic Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO (VMSE) 2 /0/7b generic Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO (VMSE) 2 /0/7c generic Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO (VMSE) 2 /0/7d generic Xeon E7 v3/Xeon E5 v3/Core i7 Power Control /0/7e generic Xeon E7 v3/Xeon E5 v3/Core i7 Power Control /0/7f generic Xeon E7 v3/Xeon E5 v3/Core i7 Power Control /0/80 generic Xeon E7 v3/Xeon E5 v3/Core i7 Power Control /0/81 generic Xeon E7 v3/Xeon E5 v3/Core i7 Power Control /0/82 generic Xeon E7 v3/Xeon E5 v3/Core i7 VCU /0/83 generic Xeon E7 v3/Xeon E5 v3/Core i7 VCU WARNING: output may be incomplete or inaccurate, you should run this program as super-user.

sourabh@mirzakhani:~$ lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 32 On-line CPU(s) list: 0-31 Thread(s) per core: 2 Core(s) per socket: 8 Socket(s): 2 NUMA node(s): 2 Vendor ID: GenuineIntel CPU family: 6 Model: 63 Model name: Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz Stepping: 2 CPU MHz: 2397.281 CPU max MHz: 3200.0000 CPU min MHz: 1200.0000 BogoMIPS: 4790.67 Virtualization: VT-x L1d cache: 32K L1i cache: 32K L2 cache: 256K L3 cache: 20480K NUMA node0 CPU(s): 0-7,16-23 NUMA node1 CPU(s): 8-15,24-31 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb invpcid_single kaiser tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc ibpb ibrs stibp dtherm ida arat pln pts

sourabh@mirzakhani:~$ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 1.8T 0 disk ├─sda1 8:1 0 232.9G 0 part / ├─sda2 8:2 0 18.6G 0 part [SWAP] └─sda3 8:3 0 1.6T 0 part sr0 11:0 1 1024M 0 rom sourabh@mirzakhani:~$

Many thanks in advance.

Best regards, Sourabh Balgi. IISc, Bangalore.

On Sat 25 Aug, 2018, 10:31 PM Amandeep Singh, notifications@github.com wrote:

Hi @sobalgi https://github.com/sobalgi, please try using the development branch. What is your operating system, there are additional instructions for unix systems, take a look there as well. Also, please list specifications of your machine you are trying to run myDIG on.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/usc-isi-i2/dig-etl-engine/issues/259#issuecomment-415982897, or mute the thread https://github.com/notifications/unsubscribe-auth/AjKwiP5652Xiz_lteqUw7ij9e3nejg8Xks5uUYLngaJpZM4WMZgw .

saggu commented 5 years ago

Ok, machine specs are not an issue. Try running from the development branch and there are a couple of special instructions for unix based systems. Make sure you have followed all the instructions very carefully. If it still doesn't work, let me know. I will take a look later.

Thanks

sobalgi commented 5 years ago

As indicated, I switched to development branch and followed the readme.md installation.

I checked out the development into seperate project with -dev suffix. I created a new project folder too. I also modified the .env file as mentioned in the instructions.

I'm getting a new error this time around with ipv4 address. Please find the screenshot below.

On Sat 25 Aug, 2018, 11:02 PM Amandeep Singh, notifications@github.com wrote:

Ok, machine specs are not an issue. Try running from the development branch and there are a couple of special instructions for unix based systems. Make sure you have followed all the instructions very carefully. If it still doesn't work, let me know. I will take a look later.

Thanks

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/usc-isi-i2/dig-etl-engine/issues/259#issuecomment-415984769, or mute the thread https://github.com/notifications/unsubscribe-auth/AjKwiIhdnYOEAhd36uaAWRt6Bv2OGDzVks5uUYoWgaJpZM4WMZgw .

saggu commented 5 years ago

That looks like the conflict from the other dig-etl-engine repo,

run ./engine.sh down from dig-etl-engine and dig-etl-engine-dev repos

This should ideally fix it, if not, then run

docker network ls from anywhere and then find the network named like digdev_dig_net And then docker network rm <network_id>

sobalgi commented 5 years ago

Thanks. The fix you suggested for ipv4 worked fine. But again I have the same error with the engine.sh . All the files are downloaded and extracted fine. But I get the error when the docker is creating the network digdev_dig_net.

Please find the screenshot attached.

I rechecked if I was on the right development brach again just to make sure. Everything is fine with the branches but I still get the same error.

Thanks a lot for your quick replies again. Best regards, Sourabh.

On Sun 26 Aug, 2018, 12:04 AM Amandeep Singh, notifications@github.com wrote:

That looks like the conflict from the other dig-etl-engine repo,

run ./engine.sh down from dig-etl-engine and dig-etl-engine-dev repos

This should ideally fix it, if not, then run

docker network ls from anywhere and then find the network named like digdev_dig_net And then docker network rm

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/usc-isi-i2/dig-etl-engine/issues/259#issuecomment-415988293, or mute the thread https://github.com/notifications/unsubscribe-auth/AjKwiAXeOrD-oMjYPVjkeQg1dcfB4UfDks5uUZiogaJpZM4WMZgw .

sobalgi commented 5 years ago

Hello Mr. Singh,

I'm able to run the dig engine now. I had to restart the docker to make it run fine.

Thank you for all your solutions. It really helped a lot. I'll try out the example project and check how the system works.

Best regards, Sourabh Balgi.

On Sun 26 Aug, 2018, 12:50 AM Sourabh Balgi, sourabhbalgi@gmail.com wrote:

Thanks. The fix you suggested for ipv4 worked fine. But again I have the same error with the engine.sh . All the files are downloaded and extracted fine. But I get the error when the docker is creating the network digdev_dig_net.

Please find the screenshot attached.

I rechecked if I was on the right development brach again just to make sure. Everything is fine with the branches but I still get the same error.

Thanks a lot for your quick replies again. Best regards, Sourabh.

On Sun 26 Aug, 2018, 12:04 AM Amandeep Singh, notifications@github.com wrote:

That looks like the conflict from the other dig-etl-engine repo,

run ./engine.sh down from dig-etl-engine and dig-etl-engine-dev repos

This should ideally fix it, if not, then run

docker network ls from anywhere and then find the network named like digdev_dig_net And then docker network rm

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/usc-isi-i2/dig-etl-engine/issues/259#issuecomment-415988293, or mute the thread https://github.com/notifications/unsubscribe-auth/AjKwiAXeOrD-oMjYPVjkeQg1dcfB4UfDks5uUZiogaJpZM4WMZgw .

saggu commented 5 years ago

Closing this issue. @sobalgi if you see any other issue, please create a new one