gilestrolab / ethoscope

a platform from monitoring animal behaviour in real time from a raspberry pi
http://lab.gilest.ro/ethoscope/
GNU General Public License v3.0
17 stars 25 forks source link

Ethoscope not detected by the node #111

Closed DenaFrancis closed 4 years ago

DenaFrancis commented 4 years ago

Hi, I am using the latest raspberry pi (Raspberry Pi v3) with the current software (20200216_ethoscope_000.img) running on a 32GB samsung evo SD card inserted.

I think I have successfully managed to run the node through the 4GB raspberry pi 4 connected to a basic netgear router. So far it seems that the node is running fine, but when I boot up the raspberry pi and see the red led is continuously on. However, I don't see that the device has been detected by the node. I have tried to plug the ethernet cable into my internet modem, but this does not solve the issue. I am sure that I am missing something, as I am not that computer savvy (just a fly biologist).

Here are some screenshots of the what the node says:

Screen Shot 2020-03-26 at 1 52 57 pm Screen Shot 2020-03-26 at 1 52 01 pm

Any help would be greatly appreciated! Thanks

ggilestro commented 4 years ago

The easiest way to debug this issue is to connect the ethoscope to a monitor & keyboard, then boot it. This will tell you on screen if the ethoscope is starting correctly and whether it gets its own appropriate IP via wifi.

DenaFrancis commented 4 years ago

Thank you. I don’t know if this will help too, I hooked the ethoscope up to my internet and got the following information when I looked up my internet router.

I can see that both the ethoscope and the node are present and have an IP address.

On 26 Mar 2020, at 9:17 pm, Giorgio Gilestro notifications@github.com wrote:

The easiest way to debug this issue is to connect the ethoscope to a monitor & keyboard, then boot it. This will tell you on screen if the ethoscope is starting correctly and whether it gets its own appropriate IP via wifi.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/gilestrolab/ethoscope/issues/111#issuecomment-604372114, or unsubscribe https://github.com/notifications/unsubscribe-auth/AENPXYJ7P46HXW4WKAFQMG3RJM2TZANCNFSM4LT543PA.

DenaFrancis commented 4 years ago

Okay, I have now booted the ethoscope with a keyboard and a monitor and the final display says:

Arch Linux 4.19.102-1-ARCH (tty1)

It asks me for a login : ETHOSCOPE000 login:

But then it automatically displayed this:

ETHOSCOPE000 login: [ 30.010543] audit: type=1130 audit (1585269076.099:36): pid=1 uid=0 auid=4294967295 ses=4294967295 msg=‘unit=mariadb comm=“systemd” exe=“/usr/lib/systemd” hostname=? addr=? terminal=? res=success’ [34.138520] audit: type=1130 audit (1585269080.219:37): pid=1 uid=0 auid=4294967295 ses=4294967295 msg=‘unit=mariadb comm=“systemd” exe=“/usr/lib/systemd” hostname=? addr=? terminal=? res=success’ [34.138847] audit: type=1130 audit (1585269080.229:38): pid=1 uid=0 auid=4294967295 ses=4294967295 msg=‘unit=mariadb comm=“systemd” exe=“/usr/lib/systemd” hostname=? addr=? terminal=? res=success’

When I tried to find out the IP address, I typed ls /sys/class/net It asks me for a login again : ETHOSCOPE000 login: Have I missed the login details somewhere?

I don’t want to mess something up so I am not sure how to access the information about the IP address from here.

On 26 Mar 2020, at 10:06 pm, Deanne Francis francis.dena@gmail.com wrote:

Thank you. I don’t know if this will help too, I hooked the ethoscope up to my internet and got the following information when I looked up my internet router.

<Screen Shot 2020-03-26 at 5.50.52 pm.png>

I can see that both the ethoscope and the node are present and have an IP address.

On 26 Mar 2020, at 9:17 pm, Giorgio Gilestro <notifications@github.com mailto:notifications@github.com> wrote:

The easiest way to debug this issue is to connect the ethoscope to a monitor & keyboard, then boot it. This will tell you on screen if the ethoscope is starting correctly and whether it gets its own appropriate IP via wifi.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/gilestrolab/ethoscope/issues/111#issuecomment-604372114, or unsubscribe https://github.com/notifications/unsubscribe-auth/AENPXYJ7P46HXW4WKAFQMG3RJM2TZANCNFSM4LT543PA.

DenaFrancis commented 4 years ago

I did manage to figure out the login and password for this using the arch linux login info. I used login root and password root, which seemed to work.

When I then type in IP address show I get this display: 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc no queue state UNKNOWN group default glen 1000 link <loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_code1 state DOWN group default qlen 1000 link/ether b8:27:eb:ef:b8:af brd ff:ff:ff:ff:ff:ff 3: wlan0:<NO-CARRIER,BROADCAST,MULTICAST,UP>mtu 1500 qdisc fq_code1 state DOWN group default qlen 1000 link/ether b8:27:eb:ba:ed:fa brd ff:ff:ff:ff:ff:ff At the moment I don’t have it connected to an ethernet cable to the router or the internet. So I am guessing that lo is the wifi?

Either this was not the right command, or there is no IP assigned. Thank you

On 27 Mar 2020, at 11:21 am, Deanne Francis francis.dena@gmail.com wrote:

Okay, I have now booted the ethoscope with a keyboard and a monitor and the final display says:

Arch Linux 4.19.102-1-ARCH (tty1)

It asks me for a login : ETHOSCOPE000 login:

But then it automatically displayed this:

ETHOSCOPE000 login: [ 30.010543] audit: type=1130 audit (1585269076.099:36): pid=1 uid=0 auid=4294967295 ses=4294967295 msg=‘unit=mariadb comm=“systemd” exe=“/usr/lib/systemd” hostname=? addr=? terminal=? res=success’ [34.138520] audit: type=1130 audit (1585269080.219:37): pid=1 uid=0 auid=4294967295 ses=4294967295 msg=‘unit=mariadb comm=“systemd” exe=“/usr/lib/systemd” hostname=? addr=? terminal=? res=success’ [34.138847] audit: type=1130 audit (1585269080.229:38): pid=1 uid=0 auid=4294967295 ses=4294967295 msg=‘unit=mariadb comm=“systemd” exe=“/usr/lib/systemd” hostname=? addr=? terminal=? res=success’

When I tried to find out the IP address, I typed ls /sys/class/net It asks me for a login again : ETHOSCOPE000 login: Have I missed the login details somewhere?

I don’t want to mess something up so I am not sure how to access the information about the IP address from here.

On 26 Mar 2020, at 10:06 pm, Deanne Francis <francis.dena@gmail.com mailto:francis.dena@gmail.com> wrote:

Thank you. I don’t know if this will help too, I hooked the ethoscope up to my internet and got the following information when I looked up my internet router.

<Screen Shot 2020-03-26 at 5.50.52 pm.png>

I can see that both the ethoscope and the node are present and have an IP address.

On 26 Mar 2020, at 9:17 pm, Giorgio Gilestro <notifications@github.com mailto:notifications@github.com> wrote:

The easiest way to debug this issue is to connect the ethoscope to a monitor & keyboard, then boot it. This will tell you on screen if the ethoscope is starting correctly and whether it gets its own appropriate IP via wifi.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/gilestrolab/ethoscope/issues/111#issuecomment-604372114, or unsubscribe https://github.com/notifications/unsubscribe-auth/AENPXYJ7P46HXW4WKAFQMG3RJM2TZANCNFSM4LT543PA.

DenaFrancis commented 4 years ago

When I try to find the hosts by typing: getent hosts I get: 127.0.0.1 localhost 192.168.0.100 node

pepelisu commented 4 years ago

Hi, LO is the localhost, and is not a network device. eth0 is the ethernet interface and wlan0 is the wifi. As I can see in your output you do not have network connection in any of the devices. Please check that the router is configured as expected, connected to node and serving wifi with the expected SSID and Password. SSID: ETHOSCOPE_WIFI, WPA-KEY: ETHOSCOPE_1234

DenaFrancis commented 4 years ago

I configured the router to the ethoscope ssid and key. When I go into ethoscope000 and type in ip address show

the wlan0 is changed to <BROADCAST, MULTICAST, UP,LOWER_UP> mtu 1500 mtu 1500 qdisc fq_code1 state UP group default qlen 1000 link/ether b8:27:eb:ba:ed:fa brd ff:ff:ff:ff:ff:ff inet 192.168.0.2/24 brd 192.168.0.255 scope global dynamic noprefixroute wlan0 valid_lft 86153sec preffered_lft 75353sec inet6 fe80:: ba27:ebff:feba:edfa/64 scope link valid_lft forever preferred_lft forever

When I check ifconfig wlan0: wlan0: flags =4163<UP, BROADCAST, RUNNING,MULTICAST> mtu 1500 inet 192.168.0.2 netmask 255.255.255.0 broadcast 192.168.0.255

When I log onto the node web server, I still see that the platform is running, but there are still no devices detected.

When I try to ping the node (not sure if this is what I should be doing) I get PING node (192.168.0.100) 56(84) bytes of data From ETHOSCOPE000 (192.168.0.2) icmp_seq=1 destination host unreachable

ggilestro commented 4 years ago

Ok so after configuring the WIFI your ethoscope now gets a proper ip 192.168.0.2 and your node gets 192.168.0.10. If they can ping each other you should be ablet get the system running as it should. Just restart both perhaps?

DenaFrancis commented 4 years ago

Yes, I tried to ping the node from my ethoscope and it showed that the node didn’t get the message. I haven’t tried it the other way around (ping the ethoscope from the node).

PING node (192.168.0.100) 56(84) bytes of data From ETHOSCOPE000 (192.168.0.2) icmp_seq=1 destination host unreachable

I also tried to ping 192.168.0.10 (I am not sure why the ethoscope sees the node IP address as 192.168.0.100, while it says 192.168.0.10 as the IP address on the node server) and I got the same result as above

I had restarted both the ethoscope and node after the router configuration, but I still don’t see the node server detecting the ethoscope.

I also tried to directly connect the two raspberry pi’s via usb cables. But again it didn’t show up. I will try to ping from the node to the ethoscope, but I am not sure what else I can do. It’s probably something I am not seeing that might be very easy to solve.

On 28 Mar 2020, at 9:36 pm, Giorgio Gilestro notifications@github.com wrote:

Ok so after configuring the WIFI your ethoscope now gets a proper ip 192.168.0.2 and your node gets 192.168.0.10. If they can ping each other you should be ablet get the system running as it should. Just restart both perhaps?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/gilestrolab/ethoscope/issues/111#issuecomment-605434692, or unsubscribe https://github.com/notifications/unsubscribe-auth/AENPXYMDKLVBHQRY3SMUSADRJXOKRANCNFSM4LT543PA.

DenaFrancis commented 4 years ago

So I have now looked at the node and when I look at the ip address show this is what happens:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc no queue state UNKNOWN group default glen 1000 link <loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_code1 state UP group default qlen 1000 link/ether b8:27:eb:ef:b8:af brd ff:ff:ff:ff:ff:ff inet 192.168.0.10/24 brd 192.168.0.255 scope global dynamic eth0 ** The raspberry pi4 is connected to the internet via an ethernet cable

3: wlan0:<BROADCAST,MULTICAST>mtu 1500 qdisc fq_code1 state DOWN group default qlen 1000

4:enp1s0u1u1: <BROADCAST,MULTICAST>mtu 1500 qdisc noop state DOWN group default qlen 1000 ** not sure what this is, I have another ethernet cable connected to the configured router.

Then when I ask getent hosts it says: 127.0.0.1 node

So when I ping the ethoscope (192.168.0.2) I get host unreachable. Which makes sense since it seems that it can't see it?

ggilestro commented 4 years ago

The node uses the zeroconf/avahi protocol to detect ethoscopes on the network. If your machines cannot ping each other you probably have some kind of firewall in the router I suppose. I would recommend trying another router/wifi - for small setups this one works well in my experience: https://www.tp-link.com/uk/home-networking/3g-4g-router/tl-mr3020/

DenaFrancis commented 4 years ago

Thank you, I might try to disable the firewall on my current router. If this doesn’t work then i will get the tp link.

Sent from my iPhone

On 1 Apr 2020, at 1:56 am, Giorgio Gilestro notifications@github.com wrote:

 The node uses the zeroconf/avahi protocol to detect ethoscopes on the network. If your machines cannot ping each other you probably have some kind of firewall in the router I suppose. I would recommend trying another router/wifi - for small setups this one works well in my experience: https://www.tp-link.com/uk/home-networking/3g-4g-router/tl-mr3020/

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

DenaFrancis commented 4 years ago

The access control on my router is enabled to detect any new devices. With this I can see the ETHOSCOPE000 has been detected by the router wirelessly, using the SSID that was assigned. So, is the node not being detected as a wired connection because of the type of protocol that it uses? I am just worried that if I get a different router, it still may not be able to detect the node. Its also strange that when the node is directly connected to the router via LAN that the LAN position lights up on the router. At least that tells me that the connection is there?

On 1 Apr 2020, at 6:59 am, Deanne Francis francis.dena@gmail.com wrote:

Thank you, I might try to disable the firewall on my current router. If this doesn’t work then i will get the tp link.

Sent from my iPhone

On 1 Apr 2020, at 1:56 am, Giorgio Gilestro notifications@github.com wrote:



The node uses the zeroconf/avahi protocol to detect ethoscopes on the network. If your machines cannot ping each other you probably have some kind of firewall in the router I suppose. I would recommend trying another router/wifi - for small setups this one works well in my experience: https://www.tp-link.com/uk/home-networking/3g-4g-router/tl-mr3020/ https://www.tp-link.com/uk/home-networking/3g-4g-router/tl-mr3020/ — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/gilestrolab/ethoscope/issues/111#issuecomment-606717456, or unsubscribe https://github.com/notifications/unsubscribe-auth/AENPXYPVJFMU64R3ZSSEU3TRKIHEVANCNFSM4LT543PA.

pepelisu commented 4 years ago

Possible problems with the router:

  1. Node has a static IP given during installation and therefore not getting an IP auto-assigned by the router. (Node could not be not accessible if the IP is out of the DHCP range) [It is extrange for me that the node gets the 192.168.0.2 and the router 192.168.0.10, normally this numbers are sequential for devices connected sequentially.
  2. Router has an active Firewall for local traffic (not so common, normally local traffic is allowed
  3. Ethernet cable is connected to WAN port instead of switch ports. (normally marked in different colours). Could you please send us a picture of the connections and the router model?
DenaFrancis commented 4 years ago

 The model is a netgear nighthawk ac1900 model d7000v2

This is a picture of the Ethernet cable connected to the Lan port 1.

This is the raspberry pi 4b with an Ethernet to USB connecting to the internet modem. And the Ethernet cable connected to the router.

This is the raspberry 3b on with the power source. I think it might be point number one from your list. Except that the node is getting the IP 192.168.0.10 while the ethoscope is assigned 192.168.0.2.

Thanks so much.

On 1 Apr 2020, at 4:28 pm, Pepelisu notifications@github.com wrote:  Possible problems with the router:

Node has a static IP given during installation and therefore not getting an IP auto-assigned by the router. (Node could not be not accessible if the IP is out of the DHCP range) [It is extrange for me that the node gets the 192.168.0.2 and the router 192.168.0.10, normally this numbers are sequential for devices connected sequentially. Router has an active Firewall for local traffic (not so common, normally local traffic is allowed Ethernet cable is connected to WAN port instead of switch ports. (normally marked in different colours). Could you please send us a picture of the connections and the router model? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

pepelisu commented 4 years ago

Sorry, that is what I meant, ethoscope= 192.168.0.2 and node=192.168.0.10. Check that the IP on the node is not manually assigned in the node configuration. Would it be possible that the Ethernet+usb is connected to another home router? In that case IP of the two nets would be the same and it will not work. Please, disconnect the Ethernet+usb, internet is not needed right now, and it only complicates things. (should be needed in then next step, when the local network is working) Ensure that ethernet cable from ethernet port in pi4 is connected to one of the black ports in the router (not the yellow one). Images are not attached to the email (or not showed by github).

DenaFrancis commented 4 years ago

Yes! I think that you hit the nail on the head. I do have the node connected to an internet modem/router which I think has the same IP address as the router. So if I disconnect the internet, I will need to connect the node to a monitor/ keyboard to figure out what the IP address the router I am using has assigned. I am not sure what I will do about using the internet server then or how to ensure that the node has a static IP address. It seems that whenever I try to add photos via email, they don’t show up in the github. Sorry about that.

Sent from my iPhone

On 1 Apr 2020, at 6:25 pm, Pepelisu notifications@github.com wrote:

 Sorry, that is what I meant, ethoscope= 192.168.0.2 and node=192.168.0.10. Check that the IP on the node is not manually assigned in the node configuration. Would it be possible that the Ethernet+usb is connected to another home router? In that case IP of the two nets would be the same and it will not work. Please, disconnect the Ethernet+usb, internet is not needed right now, and it only complicates things. (should be needed in then next step, when the local network is working) Ensure that ethernet cable from ethernet port in pi4 is connected to one of the black ports in the router (not the yellow one). Images are not attached to the email (or not showed by github).

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

pepelisu commented 4 years ago

If that is the issue you should be able to set the IP range of the DHCP server in the local router (netgear). That means that Ethoscope and Node will get an IP from the range (for example) 192.168.1.0, netmask has to be 255.255.255.0

DenaFrancis commented 4 years ago

So I have now created a new IP range, my router would only let me create one with 192.168.0.10. Now I can see through the router that the ETHOSCOPE000 (192.168.0.11) and the node (192.168.0.14) are connected to the router with their own IP addresses. The issue now is that I cannot access the HTTP://node site to see if the ethosocope is listed there or not. I have reconnected the node to the internet, but when I try to access the site I see this:

Screen Shot 2020-04-04 at 10 58 47 am

And when I click on the update tab I get this:

Screen Shot 2020-04-04 at 10 52 19 am

When I try to see the management page I get this:

Screen Shot 2020-04-04 at 11 01 55 am
ggilestro commented 4 years ago

Use http://localhost

On Sat, 4 Apr 2020, 02:03 DenaFrancis, notifications@github.com wrote:

So I have now created a new IP range, my router would only let me create one with 192.168.0.10. Now I can see through the router that the ETHOSCOPE000 (192.168.0.11) and the node (192.168.0.14) are connected to the router with their own IP addresses. The issue now is that I cannot access the HTTP://node site to see if the ethosocope is listed there or not. I have reconnected the node to the internet, but when I try to access the site I see this: [image: Screen Shot 2020-04-04 at 10 58 47 am] https://user-images.githubusercontent.com/18545633/78415168-5cdc4200-7663-11ea-9b1d-8715a439b3e8.png And when I click on the update tab I get this: [image: Screen Shot 2020-04-04 at 10 52 19 am] https://user-images.githubusercontent.com/18545633/78415185-7d0c0100-7663-11ea-906c-c829ccb81016.png When I try to see the management page I get this: [image: Screen Shot 2020-04-04 at 11 01 55 am] https://user-images.githubusercontent.com/18545633/78415238-dd02a780-7663-11ea-9611-a197abac06b2.png

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/gilestrolab/ethoscope/issues/111#issuecomment-608945716, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFKERM5SI3OJHCCWCSAJK3RK2BPXANCNFSM4LT543PA .

DenaFrancis commented 4 years ago

I have tried http://local host, but it gives the same error as when I try http://node.

I did manage to ping the node from the ethoscope though. So it seems that it’s working/connected.

The only thing is that when I look at getent hosts on the ethoscope it still says that node has an IP address of 192.168.0.100. That is not the IP address the node gets from the router.not sure if this matters.

Sent from my iPhone

On 5 Apr 2020, at 4:31 am, Giorgio Gilestro notifications@github.com wrote:

 Use http://localhost

On Sat, 4 Apr 2020, 02:03 DenaFrancis, notifications@github.com wrote:

So I have now created a new IP range, my router would only let me create one with 192.168.0.10. Now I can see through the router that the ETHOSCOPE000 (192.168.0.11) and the node (192.168.0.14) are connected to the router with their own IP addresses. The issue now is that I cannot access the HTTP://node site to see if the ethosocope is listed there or not. I have reconnected the node to the internet, but when I try to access the site I see this: [image: Screen Shot 2020-04-04 at 10 58 47 am] https://user-images.githubusercontent.com/18545633/78415168-5cdc4200-7663-11ea-9b1d-8715a439b3e8.png And when I click on the update tab I get this: [image: Screen Shot 2020-04-04 at 10 52 19 am] https://user-images.githubusercontent.com/18545633/78415185-7d0c0100-7663-11ea-906c-c829ccb81016.png When I try to see the management page I get this: [image: Screen Shot 2020-04-04 at 11 01 55 am] https://user-images.githubusercontent.com/18545633/78415238-dd02a780-7663-11ea-9611-a197abac06b2.png

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/gilestrolab/ethoscope/issues/111#issuecomment-608945716, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFKERM5SI3OJHCCWCSAJK3RK2BPXANCNFSM4LT543PA .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

pepelisu commented 4 years ago

Please be sure you are trying in the node and only in the node:  http://localhost  (no spaces),  if it does not work, please try 127.0.0.1This two addresses are independent of the router address.  They should open the node website. 

DenaFrancis commented 4 years ago

Yes, I used the local host link from the Ethoscope manual and I still get the same issue. I also tried 127.0.0.1 and could not get through due to refused to connect. My firewall setting are open to all devices and there are no proxies. I have done this using through the wireless internet and the local connection ETHOSCOPE_WIFI which I can access on my laptop.

I suspect that there is something wrong with the usb to ethernet connector? I switched to connecting the internet cable directly into the RPI ethernet port. And using the usb to ethernet connector to connect to the router. I can see the node on my internet modem page with the IP address of the router (192.168.0.10) not the one that the ETHOSCOPE000 would see (192.168.0.14). When I connect things up this way and go on the internet I now get the server website back, but it still can’t detect any devices connected to the node.

I can also see that when I access the router, the node is no longer connected via LAN. Even though the LAN is lit up in the router. Seems that there needs to be a direct connection through the ethernet port. This makes me suspect that the usb to ethernet cable is not working or is not fast enough to transmit the data compared to the ethernet port on the RPI 4.

On 5 Apr 2020, at 4:07 pm, Pepelisu notifications@github.com wrote:

Please be sure you are trying in the node and only in the node: http://localhost (no spaces), if it does not work, please try 127.0.0.1This two addresses are independent of the router address. They should open the node website. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/gilestrolab/ethoscope/issues/111#issuecomment-609364665, or unsubscribe https://github.com/notifications/unsubscribe-auth/AENPXYOIZ25FUYENY5U26EDRLAN2JANCNFSM4LT543PA.

DenaFrancis commented 4 years ago

In terms of the usb to ethernet connector that is connected to the internet. It seems that the node can see it and has a driver for this. This is some of the code I have used.

[root@node "] # lsusb
Bus 001 Device 003: ID 0b95:772a ASIX ELEctronics Corp. AX88772A Fast Ethernet
[root@node "] # dmesg | grep -1 enp1s0u1u3
USB core: registered new interface driver asix
asix 1-1.3:1.0 enp1s0u1u3: renamed from eth1
USB 1-1.4: new full-speed USB device number 4 using xhci_hcd
IPv6:ADDRCONF(NETDEV_UP): enp1s0u1u3: link is not ready
IPv6:ADDRCONF(NETDEV_UP): enp1s0u1u3: link becomes ready
asix 1-1.3:1.0 enp1s0u1u3: link up, 100Mbps, full-duplex, lpa 0xC5E1

I don't know if this helps. Unfortunately, this has been keeping me up at night.

pepelisu commented 4 years ago

Try this: Connect the ethernet port of the node to the local router. Disconnect the USB-ethernet and do not connect the node or the ethoscope to your home/internet router. You don't need internet at this point.  If necesarry attach a keyboard, mouse monitor to the node.  Load the localhost website directly from the node. If this does not show the node website with the ethoscope the we should look into the configuration of you local router(netgear). On Apr 9, 2020 01:26, DenaFrancis notifications@github.com wrote: In terms of the usb to ethernet connector that is connected to the internet. It seems that the node can see it and has a driver for this. This is some of the code I have used. [root@node "] # lsusb Bus 001 Device 003: ID 0b95:772a ASIX ELEctronics Corp. AX88772A Fast Ethernet

[root@node "] # dmesg | grep -1 enp1s0u1u3 USB core: registered new interface driver asix asix 1-1.3:1.0 enp1s0u1u3: renamed from eth1 USB 1-1.4: new full-speed USB device number 4 using xhci_hcd IPv6:ADDRCONF(NETDEV_UP): enp1s0u1u3: link is not ready IPv6:ADDRCONF(NETDEV_UP): enp1s0u1u3: link becomes ready asix 1-1.3:1.0 enp1s0u1u3: link up, 100Mbps, full-duplex, lpa 0xC5E1

I don't know if this helps. Unfortunately, this has been keeping me up at night.

—You are receiving this because you commented.Reply to this email directly, view it on GitHub, or unsubscribe.

DenaFrancis commented 4 years ago

Thank you. Yes, I have connected only the node Ethernet cable to the router. My MacBook has access to the ethoscope wifi and when I type in 192.169.123.1 which I have changed as a reserved IP address for the node on the router( according to the latest manual). I can now load the website when I am connected to this local network through my MacBook. But I see that the there are still no devices detected. On the same local network I can access my router and it shows that 3 devices are connected 1) the node - wired 2) the ethoscope000 at an updated IP address of 192.169.123.7- wireless 3) my MacBook.- wireless

I can also ping both the node and the ethoscope from my MacBook terminal and they both respond.

Sent from my iPhone

On 9 Apr 2020, at 4:54 pm, Pepelisu notifications@github.com wrote:

 Try this: Connect the ethernet port of the node to the local router. Disconnect the USB-ethernet and do not connect the node or the ethoscope to your home/internet router. You don't need internet at this point. If necesarry attach a keyboard, mouse monitor to the node. Load the localhost website directly from the node. If this does not show the node website with the ethoscope the we should look into the configuration of you local router(netgear). On Apr 9, 2020 01:26, DenaFrancis notifications@github.com wrote: In terms of the usb to ethernet connector that is connected to the internet. It seems that the node can see it and has a driver for this. This is some of the code I have used. [root@node "] # lsusb Bus 001 Device 003: ID 0b95:772a ASIX ELEctronics Corp. AX88772A Fast Ethernet

[root@node "] # dmesg | grep -1 enp1s0u1u3 USB core: registered new interface driver asix asix 1-1.3:1.0 enp1s0u1u3: renamed from eth1 USB 1-1.4: new full-speed USB device number 4 using xhci_hcd IPv6:ADDRCONF(NETDEV_UP): enp1s0u1u3: link is not ready IPv6:ADDRCONF(NETDEV_UP): enp1s0u1u3: link becomes ready asix 1-1.3:1.0 enp1s0u1u3: link up, 100Mbps, full-duplex, lpa 0xC5E1

I don't know if this helps. Unfortunately, this has been keeping me up at night.

—You are receiving this because you commented.Reply to this email directly, view it on GitHub, or unsubscribe. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

pepelisu commented 4 years ago

OK, So now we have a little bit of a confusion. I will try to clarify things here:

  1. The local router should have a local IP address, those are in the range of 192.168.xxx.0 where xxx is a number between 0 and 254. Also 10.xxx.xxx.0 (among others). I will recommend to set up the local network to 192.168.1.0 (that would avoid configuration problems with your "home router" that is given 192.168.0.0 addresses. It is not recommended to use 192.169.123.0/24 because that is a real IP in US and it could create problems of resolving addresses.
  2. The easiest way to set the node up is with a screen attached to it. The reason for this is that network traffic in the interface connected to the local router has to be restricted to that network. (in NetworkManager in Gnome is a tick box that has to be marked "Use this connection only for resources in this network". If this is not possible then connect the node, ethoscope and your mac (or pc) to the local router. Do not connect any other router or network interface until this is working, otherwise it would be difficult to debug.
  3. (the step in which you are right now, apart from the ip number (step1)) Ethoscope000 is an uninitialized ethoscope that should use avahi daemon to be detected by the node. I have experienced some problems with the auto configuration, and it is not yet clear where the bug is. To manually intialize an ethoscope you can connect to it using ssh connection. Here are the recommended steps to try to solve the problem of auto initialization:

avahi-browse --all

This should give you an output like the following:

[gg@node ~]$ avahi-browse --all | grep etho
+ wlp2s0 IPv4 ETHOSCOPE000-106ca441c9ec45f2a09dba9f5b8a612e _ethoscope._tcp      local

Of course you expect to see ethoscope000 come online.

If it comes online, try restart the node daemon. Execute on the node: sudo systemctl restart ethoscope_node.service

That should make it appear and you can continue from there. If still does not work, you can ssh into the ethoscope (ssh alarm@ethoscope.ip.adrress) and change the following files:

/etc/machine-name
/etc/hostname
/etc/machine-id
/etc/hosts

For making changes to this files you have to be root, execute in terminal su password root. machine-name and hostname should have the name of the ethoscope (e.g. ETHOSCOPE001). for machine-id, just change the first three characters to the number of the ethoscope (001) in /etc/hosts you need to hardcode the IP address of the node and make sure you have a value for localhost 127.0.0.1 too

After doing this changes a restart of the ethoscope is needed.

DenaFrancis commented 4 years ago

Thanks!

  1. Okay, I apologise for the confusion. I have now changed the router IP again to 192.168.1.6, so the range is set to 192.168.1.6 to 254. I still reserved the node IP address to 192.168.1.5. I hope that that is okay.

  2. I have hooked up the PI and typed in

Avahi-browse —all 

Failed to create client object: Daemon not running.

Then I tried

ssh alarm@ethoscope.ip.address

ssh : Could not resolve hostname ethoscope.ip.address: Name or service not known.

Also, when I go onto the website through the local area network on my MacBook using 192.168.1.5 I see that the time and date on the platform is very different from that of the machine. I am not sure what this means.

On the platform, Sat Aug 17 2019 14:45:05 GMT+1000 (Australian Eastern Standard Time)

On your machine, Thu Apr 09 2020 22:17:28 GMT+1000 (Australian Eastern Standard Time)

Perhaps I did something wrong?

pepelisu commented 4 years ago

Ok, so... I will reply over yours. Edit: While replying I saw where all the possible confusion is coming. The online manual has somehow reverted to an old version.

Thanks!

You are welcome.

  1. Okay, I apologise for the confusion. I have now changed the router IP again to 192.168.1.6, so the range is set to 192.168.1.6 to 254. I still reserved the node IP address to 192.168.1.5. I hope that that is okay.

Nothing to apologise for, I was just clarifying it to avoid further problems.

2. I have hooked up the PI and typed in
Avahi-browse —all> 
Failed to create client object: Daemon not running.

avahi-browse --all (mind the double dash and small letters) If its still giving you an error then, avahi is not installed. Did you install the custom packages in the node? To do that, you have to install yay. and from aur you should install ethoscope-node. However while I am writing this I just checked the documentation and somehow the online version has been reverted to an old documentation. I will check into this.

Then I tried

ssh alarm@ethoscope.ip.address
ssh : Could not resolve hostname ethoscope.ip.address: Name or service not known.

ethoscope.ip.address has to be replaced by the actual ip address for example 192.168.1.6

Also, when I go onto the website through the local area network on my MacBook using 192.168.1.5 I see that the time and date on the platform is very different from that of the machine. I am not sure what this means. It is ok, Rpi's does not have a real time clock (no battery etiher) so they do not keep the time, they remember the last time they were connected to internet. Don't worry abou it for now.

On the platform, Sat Aug 17 2019 14:45:05 GMT+1000 (Australian Eastern Standard Time)

On your machine, Thu Apr 09 2020 22:17:28 GMT+1000 (Australian Eastern Standard Time)

Perhaps I did something wrong?

DenaFrancis commented 4 years ago

Sent from my iPhone

On 9 Apr 2020, at 11:19 pm, Pepelisu notifications@github.com wrote:

 Ok, so... I will reply over yours. Edit: While replying I saw where all the possible confusion is coming. The online manual has somehow reverted to an old version.

Thanks! You are welcome.

  1. Okay, I apologise for the confusion. I have now changed the router IP again to 192.168.1.6, so the range is set to 192.168.1.6 to 254. I still reserved the node IP address to 192.168.1.5. I hope that that is okay. Nothing to apologise for, I was just clarifying it to avoid further problems.

  2. I have hooked up the PI and typed in

    Avahi-browse —all avahi-browse --all (mind the double dash and small letters) If its still giving you an error then, avahi is not installed. Did you install the custom packages in the node? Yes, I used the double dash when I wrote the code in the node, sorry just wrote it wrong here. I downloaded the node img.zip file(20190819 ethoscope node OS pi4.img.zip), unzipped it to burn onto the sd card using balena etcher. I didn’t manually install. I will await instructions on how to do this once the manual is up again. I assume that I would do this from my MacBook and reformat the sd card? Or can I do it on the node? Thanks! To do that, you have to install yay. and from aur you should install ethoscope-node. However while I am writing this I just checked the documentation and somehow the online version has been reverted to an old documentation. I will check into this.

    Failed to create client object: Daemon not running.

    
    Then I tried
    

    ssh alarm@ethoscope.ip.address ethoscope.ip.address has to be replaced by the actual ip address for example 192.168.1.6

    ssh : Could not resolve hostname ethoscope.ip.address: Name or service not known.

    
    
    Also, when I go onto the website through the local area network on my MacBook using 192.168.1.5 I see that the time and date on the platform is very different from that of the machine. I am not sure what this means.
    It is ok, Rpi's  does not have a real time clock (no battery etiher) so they do not keep the time, they remember the last time they were connected to internet. Don't worry abou it for now.
    
    On the platform, Sat Aug 17 2019 14:45:05 GMT+1000 (Australian Eastern Standard Time)
    
    On your machine, Thu Apr 09 2020 22:17:28 GMT+1000 (Australian Eastern Standard Time)
    
    Perhaps I did something wrong?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

DenaFrancis commented 4 years ago

I also forgot to add that I have a very difficult time with the manual installation using terminal on my MacBook as it doesn't let me partition the sd card.

Using balena-etcher was super easy and worked very well with the .img files for the node and ethoscope.

pepelisu commented 4 years ago

Please check the re-covered documentation here: https://qgeissmann.gitbooks.io/ethoscope-manual/building-and-installation/node.html In the section, install the node automatic, there is section for Pi4 node installation and configuration. I think that part was missing. We will migrate it soon to another service but we will announce it.

DenaFrancis commented 4 years ago

I just want to clarify so that I don't get something wrong. I will take pieces from the manual here. Since the node will be on raspberry pi, I am using the installation instructions for raspberry pi.

Installing the node - raspberry PI 4. The newest Raspberry PI 4 has enough power to run a node. Installation of the node on a rPI4 is straightforward. Download the image file from this link and burn it, using Balena Etcher, on a Samsung 32GB SD Card.

Done this...

Place the card in the PI and apply power. The PI can run "headless" , meaning that does not need to be connected to a monitor or keyboard. The Web Server can be accessed through this URL on the same subnet: http://node.

Done this...

Basic configuration - saving files Most of the ethoscope activity is setup out of the box. Data created on the ethoscopes need to be periodically copied to the node. The node runs two independent daemons with this job. You can check their status and make sure they are active from the Admin tab on the management page (gear icon).

When I check the management page, all of the daemons show that they are active.

It is not clear to me what I am to do next or where I would do it from? Should I attach a keyboard and monitor to the node and install something from there? I apologize for my ignorance, I am not a programmer so the instructions have to be really dumbed down (LOL).

Now, there is another section in the manual with reference to Raspberry pi:

Installing the node - manual. A raspberry pi 4 is powerful enough to run a node. If you decide to go that route, install the archlinux OS from here then follow the same instructions below. You will have to manually install yay from AUR. An IMG file with a full node installation ready to run on a raspberry pi 4 can be downloaded from here (there is no link attached).

I assume from your answer that you want to use the automatic installation, which I have done. I just wonder what I am missing here.

DenaFrancis commented 4 years ago

This is bizarre, but when I just logged into the node IP address, it now can see the device! Seems that something resolved itself? I have disconnected the node from the power source several times and gotten no results after reconnecting. I am not sure why it worked this time but seems that it is now able to detect the ethoscope. I don't know what will happen when I connect the node to the internet, but it seems to work when I log onto the host network on the MacBook. Amazing, I am still in disbelief.

pepelisu commented 4 years ago

I mark this as resolved.

DenaFrancis commented 4 years ago

Thank you for all of your help!

Sent from my iPhone

On 15 Apr 2020, at 4:35 am, Pepelisu notifications@github.com wrote:

 I mark this as resolved.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

DenaFrancis commented 4 years ago

Hi Pepelisu, I am still having issues after the power to the node is turned off for whatever reason, its impossible for the node web server to re-establish a connection to the Ethoscope. I got it to work once, but then the power went out and when I restored power to the node, the ethoscope was not detected again. I tried to power cycle it and but it’s back to not being detected. Why is it that the ethoscope is displayed on the web server sometimes, but not other times? This time when I ssh from the node, I can ping the ethoscope and there is a connection and the ahavi daemon seems to be working fine.

One other issue is that the ethoscope software is out of date, but I cannot update it because I cannot connect to the internet, or the node server will not work. I can only access the node through the local connection router.

I am out of ideas and I am thinking that I should just give up?

Thanks, Dena

On 15 Apr 2020, at 4:35 am, Pepelisu notifications@github.com wrote:

I mark this as resolved.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/gilestrolab/ethoscope/issues/111#issuecomment-613610290, or unsubscribe https://github.com/notifications/unsubscribe-auth/AENPXYIOBPA5GJC2HNXFRP3RMSUGTANCNFSM4LT543PA.

pepelisu commented 4 years ago

Hi Dena, I really think that all the trouble that you are having comes from the network configuration. I also think that the solution is easy but difficult to explain, I will try.

Router to internet

This router will be connected to your home phone (or cable) socket and it will be configured automatically by the provider. I will leave it like that.

Router to ethoscopes (or local router)

This router provides a only local connection between ethoscopes (wifi) and node (ethernet). It should be configured with DHCP and the range should be different from the internet router. For example:192.168.1.0/24, that means ip's from 192.168.1.0 to 192.168.254, The netmask has to be set to 255.255.255.0

Node

It is providing a first place to backup all the data from ethoscopes, provides an interface to manage the tracking and provides internet for updates. Ethoscopes are connected to the node via the local router and they are detected by the node using the avahi daemon.

The problem here appears to be the configuration of the RPI4 with two Ethernets. If the two rotuers are providing IP addresses in the same range (in this case, both are providing 192.168.0.XXX ip's) then when both are connected one router is prioritized over the other and all the traffic is done through the preferred one. (in this case appears to be the home router) The suggestion to solve this issue will be to modify the configuration in the local router, so it provides IP's in the range of 192.168.1.xxx. The node needs a fixed IP address for example 192.168.1.2, it has to set up in the local router. The next step is to tell the Node that the network with the IP 192.168.1.XXX range is only for that addresses. To do this the easiest is to use the "network manager" in gnome (this is the default network manager in gnome) and selecting the Wired connection attached to the local router (you will know by the ip address) network manager > IPv4 options > routes menu you should mark the "Use this connection only for resources on its network" option. This will force the system to use this connection only for the local network. If the ethoscopes are not shown even after a reboot, you can perform a full system upgrade on the node (execute this on a terminal yay -Syu) and also reinstall the node package (yay -S ethoscope-node, you need internet for this).

After all of this is working you should be able to see the ethoscope (maybe you need to restart node and ethoscope, in that order). Node should also have internet.

This is a rather long comment but I am not sure where you are finding the problem, so I am describing the whole configuration in the hope that you can find where it is failing.

DenaFrancis commented 4 years ago

Hi, Thank you, I have commented on top of your comments.

I got some help from a raspberry Pi expert at my university and we found that the ahavi daemon on the ethoscope was not able to be loaded or running because there was a storage space issue in the ethoscope OS. Most of this was done by ssh onto the node or the ethoscope from my MacBook.

Once, I cleared up some space in the journal logs and rebooted the ethoscope, it was automatically detected by the node. Since I installed the latest OS img file onto the SD card, I wonder if that version may have some issues with how much space is allocated to different partitions?

On 22 Apr 2020, at 10:43 pm, Pepelisu notifications@github.com wrote:

Hi Dena, I really think that all the trouble that you are having comes from the network configuration. I also think that the solution is easy but difficult to explain, I will try.

Router to internet

This router will be connected to your home phone (or cable) socket and it will be configured automatically by the provider. I will leave it like that.

Router to ethoscopes (or local router)

This router provides a only local connection between ethoscopes (wifi) and node (ethernet). It should be configured with DHCP and the range should be different from the internet router. For example:192.168.1.0/24, that means ip's from 192.168.1.0 to 192.168.254, The netmask has to be set to 255.255.255.0

In terms of the configuration, the IP addresses of the node, router and ethoscope are all 192.168.1.XXX while the IP addresses of the internet router are 192.168.0.XXX. Node

It is providing a first place to backup all the data from ethoscopes, provides an interface to manage the tracking and provides internet for updates. Ethoscopes are connected to the node via the local router and they are detected by the node using the avahi daemon.

The problem here appears to be the configuration of the RPI4 with two Ethernets.

If the two rotuers are providing IP addresses in the same range (in this case, both are providing 192.168.0.XXX ip's) then when both are connected one router is prioritized over the other and all the traffic is done through the preferred one. (in this case appears to be the home router) The suggestion to solve this issue will be to modify the configuration in the local router, so it provides IP's in the range of 192.168.1.xxx. The node needs a fixed IP address for example 192.168.1.2, it has to set up in the local router.

Yes, this is what I have already done. The next step is to tell the Node that the network with the IP 192.168.1.XXX range is only for that addresses. To do this the easiest is to use the "network manager" in gnome (this is the default network manager in gnome) and selecting the Wired connection attached to the local router (you will know by the ip address) network manager > IPv4 options > routes menu you should mark the "Use this connection only for resources on its network" option. This will force the system to use this connection only for the local network.

I apologise, but how to I access gnome? Is it in the node OS or do I access it through the web server? If the ethoscopes are not shown even after a reboot, you can perform a full system upgrade on the node (execute this on a terminal yay -Syu) and also reinstall the node package (yay -S ethoscope-node, you need internet for this).

After all of this is working you should be able to see the ethoscope (maybe you need to restart node and ethoscope, in that order). Node should also have internet.

This is a rather long comment but I am not sure where you are finding the problem, so I am describing the whole configuration in the hope that you can find where it is failing.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/gilestrolab/ethoscope/issues/111#issuecomment-617756222, or unsubscribe https://github.com/notifications/unsubscribe-auth/AENPXYJXWEBMFWZLEWVCK6TRN3RAZANCNFSM4LT543PA.

pepelisu commented 4 years ago

Let's see if now it works. If avahi was not working then it was also a big problem. We will check the image, can you do that @ggilestro? Probably the journal configuration has to be changed to a lower limit for the space. For the connection issue, gnome is the desktop environment. if you are logging in by ssh then you should try with the terminal command, use this to force the desired connection to work only for local resources. nmcli con modify eth0 ipv4.never-default yes
Please change eth0 for the right interface, it could be eth0 or enp1s0u1u1 in your case. Check for the right one using the ip a command, you can tell by checking the IP address, it should be the one with the ip address 192.168.1.2.

ggilestro commented 4 years ago

Hi @DenaFrancis and thanks for the detailed report. Would it be possible to get your "raspberry pi expert" to participate in this conversation?

We occasionally have this issue with zeroconf not working immediately but it usually nails down to two problems: 1) the router is somehow blocking the avahi protocol (in this case it will never work unless you change firewall settings or change routers) 2) the node's network configuration is such that the node is looking for ethoscopes in the wrong subnet.

I thought your case was a case of number 2. The logs of the ethoscope should be saved on tmpfs as far as I can remember so I don't think this is a problem with the image also because we're using the same image ourselves and never had this issue.

I would suggest getting the person who helped you with this on this thread or, alternatively, organise a video call to sort out the issue.

DenaFrancis commented 4 years ago

Hi, Yes, I have cc’d Nigel and I can also show you what I did here if it helps? First I looked at the node to see if the avahi daemon was running...

[node@node ~]$ systemctl status avahi-daemon

Aug 17 14:39:56 node avahi-daemon[293]: Successfully dropped remaining capabilities. Aug 17 14:39:56 node avahi-daemon[293]: No service file found in /etc/avahi/services. Aug 17 14:39:56 node avahi-daemon[293]: Network interface enumeration completed. Aug 17 14:39:56 node avahi-daemon[293]: Server startup complete. Host name is node.local. Local service co> Aug 17 14:40:02 node avahi-daemon[293]: Joining mDNS multicast group on interface eth0.IPv4 with address 1> Aug 17 14:40:02 node avahi-daemon[293]: New relevant interface eth0.IPv4 for mDNS. Aug 17 14:40:02 node avahi-daemon[293]: Registering new address record for 192.168.1.5 on eth0.IPv4. Aug 17 14:40:03 node avahi-daemon[293]: Joining mDNS multicast group on interface eth0.IPv6 with address f> Aug 17 14:40:03 node avahi-daemon[293]: New relevant interface eth0.IPv6 for mDNS. Aug 17 14:40:03 node avahi-daemon[293]: Registering new address record for fe80::dea6:32ff:fe59:2bec on et> ~ ~ lines 1-21/21 (END)

Then I tested the node again and it didn’t pick up the ethoscope

[node@node ~]$ avahi-browse --all

Then I pinged the ethoscope from the node and it could “see” it.

[node@node ~]$ ping 192.168.1.7 PING 192.168.1.7 (192.168.1.7) 56(84) bytes of data. 64 bytes from 192.168.1.7: icmp_seq=1 ttl=64 time=156 ms 64 bytes from 192.168.1.7: icmp_seq=2 ttl=64 time=4.22 ms 64 bytes from 192.168.1.7: icmp_seq=3 ttl=64 time=3.61 ms 64 bytes from 192.168.1.7: icmp_seq=4 ttl=64 time=4.44 ms 64 bytes from 192.168.1.7: icmp_seq=5 ttl=64 time=4.20 ms ^C

* Now I looked at the ethoscope**

Checked the avahi daemon:

[root@ETHOSCOPE001 ~]# avahi-browse --all Failed to create client object: Daemon not running

Checked the connection with the node:

[root@ETHOSCOPE001 ~]# ping 192.168.1.5 PING 192.168.1.5 (192.168.1.5) 56(84) bytes of data. 64 bytes from 192.168.1.5: icmp_seq=1 ttl=64 time=3.67 ms 64 bytes from 192.168.1.5: icmp_seq=2 ttl=64 time=5.03 ms 64 bytes from 192.168.1.5: icmp_seq=3 ttl=64 time=4.30 ms 64 bytes from 192.168.1.5: icmp_seq=4 ttl=64 time=4.43 ms ^C --- 192.168.1.5 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3005ms rtt min/avg/max/mdev = 3.670/4.357/5.029/0.482 ms

Checked the status of avahi daemon:

[root@ETHOSCOPE001 ~]# systemctl status avahi-daemon ● avahi-daemon.service - Avahi mDNS/DNS-SD Stack Loaded: loaded (/usr/lib/systemd/system/avahi-daemon.service; disabled; ve> Active: inactive (dead) TriggeredBy: ● avahi-daemon.socket lines 1-4/4 (END)

Tried to enable the avahi daemon:

[root@ETHOSCOPE001 ~]# systemctl enable avahi-daemon Failed to enable unit: File /etc/systemd/system/dbus-org.freedesktop.Avahi.service: No space left on device

Tried to find out why there was no space...

[root@ETHOSCOPE001 ~]# df -h Filesystem Size Used Avail Use% Mounted on /dev/root 6.0G 6.0G 0 100% / devtmpfs 368M 0 368M 0% /dev tmpfs 372M 0 372M 0% /dev/shm tmpfs 372M 5.1M 367M 2% /run tmpfs 372M 0 372M 0% /sys/fs/cgroup tmpfs 372M 0 372M 0% /tmp /dev/mmcblk0p1 100M 39M 62M 39% /boot /dev/mmcblk0p3 23G 952M 22G 5% /var/lib/mysql

Looked deeper into it

[root@ETHOSCOPE001 ~]# du -hs /* 0 /bin 39M /boot 0 /dev 3.2M /etc 598M /ethoscope_data 256M /home 0 /lib 3.5K /lost+found 3.5K /mnt 118M /opt du: cannot access '/proc/618/task/618/fd/3': No such file or directory du: cannot access '/proc/618/task/618/fdinfo/3': No such file or directory du: cannot access '/proc/618/fd/3': No such file or directory du: cannot access '/proc/618/fdinfo/3': No such file or directory 0 /proc 21K /root 5.1M /run 0 /sbin 11K /srv 0 /sys 0 /tmp 2.9G /usr 1.7G /var

[root@ETHOSCOPE001 ~]# du -hs /var/* 4.6M /var/cache 22K /var/db 3.5K /var/empty 3.5K /var/games 223M /var/lib 3.5K /var/local 0 /var/lock 1.5G /var/log 0 /var/mail 3.5K /var/opt 0 /var/run 7.0K /var/spool 3.5K /var/tmp

Then journal log looks like it is take up space:

[root@ETHOSCOPE001 ~]# du -hs /var/log/* 3.5K /var/log/audit 1.0K /var/log/btmp 1.5K /var/log/btmp.1 1.5G /var/log/journal 288K /var/log/lastlog 3.5K /var/log/old 120K /var/log/pacman.log 3.5K /var/log/private 64K /var/log/tallylog 164K /var/log/wtmp

Looks like journal is the largest file:

[root@ETHOSCOPE001 ~]# du -hs /var/log/journal/* 313M /var/log/journal/00110b968d844d78817a7941db36aea8 185M /var/log/journal/00155fa83cc848019df1c9029bfb899f 41M /var/log/journal/00158b1703e84f0c92753525cb819b38 41M /var/log/journal/001916f944d54aaa9b7743793dc0bd16 361M /var/log/journal/100cb4e900344738a2498cdc748447cc 57M /var/log/journal/120811da4935473ebe2b0f5fd03e99a8 241M /var/log/journal/131acd14c6bb425a90ac24f2c6ba3198 8.1M /var/log/journal/19d4470932094532afdcadeabba61028 8.1M /var/log/journal/2416a1ec4c024965836db3df8f3e1a56 17M /var/log/journal/2e305bf68a51492ba64829188724596e 8.1M /var/log/journal/743b0bc3f72c48d58e49cbf9b069d0b3 17M /var/log/journal/b395e11cfddb4657aa9d8d24f4499349 105M /var/log/journal/ba499cde0c8e4eefa87a14e67f705d70 17M /var/log/journal/c3d48b0b0c9743e98b083dc8824a4ff7 73M /var/log/journal/fb20b293281d4ff18688837093240b38 3.5K /var/log/journal/remote

Then I looked the journal logs:

journalctl -f -- Logs begin at Thu 2020-02-13 20:07:04 UTC. -- Mar 10 04:50:25 ETHOSCOPE001 dbus-daemon[242]: [system] Failed to activate service 'org.freedesktop.login1': timed out (service_start_timeout=25000ms) Mar 10 04:50:25 ETHOSCOPE001 audit[606]: USER_START pid=606 uid=0 auid=0 ses=4 msg='op=PAM:session_open grantors=pam_loginuid,pam_keyinit,pam_limits,pam_unix,pam_permit,pam_mail,pam_env acct="root" exe="/usr/bin/sshd" hostname=192.168.1.8 addr=192.168.1.8 terminal=ssh res=success' Mar 10 04:50:25 ETHOSCOPE001 sshd[606]: pam_systemd(sshd:session): Failed to create session: Failed to activate service 'org.freedesktop.login1': timed out (service_start_timeout=25000ms) Mar 10 04:50:25 ETHOSCOPE001 kernel: kauditd_printk_skb: 14 callbacks suppressed Mar 10 04:50:25 ETHOSCOPE001 kernel: audit: type=1105 audit(1583815825.069:214): pid=606 uid=0 auid=0 ses=4 msg='op=PAM:session_open grantors=pam_loginuid,pam_keyinit,pam_limits,pam_unix,pam_permit,pam_mail,pam_env acct="root" exe="/usr/bin/sshd" hostname=192.168.1.8 addr=192.168.1.8 terminal=ssh res=success' Mar 10 04:50:25 ETHOSCOPE001 audit[616]: CRED_REFR pid=616 uid=0 auid=0 ses=4 msg='op=PAM:setcred grantors=pam_tally2,pam_shells,pam_unix,pam_permit acct="root" exe="/usr/bin/sshd" hostname=192.168.1.8 addr=192.168.1.8 terminal=ssh res=success' Mar 10 04:50:25 ETHOSCOPE001 kernel: audit: type=1110 audit(1583815825.249:215): pid=616 uid=0 auid=0 ses=4 msg='op=PAM:setcred grantors=pam_tally2,pam_shells,pam_unix,pam_permit acct="root" exe="/usr/bin/sshd" hostname=192.168.1.8 addr=192.168.1.8 terminal=ssh res=success' Mar 10 04:50:53 ETHOSCOPE001 systemd[1]: proc-sys-fs-binfmt_misc.automount: Got automount request for /proc/sys/fs/binfmt_misc, triggered by 618 (du) Mar 10 04:50:53 ETHOSCOPE001 systemd[1]: Mounting Arbitrary Executable File Formats File System... Mar 10 04:50:53 ETHOSCOPE001 systemd[1]: Mounted Arbitrary Executable File Formats File System.

After I used journalctl —vacuum-time=1 DAY, I freed some space

[root@ETHOSCOPE001 ~]# du -hs /var/log/journal 1.3G /var/log/journal

I apologise for the long email, I just wanted to show you what Nigel and I did. After I rebooted the ethoscope, it was automatically detected by the node.

Thank you, Dena

On 23 Apr 2020, at 11:03 pm, Giorgio Gilestro notifications@github.com wrote:

Hi @DenaFrancis https://github.com/DenaFrancis and thanks for the detailed report. Would it be possible to get your "raspberry pi expert" to participate in this conversation?

We occasionally have this issue with zeroconf not working immediately but it usually nails down to two problems:

the router is somehow blocking the avahi protocol (in this case it will never work unless you change firewall settings or change routers) the node's network configuration is such that the node is looking for ethoscopes in the wrong subnet. I thought your case was a case of number 2. The logs of the ethoscope should be saved on tmpfs as far as I can remember so I don't think this is a problem with the image also because we're using the same image ourselves and never had this issue.

I would suggest getting the person who helped you with this on this thread or, alternatively, organise a video call to sort out the issue.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/gilestrolab/ethoscope/issues/111#issuecomment-618390321, or unsubscribe https://github.com/notifications/unsubscribe-auth/AENPXYJ2N7KV6Y2UAWAP5WDROA4AZANCNFSM4LT543PA.

DenaFrancis commented 4 years ago

Hi Pepelisu, If I ssh from the node on my MacBook:

[node@node ~]$ nmcli con modify eth0 ipv4.never-default yes -bash: nmcli: command not found

[node@node ~]$ systemctl status NetworkManager
Unit NetworkManager.service could not be found.

Not sure if this is what I should be looking for… ~thanks

On 23 Apr 2020, at 10:51 pm, Pepelisu notifications@github.com wrote:

Let's see if now it works. If avahi was not working then it was also a big problem. We will check the image, can you do that @ggilestro https://github.com/ggilestro? Probably the journal configuration has to be changed to a lower limit for the space. For the connection issue, gnome is the desktop environment. if you are logging in by ssh then you should try with the terminal command, use this to force the desired connection to work only for local resources. nmcli con modify eth0 ipv4.never-default yes Please change eth0 for the right interface, it could be eth0 or enp1s0u1u1 in your case. Check for the right one using the ip a command, you can tell by checking the IP address, it should be the one with the ip address 192.168.1.2.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/gilestrolab/ethoscope/issues/111#issuecomment-618384630, or unsubscribe https://github.com/notifications/unsubscribe-auth/AENPXYP7IERTMWRBNEN7VTDROA2WVANCNFSM4LT543PA.

nigelb commented 4 years ago

Hi Dena,

The nmcli command is not installed on the base (Node) image.

The other issue is that the Australian Arch Linux mirror is broken making the installation other software problematic. To remedy this you will need to edit the mirrorlist file: /etc/pacman.d/mirrorlist and comment out the line:

Server = http://mirror.archlinuxarm.org/$arch/$repo

By placing a # at the beginning of the line like so:

#Server = http://mirror.archlinuxarm.org/$arch/$repo

after that, un-comment one of the non-Australian servers.

Then synchronise the package database:

#>  pacman -Syy
:: Synchronizing package databases...
 core                                                             214.7 KiB   302K/s 00:01 [####################################################] 100%
 extra                                                              2.2 MiB  3.14M/s 00:01 [####################################################] 100%
 community                                                          5.1 MiB  3.21M/s 00:02 [####################################################] 100%
 alarm                                                            121.2 KiB  11.8M/s 00:00 [####################################################] 100%
 aur                                                                4.4 KiB  0.00B/s 00:00 [####################################################] 100%

And then you can install the NetworkManager.

Nigel.

nigelb commented 4 years ago

Also, I have been looking into why the /var/log/journel directory is so big, even after you vacuum it. Turns out that systemd-journald service writes its logs to /var/log/journel/<machine-id> where <machine-id> can be found in the file /etc/machine-id which we can view it like this:


$> cat /etc/machine-id
00120b968d844d78817a7941db36aea8

Your file may have a different value in it, as it can be regenerated. Now when you issue a command like journalctl --vacuum-time=1d it seems to only remove data from the /var/log/journel/<machine-id> directory. I spent a little bit of time searching for the proper procedure to remove the logs files from these other machine-ids but did not find anything. However I found the following commands can clean up the directory (run as the root user):

This will delete all of the logs so if you need them, back them up first.

systemctl stop systemd-journald
Warning: Stopping systemd-journald.service, but it can still be activated by:
  systemd-journald.socket
  systemd-journald-dev-log.socket
  systemd-journald-audit.socket
$> rm -Rf /var/log/journal/*
$> reboot

This may not be the proper procedure but it seems to get the job done without breaking anything. Now when I have a look at the size of the journal directory:

$> du -hs /var/log/journal/
49M     /var/log/journal/

And the disk space usage:

$> df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/root       6.0G  4.1G  2.0G  68% /
devtmpfs        368M     0  368M   0% /dev
tmpfs           372M     0  372M   0% /dev/shm
tmpfs           372M  520K  372M   1% /run
tmpfs           372M     0  372M   0% /sys/fs/cgroup
tmpfs           372M     0  372M   0% /tmp
/dev/mmcblk0p1  100M   38M   63M  38% /boot
/dev/mmcblk0p3   23G  966M   22G   5% /var/lib/mysql
tmpfs            75M     0   75M   0% /run/user/1001

Hope this helps,

Nigel.

pepelisu commented 4 years ago

Hi Nigel, Thanks for spotting a full journal log.

What it was missing in the ethoscope OS configuration was a parameter in /etc/systemd/journald.conf SystemMaxUse=250MB that forces the system to save the last 250MB of logs, More than enought for debugging. Journal logs can be deleted at any time, just keep the folder /var/log/journal. I think next image of the ethoscope OS will be uploaded with the journal directory empty and with the SystemMaxUse= parameter updated.

ggilestro commented 4 years ago

Thanks everyone. I have committed those changes to the package in AUR and will create a fresh image. I don't think, however, that Dena's issue was linked to the journal because the system is independent of the avahi-daemon and runs its own python implementation of avahi.

My gut feeling is that the node is using the wrong network to look for devices.

nigelb commented 4 years ago

In the past I was working on a project using multicast traffic and I was having a problem where my program just would not receive the traffic, until I fired up avahi-daemon. Turns out I was not joining the multicast group correctly. But starting up the avahi-daemon joined the multicast group correctly and then all of a sudden my program would receive the multicast traffic. Maybe your device discovery issues have a similar root cause.

ggilestro commented 4 years ago

Thanks for the tip, @nigelb - I have enabled avahi-daemon in the AUR packages and next images will come with that enabled by default. It can't hurt.