Move server 3 to use new ethernet card and have external IP

[mitar]

Prerequisite for: #83

At the same time it would be useful to do: #82

[ck2qsuZT]

Did we order a new network card? Does the one you showed me last time we were in the network closet work?

[mitar]

I upgraded the server to new kernel and after that I have not noticed issues with the one I bought before (4 ports one) anymore. So I didn't know any new one.

[mitar]

(I worry now more that server 3 might not have enough CPU power for nodewatcher. I had to move online council app to campus server because server 2 was too slow. Those servers are really meant for file sharing only.)

[ck2qsuZT]

Would be pretty easy and not to expensive to upgrade, especially if we get a used server process, those things are a dime a dozen. Would we need more ram also?

On Nov 15, 2016 01:35, "Mitar" notifications@github.com wrote:

(I worry now more that server 3 might not have enough CPU power for nodewatcher. I had to move online council app to campus server because server 2 was too slow. Those servers are really meant for file sharing only.)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/cloyne/network/issues/95#issuecomment-260592262, or mute the thread https://github.com/notifications/unsubscribe-auth/AHwZSvYobm1VFVavQ_Sct43DdtFHEMwyks5q-XzXgaJpZM4KqFly .

[ck2qsuZT]

Especially if its for all house Internet all houses can chip in a bit?

On Nov 15, 2016 07:33, "James Martins" jsnm@berkeley.edu wrote:

Would be pretty easy and not to expensive to upgrade, especially if we get a used server process, those things are a dime a dozen. Would we need more ram also?

On Nov 15, 2016 01:35, "Mitar" notifications@github.com wrote:

(I worry now more that server 3 might not have enough CPU power for nodewatcher. I had to move online council app to campus server because server 2 was too slow. Those servers are really meant for file sharing only.)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/cloyne/network/issues/95#issuecomment-260592262, or mute the thread https://github.com/notifications/unsubscribe-auth/AHwZSvYobm1VFVavQ_Sct43DdtFHEMwyks5q-XzXgaJpZM4KqFly .

[mitar]

So yes, server 1 and server 2 should be probably replaced with a proper server, it does not need too much disk (2 TB), but memory is good to have (32 GB?).

But the question is where to get funding from for this. So for now we will be running on server 3, and later on we can migrate this to a new server.

[ck2qsuZT]

Why not convert the makerspace computer into a server? it has a pretty nice server CPU (i7-5820k), decent server motherboard (ASRock Fatal1ty X99X Killer, it supports ECC if we get a XEON E5 processor even though that's probably unecessary) and honestly we could make a Windows virtual machine with PCI passthrough of a graphics card and then get an intel nuc ($100-300 Maybe BSCIT? ) into the makerspace as thin client. NX protocol is extremely responsive and if we wanted to go crazy we could have a dedicated cable and ethernet port from the server to the thin client. We would need more RAM if we were to do this though but it prevents the nice expensive makerspace computer from getting stolen. The only thing we would need then is:

• 2 2TB disks in RAID1 ($100*2)
• maybe a system SSD since they're pretty cheap ($100 for 256GB) (we can probably get this through BSCIT)
• The makerspace motherboard is DDR4 so RAM costs ($6.5/GB for really good server ram, $6/GB for desktop ram) I'd say just populate a slot or two with maximum size (16GB/slot with our current CPU, 32GB/slot with a XEON E5 CPU) then we can upgrade later if needed when RAM gets cheaper (keep in mind though that server RAM doesn't change in price much.)
• a graphics card if we want to do the pci passthrough option ($100-$300) (we can probably get this through BSCIT)
• a rack-mount case runs from ($100-$600)
• we can sell our current CPU for a slightly better but mostly similar XEON (Intel Xeon E5-1650 v3) if we really need 32GB RAM/slot, ECC, etc. for about an extra $400
• we can also probably sell the current case for about $100 which will help a bit.

so in all:

• $300(16GB RAM Desktop RAM, cheapest case, no GPU, no SSD) to
• $2000 (64GB Server RAM, SSD, nice case, nice GPU, upgraded CPU, nice new intel nuc (up to $700from BSCIT)

[mitar]

I think we do not need a different CPU and stuff. To my knowledge we got money to buy replacement hardware for stolen hardware, so maybe the only thing which we need is a rack-mount case and move whatever is there there.

I would not go with thin client. And I would not go with changing what we have. Whatever we have is better than those file-server servers.

@clohard, how is makerspace computer fixing going?

[ck2qsuZT]

Did we get money or hardware from BSCIT?

[mitar]

Hm, we got money, but maybe then central office bought the equipment for us. This is something new they are trying this semester.

[ck2qsuZT]

Yeah, this semester central has been buying hardware for NM then a manager needs to go pick it up. If that be the case and we don't want to do a thin client then we would have a worthless GPU

[mitar]

We can do deep-learning on the GPU. :-)

[ck2qsuZT]

or folding at home ^_^

On Wed, Nov 16, 2016 at 4:22 AM, Mitar notifications@github.com wrote:

We can do deep-learning on the GPU. :-)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/cloyne/network/issues/95#issuecomment-260850610, or mute the thread https://github.com/notifications/unsubscribe-auth/AHwZShmkoyf0JqpHyOvhVSqSx7KxQXATks5q-oUGgaJpZM4KqFly .

[mitar]

Or we can sell the GPU. Motherboard probably have internal GPU no?

[ck2qsuZT]

Or we can use it as a thin client host =p. There is usually a relatively powerful iGPU in modern desktop CPUs but not server CPUs. The GPU built into the motherboard is pretty much text only.

[mitar]

The GPU built into the motherboard is pretty much text only.

Good enough. :-)

[ck2qsuZT]

I feel like the PCI issues may be related to our 9 year old Motherboard: www.supermicro.com/products/motherboard/Xeon3000/3000/PDSML-LN2_.cfm

[mitar]

I think that since I upgraded kernel to new version there was no ethernet issues anymore. So we should just proceed and start using the new network card with public IP.

[ck2qsuZT]

Why use an external network card anyways, is the advantage more ports (not saying this is a negligible advantage) or is there something else going on with the motherboard's Ethernet ports?

[mitar]

We need extra ports: internal network, antenna network, external network.

[mitar]

Also, one can do in the future batching over all ports and achieve 4 Gbit/s throughput.

[ck2qsuZT]

This and #104 can probably both be fixed at the same time. This should be done via salt, correct?

[mitar]

Yes.

[mitar]

Let's start work on this. We have 6 physical interfaces now in the server, one card with 4 ports, and two on the mother board:

$ ip link show
2: p1p1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:26:55:d9:cf:05 brd ff:ff:ff:ff:ff:ff
3: p1p2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:26:55:d9:cf:04 brd ff:ff:ff:ff:ff:ff
4: p1p3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:26:55:d9:cf:07 brd ff:ff:ff:ff:ff:ff
5: p1p4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:26:55:d9:cf:06 brd ff:ff:ff:ff:ff:ff
6: p5p1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
    link/ether 00:30:48:8c:bf:78 brd ff:ff:ff:ff:ff:ff
7: p6p1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
    link/ether 00:30:48:8c:bf:79 brd ff:ff:ff:ff:ff:ff

The plan is to connect:

• one interface to our external IP for the server, p1p1
• one interface to local network, p1p2
• one interface to debug network for antenna uplink, p1p3

[mitar]

This now work, but during boot it was for 60 seconds for some reason because of some network configuration issues. Not sure what. /var/log/boot.log contains many:

 * Starting configure network device security                                                                            [ OK ]
 * Starting configure network device                                                                                     [ OK ]
 * Starting configure network device security                                                                            [ OK ]
 * Starting configure network device security                                                                            [ OK ]
 * Starting configure network device                                                                                     [ OK ]
 * Starting configure network device                                                                                     [ OK ]
 * Starting configure network device security                                                                            [ OK ]
 * Starting configure network device security                                                                            [ OK ]
 * Starting configure network device                                                                                     [ OK ]
 * Starting configure network device                                                                                     [ OK ]
 * Starting configure network device security                                                                            [ OK ]
 * Starting configure network device security                                                                            [ OK ]
 * Starting configure network device                                                                                     [ OK ]
 * Starting configure network device                                                                                     [ OK ]
 * Starting configure network device security                                                                            [ OK ]
 * Starting configure network device security                                                                            [ OK ]
 * Starting configure network device                                                                                     [ OK ]
 * Starting configure network device                                                                                     [ OK ]
 * Starting configure network device security                                                                            [ OK ]
 * Starting configure network device security                                                                            [ OK ]
 * Starting configure network device                                                                                     [ OK ]
 * Starting configure network device                                                                                     [ OK ]
 * Starting configure network device security                                                                            [ OK ]
 * Starting configure network device security                                                                            [ OK ]
 * Starting configure network device security                                                                            [ OK ]
 * Starting configure network device security                                                                            [ OK ]
 * Starting configure network device                                                                                     [ OK ]
 * Starting configure network device                                                                                     [ OK ]
 * Starting configure network device                                                                                     [ OK ]
 * Starting configure network device                                                                                     [ OK ]
 * Stopping configure network device                                                                                     [ OK ]
 * Stopping configure network device security                                                                            [ OK ]
 * Starting configure network device security                                                                            [ OK ]
 * Starting configure network device security                                                                            [ OK ]
 * Starting configure network device                                                                                     [ OK ]
 * Starting configure network device                                                                                     [ OK ]
 * Starting configure network device security                                                                            [ OK ]
 * Starting configure network device security                                                                            [ OK ]
 * Starting configure network device                                                                                     [ OK ]
 * Starting configure network device                                                                                     [ OK ]
 * Stopping configure network device                                                                                     [ OK ]
 * Stopping configure network device security                                                                            [ OK ]
 * Stopping configure network device                                                                                     [ OK ]
 * Stopping configure network device security                                                                            [ OK ]
 * Stopping configure network device                                                                                     [ OK ]
 * Stopping configure network device security                                                                            [ OK ]
 * Stopping configure network device                                                                                     [ OK ]
 * Stopping configure network device security                                                                            [ OK ]
 * Stopping configure network device                                                                                     [ OK ]
 * Stopping configure network device security                                                                            [ OK ]
 * Stopping configure network device                                                                                     [ OK ]
 * Stopping configure network device security                                                                            [ OK ]
 * Stopping configure network device                                                                                     [ OK ]
 * Stopping configure network device security                                                                            [ OK ]
 * Stopping configure network device                                                                                     [ OK ]
 * Stopping configure network device security                                                                            [ OK ]

[mitar]

So now the output of ip link is:

2: p1p1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
    link/ether 00:26:55:d9:cf:05 brd ff:ff:ff:ff:ff:ff
3: p1p2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
    link/ether 00:26:55:d9:cf:04 brd ff:ff:ff:ff:ff:ff
4: p1p3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
    link/ether 00:26:55:d9:cf:07 brd ff:ff:ff:ff:ff:ff
5: p1p4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:26:55:d9:cf:06 brd ff:ff:ff:ff:ff:ff
6: p5p1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:30:48:8c:bf:78 brd ff:ff:ff:ff:ff:ff
7: p6p1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:30:48:8c:bf:79 brd ff:ff:ff:ff:ff:ff

[mitar]

The issue with network configuration was that I configured a gateway both for external IP interface and lan interface. Removing it from the lan interface solved the problem.

[mitar]

I also upgraded the server to Ubuntu Xenial 16.04.