===== Network Bonds, Bridges, VLANs in Linux (specifically CentOS8/RHEL8) =====
* **Bonding** is used to bond 2 or more network interfaces in order to make them act as 1; this can be used for increased throughput and or fault tolerance.
* **Bridging** is used to unite 2 or more network segments, a bridge acts as a virtual switch; this can be used to connected virtual network interfaces of virtual guests to the physical interface of a physical server.
* **VLANs** are layer 2 broadcast domains that are partitioned from each other; they are typically used to create mutilple virtual LANs on switch instead of having a single broadcast domain. They are useful for creating separate networks for different devices, departments, security sensitive devices, etc on the same switching hardware.
===== Bonded Network Interface =====
sources: [[https://phreek.org/blog/2014/11/centos-7-lacp-network-bonding|1]], [[https://access.redhat.com/discussions/3357541|2]]
* Take note of the names of your NICs and their MAC addresses
ip link
or
ethtool -P interfaceName
In my case they are called eno1 and eno2 so you will need to substitute your own values accordingly in the following steps. \\ Setup your switch ports as an LACP trunk. You'll need to consult your documentation on how to do that.
* Edit /etc/sysconfig/network-scripts/ifcfg-eno1
TYPE=Ethernet
BOOTPROTO=none
NAME=eno1
HWADDR=d4:ae:52:c7:20:ea
DEVICE=eno1
ONBOOT=yes
MASTER=bond0
SLAVE=yes
* Edit /etc/sysconfig/network-scripts/ifcfg-eno2
TYPE=Ethernet
BOOTPROTO=none
NAME=eno2
HWADDR=d4:ae:52:c7:20:eb
DEVICE=eno2
ONBOOT=yes
MASTER=bond0
SLAVE=yes
* Edit /etc/sysconfig/network-scripts/ifcfg-bond0
Note: Redhat states as of RHEL7 that the bonding module doesn't support STP so the network switch shouldn't send BDPU packets over the bonded ports: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/deployment_guide/sec-configuring_a_vlan_over_a_bond
BONDING_OPTS="mode=4 miimon=100 lacp_rate=1"
TYPE=Bond
BONDING_MASTER=yes
BOOTPROTO=none
NAME=bond0
DEVICE=bond0
ONBOOT=yes
IPADDR=10.1.1.5
NETMASK=255.255.255.0
GATEWAY=10.1.1.1
DNS1=10.1.1.1
DNS2=10.1.1.2
Restart your server so that the bonding module is loaded with the correct options during boot.
* Check that the bond0 interface is up
ip addr
===== Bridged Network Interface =====
To find the HWADDR do this: ethtool -P
In the /etc/sysconfig/network-scripts directory it is necessary to create 2 config files. The first (ifcfg-eth1) (or ifcfg-em1 or em0 or eth0 etc) defines your physical network interface, and says that it will be part of a bridge:
vim /etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE=eth1
NAME=eth1
HWADDR=00:16:76:D6:C9:45 (Use your HWADDR/mac address here)
ONBOOT=yes
BRIDGE=br0
*The second config file (ifcfg-br0) defines the bridge device
vim /etc/sysconfig/network-scripts/ifcfg-br0
DEVICE=br0
TYPE=Bridge
BOOTPROTO=none
ONBOOT=yes
DELAY=2
WARNING: The line TYPE=Bridge is case-sensitive - it must have uppercase 'B' and lower case 'ridge'
Also, if you have only 1 Ethernet adapter you will want to give the Bridge device an IP on your LAN for management, see static IP example below.
After changing this restart networking (or simply reboot) .
nmcli connection reload && systemctl restart NetworkManager
Example of ifcfg-br0 for static IP:
DEVICE=br0
TYPE=Bridge
BOOTPROTO=none
ONBOOT=yes
DELAY=2
IPADDR=172.18.18.181
NETMASK=255.255.255.0
GATEWAY=172.18.18.1
DNS1=9.9.9.9
DNS2=208.67.220.220
===== VLAN configuration =====
Sources: [[https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/networking_guide/sec-configure_802_1q_vlan_tagging_using_the_command_line|1]]
Configure the parent interface in /etc/sysconfig/network-scripts/ifcfg-ethX, where X is a unique number corresponding to a specific interface, as follows:
DEVICE=ethX
TYPE=Ethernet
BOOTPROTO=none
ONBOOT=yes
Configure the VLAN interface configuration in the /etc/sysconfig/network-scripts/ directory. The configuration file name should be the parent interface plus a . character plus the VLAN ID number. For example, if the VLAN ID is 192, and the parent interface is eth0, then the configuration file name should be ifcfg-eth0.192:
VLAN=yes
TYPE=Vlan
PHYSDEV=ethX
VLAN_ID=14
BOOTPROTO=none
DEVICE=ethX.14
ONBOOT=yes
IPADDR=172.18.18.181
NETMASK=255.255.255.0
GATEWAY=172.18.18.1
DNS1=9.9.9.9
DNS2=208.67.220.220
If there is a need to configure a second VLAN, with for example, VLAN ID 193, on the same interface, eth0, add a new file with the name eth0.193 with the VLAN configuration details.
Restart the networking service in order for the changes to take effect. As root issue the following command:
nmcli connection reload && systemctl restart NetworkManager
===== Steps for a Virtualization Host =====
This will bond 2 or more NICs for LACP, then create a bridge to that bond and add a static IP for the management LAN, then create VLANs for the different networks that virtual guests will be connected to, then create bridges for each of those VLANs.
Why is the management LAN bridged directly to the bond? Because we want to be able to access the virtual host without a managed switch in case of emergency. This means on the switch port, the management VLAN should be untagged and all other VLANs should be tagged.
When creating the bridge name, make it's name reflect the tagged VLAN it's to be used with, e.g. VLAN 20 would be DEVICE=bond0.20 and BRIDGE=br20
==== 1. Create the bond file ====
BONDING_OPTS="mode=4 miimon=100 lacp_rate=1"
TYPE=Bond
BONDING_MASTER=yes
BOOTPROTO=none
NAME="LACP bond0"
DEVICE=bond0
ONBOOT=yes
BRIDGE=br0
==== 2. Modify the interface files for use with bond ====
TYPE=Ethernet
BOOTPROTO=none
NAME=eno1
HWADDR=d4:ae:52:c7:20:ea (use your own)
DEVICE=eno1
ONBOOT=yes
MASTER=bond0
SLAVE=yes
IPV6INIT=no
IPV6_AUTOCONF=no
TYPE=Ethernet
BOOTPROTO=none
NAME=eno2
HWADDR=d4:ae:52:c7:20:eb (use your own)
DEVICE=eno2
ONBOOT=yes
MASTER=bond0
SLAVE=yes
IPV6INIT=no
IPV6_AUTOCONF=no
==== 3. Create Management LAN bridge ====
DEVICE=br0
TYPE=Bridge
BOOTPROTO=none
ONBOOT=yes
DELAY=2
IPADDR=172.18.18.181
NETMASK=255.255.255.0
GATEWAY=172.18.18.1
DNS1=9.9.9.9
DNS2=208.67.220.220
NAME="Management LAN/Interface via br0"
IPV6INIT=no
IPV6_AUTOCONF=no
==== 4. Create VLANs to be used by virtual guests ====
VLAN=yes
TYPE=Vlan
PHYSDEV=bond0
VLAN_ID=20
BOOTPROTO=none
NAME="VLAN20"
DEVICE=bond0.20
ONBOOT=yes
BRIDGE=br20
VLAN=yes
TYPE=Vlan
PHYSDEV=bond0
VLAN_ID=30
BOOTPROTO=none
NAME="VLAN30"
DEVICE=bond0.30
ONBOOT=yes
BRIDGE=br30
==== 5. Create Bridges for use with VLANs ====
(DELAY=2 seems to be needed otherwise the interfaces don't come up)
DEVICE=br20
TYPE=Bridge
BOOTPROTO=none
ONBOOT=yes
DELAY=2
NAME="Bridge for VLAN20"
IPV6INIT=no
IPV6_AUTOCONF=no
DEVICE=br30
TYPE=Bridge
BOOTPROTO=none
ONBOOT=yes
DELAY=2
NAME="Bridge for VLAN30"
IPV6INIT=no
IPV6_AUTOCONF=no
Reboot (sometimes nmcli connection reload && systemctl restart NetworkManager doesn't work...)
==== Notes ====
Per [[https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Virtualization/3.1/html/Installation_Guide/Switch_Configuration_for_Bonding.html|Red Hat the switch ports for bonded ports should be configured]] in a specific way:
11.4.3. Switch Configuration for Bonding
The following is an bond example configuration for a switch. Your switch configuration may look different.
interface Port-channel11
switchport access vlan 153
switchport mode access
spanning-tree portfast disable
spanning-tree bpduguard disable
spanning-tree guard root
interface GigabitEthernet0/16
switchport access vlan 153
switchport mode access
channel-group 11 mode active
interface GigabitEthernet0/17
switchport access vlan 153
switchport mode access