Linux Machine Up on Multiple Network/VLAN/Subnet

Use Cases to enable multiple subnet/vlan/network on your machine:

Some times you will need your server to be talking through different networks (VLANS) for some reasons. Example, I need to create a hypervisor (Physical server which holds all my Virtual Machines) with KVM which gives VMs in multiple VLANs. Taking this example to explain this setup:

I have two VLAS (Virtual Lan or Two physically different LAN):

1. Production VLAN/SUBNET:

192.169.75.1/24
Gateway: 192.168.75.1

2. Dev QA VLAN:

192.168.76.1/24
GateWay: 192.168.76.1

Need to Create the hypervisor which can Give Virtual Machines on both the VLAN/SUBNET. If we simply bring up the hypervisor with two NICs up on both the VLAN, either of the NICs will be working not both. The ideal setup should be as below:

1. Create two bridge interfaces on the active physical interfaces in "/etc/network/interfaces" file

auto lo
iface lo inet loopback

auto br0
 iface br1 inet static
 address 192.168.75.2
 netmask 255.255.255.0
 network 192.168.75.1
 bridge_ports eth0
 bridge_stp off
 bridge_fd 0
 bridge_maxwait 0
 post-up route add default gw 192.168.75.1 metric 1
 pre-down route del default gw 192.168.75.1

auto br1
 iface br2 inet static
 address 192.168.76.2
 netmask 255.255.255.0
 network 192.168.76.1
 bridge_ports eth1
 bridge_stp off
 bridge_fd 0
 bridge_maxwait 0
 post-up route add default gw 192.168.76.1 metric 2
 pre-down route del default gw 192.168.76.1

2. Restart the Network #/etc/init.d/networking restart
3. There will be rp_filters enabled for the bridge interfaces. Need to disable that. In this case br0 and br1:

As Root
#echo "0" >/proc/sys/net/ipv4/conf/br0/rp_filter
#echo "0" >/proc/sys/net/ipv4/conf/br1/rp_filter
#echo "0" >/proc/sys/net/ipv4/conf/all/rp_filter

This will make sure both the interfaces are active and reachable on the hypervisor.

Make sure the rp_filters are disabled on reboot as well by adding the above commands in some startup files.

You are Done!