Why did I do this?

1. Adding/removing NICs from a FreeBSD VM (pfSense) will require the VM be rebooted for the changes to take effect. The “sub-interface” VLAN method is much more production-friendly.
2. OVS handles VLANs where Linux Bridges do not
3. OVS handles VXLAN encapsulation where Linux Bridges do not
4. I should be able to fail over to a single piece of hardware completely redundantly.
5. I should be able to deploy only a single piece of hardware in the exact same configuration that I would set up a 3+ piece cluster.
6. Because immutable infrastructure is the shit and scales with an acceptable ratio for federated services.
7. Because Cloud-in-a-ProxMox-Box sounds pretty darn cool.
8. Because this can elastically scale out
9. Because it should be able to federate AAA

Quotes

For corporate IT leaders, it’s practically impossible to deliver best-in-class IT solutions across the enterprise without a multi-cloud strategy,” Murr explains. “However, this isn’t the case for customer-facing products, which quite often ride atop a single IaaS or PaaS provider

References:

• http://www.practicalnetworking.net/stand-alone/routing-between-vlans/
• Configuring VLANs on pfSense
• Using VLANs with OVS and libvirt
• Multi-tenant/VLANs behind a virtualized pfSense firewall
• Open vSwitch Cheat Sheet
• Cluster Networking for Multi-tenant isolation in Proxmox with OpenVSwitch
• Configure Open vSwitch Tunnels
• Configuring VXLAN an dGRE Tunnels on OpenvSwitch
• Proxmox Wiki: Open VSwitch
• Firewall Deployment for Multitier Applications
• Vendor lock-in fears
• Deploy OpenFaaS and Kubernetes on DigitalOcean with Ansible