Wednesday 29 January 2014

Openstack Folsom cant ping instance/VM

The most annoying bug that I came across when setting up Openstack Folsom was that, after creating the a virtual machine it was not able to reach the metadata service and I was not able to reach the virtual machine.

After days of debugging, I found out that it was not only me that was having the same issue. Finally it was reported as a known bug to the openstack developers.

https://bugs.launchpad.net/neutron/+bug/1091605

The problem seems to be that OpenvSwitch devices have been brought back up after after rebooting.  Hence before I upgraded to Grizzily from Folsom, I created the following script and had it running at the boot-up in the network node.

 #!/bin/sh  
   
 #  
 # This script fixes the following bug.  
 # Has to run after reboot.  
 # Bug: https://bugs.launchpad.net/quantum/+bug/1091605  
 #  
 #  
 # Chathura S. Magurawalage  
   
 ((  
 #Check for root permissions  
 # Make sure only root can run our script  
 if [ "$(id -u)" != "0" ]; then  
   echo "This script must be run as root"  
   exit 1  
 fi  
   
 BRIDGE=${BRIDGE:-br0}  
   
 while getopts b:hv option  
 do   
   case "${option}"  
   in  
     b) BRIDGE=${OPTARG};;  
      v) set -x;;  
     h) cat <<EOF   
 Usage: $0 [-b bridge_name]  
   
 Add -v for verbose mode, -h to display this message.  
 EOF  
 exit 0  
 ;;  
      \?) echo "Use -h for help"  
        exit 1;;  
   esac  
 done  
   
 ovs-vsctl del-br br-int  
 ovs-vsctl del-br br-ex  
 ovs-vsctl add-br br-int  
 ovs-vsctl add-br br-ex  
 ovs-vsctl add-port br-ex $BRIDGE  
 ip link set up br-ex  
   
 service quantum-plugin-openvswitch-agent restart  
 service quantum-dhcp-agent restart  
 service quantum-l3-agent restart  
   
 )) 2>&1 | tee $0.log  
   

Ubuntu Kernel panic when creating Openstack instance

Currently I am using Openstack Folsom, nova version  2013.1.4, Installed on Ubuntu 12.04.  I upgraded my compute and network nodes's kernel to  "Linux 3.2.0-58-generic". Since then, when I try to create an instance on a the compute nodes or when I restart the network node, the systems goes to Kernel panic. 

The only fix for this issue I found, is to downgrade the kernel from "Linux 3.2.0-58-generic" to "Linux 3.2.0-55-generic" on all compute nodes and network node to keep things consistent.  The controller nodes does not seem to be affected by this kernel bug.

Also make sure to install the header if the corresponding kernel, if you are installing the kernel. Or if you have already got "Linux 3.2.0-55-generic" kernel installed, then just change it to the default kernel in grub records. 

First find out the index of the kernel in the grub records.

grep menuentry /boot/grub2/grub.cfg

Then the fist answer of the following question should help when editing the grub config.

http://askubuntu.com/questions/216398/set-older-kernel-as-default-grub-entry

Openstack Grizzily Error "FATAL: Module openvswitch_mod not found" when start openvswitch or at startup

I came across this error on Openstack Grizzily installed on Ubuntu 12.04 LTS, when I upgrade or degrade the linux kernel. So its very likely that you will get this error after running a "dist-upgrade" on your system.

The cause of this problem seem to be that when a different kernel is used, the openvswitch modules have not been built  with it.

Error:
FATAL: Module openvswitch_mod not found.
 * Inserting openvswitch module
 * not removing bridge module because bridges exist (virbr0)
invoke-rc.d: initscript openvswitch-switch, action "load-kmod" failed.

Solution:

Copy the following script to a file. Then save.

 #!/bin/bash  
   
 sudo apt-get install linux-headers-$(uname -r)  
   
 depmod  
 modprobe openvswitch  
   
 sudo apt-get remove openvswitch-switch openvswitch-datapath-dkms quantum-plugin-openvswitch-agent  
   
 sudo apt-get install openvswitch-switch openvswitch-datapath-dkms quantum-plugin-openvswitch-agent  
   
   


Then make it make the file executable.

Go to the file directory then execute following command in terminal:

sudo chmod +x filename

Finally execute the file.

./filename 

Openstack Folsom Error "FATAL: Module openvswitch_mod not found" when start openvswitch or at startup

I came across this error on Openstack Folsom installed on Ubuntu 12.04 LTS, when I upgrade or degrade the linux kernel. So its very likely that you will get this error after running a "dist-upgrade" on your system.

The cause of this problem seem to be that when a different kernel is used, the openvswitch modules have not been built  with it.

Error:

FATAL: Module openvswitch_mod not found.
 * Inserting openvswitch module
 * not removing bridge module because bridges exist (virbr0)
invoke-rc.d: initscript openvswitch-switch, action "load-kmod" failed.

Solution:

Copy the following script to a file. Then save.

 #!/bin/bash  
   
 sudo apt-get install linux-headers-$(uname -r)  
   
 sudo apt-get remove quantum-plugin-openvswitch openvswitch-switch quantum-plugin-openvswitch-agent openvswitch-datapath-dkms openvswitch-common quantum-common python-quantum  
   
 apt-get install openvswitch-switch  
 mkdir -p /etc/quantum/  
 apt-get install quantum-plugin-openvswitch-agent quantum-dhcp-agent quantum-l3-agent quantum-server  
   
 depmod  
 modprobe openvswitch  
   

Then make it make the file executable.

Go to the file directory then execute following command in terminal:

sudo chmod +x filename

Finally execute the file.

./filename