niemueller.de

Extending VM disk (1 comment)

Thu, 12 Jan 2012 13:37:58 +0100

Consider the following situation: a CentOS 6.2 host (this can also be Fedora or RHEL for that matter) using KVM to run virtual machines (VM). The host uses the Logical Volume Manager (LVM) as storage to create disks for the VMs. Now one of the machines needs more disk space then was originally envisioned and thus the space must be extended. The guest system in turn is also running CentOS 6.2 and itself using LVM to setup its storage.

So we have the following entities to deal with:
Host: The physical machine running CentOS 6.2 and KVM.
Guest: Virtual machine running in KVM.
vg_host/lv_vm: The logical volume on the host system which constitutes the disk drive of the Guest.
/dev/vda2: The second partition on the first virtio disk in the Guest. It contains the only physical volume used on the Guest.
vg_vm: The volume group on the Guest where the only physical volume is /dev/vda2.
vg_vm/lv_root: The logical volume in the Guest volume group which facilitates the root of Guest's filesystem.

To keep things simple we assume that there is only a single mount point for the Guest, i.e. /usr etc. are not split off onto own partitions or logical volumes. Replace vg_host, lv_vm, and vg_vm with the actual values from your system. vgdisplay and lvdisplay can help you to find out about it.

So the following steps are necessary to get more disk space into the VM.

On Host: Extend vg_host/lv_vm
On Guest Extend /dev/vda2 partition
On Guest Extend /dev/vda2 physical volume
On Guest Extend vg_vm/lv_root logical volume
On Guest Extend vg_vm/lv_root file system

Therefore, execute the following steps.

0. Stop the VM:

virsh shutdown VM

1. On Host: Extend vg_host/lv_vm: this code extends by 10GB, change as appropriate, make sure there is enough free space left in the volume group.

lvresize -L+10G vg_host/lv_vm

2. On Guest Extend /dev/vda2 partition: This is critical, take special care! It is important to choose the proper starting cylinder. The suggested start might be wrong, therefore first print the current start and keep it. Sorry this is the German output, but you should be able to match it properly.

fdisk /dev/vda2
Befehl (m fÃ¼r Hilfe): p
   GerÃ¤t  boot.     Anfang        Ende     BlÃ¶cke   Id  System
/dev/vda1   *           3        1018      512000   83  Linux
Partition 1 endet nicht an einer Zylindergrenze.
/dev/vda2            1018       21391    10267648   8e  Linux LVM
Partition 2 endet nicht an einer Zylindergrenze.
Befehl (m fÃ¼r Hilfe): d
Partitionsnummer (1-4): 2

Befehl (m fÃ¼r Hilfe): n
Befehl  Aktion
   e      Erweiterte
   p      PrimÃ¤re Partition (1-4)
p
Partitionsnummer (1-4): 2
Erster Zylinder (1-62415, Vorgabe: 1): 1018
Last Zylinder, +Zylinder or +size{K,M,G} (1018-62415, Vorgabe: 62415): 
Benutze den Standardwert 62415

Befehl (m fÃ¼r Hilfe): p

Platte /dev/vda: 32.2 GByte, 32212254720 Byte
16 KÃ¶pfe, 63 Sektoren/Spur, 62415 Zylinder
Einheiten = Zylinder von 1008 Ã� 512 = 516096 Bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000407f8

   GerÃ¤t  boot.     Anfang        Ende     BlÃ¶cke   Id  System
/dev/vda1   *           3        1018      512000   83  Linux
Partition 1 endet nicht an einer Zylindergrenze.
/dev/vda2            1018       62415    30944136   83  Linux

You now need to reboot because the partition is currently in use.

3. On Guest Extend /dev/vda2 physical volume: extend to maximal size. Make sure you have rebooted your system or otherwise this won't work (even though it doesn't report any errors).

# pvresize /dev/vda2
  Physical volume "/dev/vda2" changed
  1 physical volume(s) resized / 0 physical volume(s) not resized

4. On Guest Extend vg_vm/lv_root logical volume

lvresize -l+100%FREE vg_vm/lv_root

5. On Guest Extend vg_vm/lv_root file system: this can be done while the file system is mounted and active. But you'd probably be screwed on a power failure or so.

resize2fs vg_vm/lv_root

Done! Now df -h should report the full space.

The long road to Kerberos/LDAP/NFSv4 and a shortcut (1 comment)

Fri, 22 Apr 2011 00:38:04 +0100

Recently we got a new server at university. It will replace the current file and authentication server which uses NIS and NFSv3. It provides us with a good opportunity to upgrade to something more secure and efficient. So I took on the journey to setup OpenLDAP as user directory, Kerberos for authentication, and NFSv4 for file sharing. But the way took a little time, only to find out later a shortcut which makes it almost a piece of cake. The long way still provides insightful information, therefore it's still useful to try it do-it-yourself style first.

Rather than writing yet another howto, I will link to documents that I used during the initial setup. The most relevant source is the Kerberos/LDAP/NFSv4 HOWTO. It describes the way pretty much step by step. If you want to replace NIS like us the Replacing NIS with Kerberos and LDAP HOWTO is a good read. We use CentOS 5.6 on the file server. The most common pitfall to run into then is that the NFS code there only supports weak ciphers. As client we used a Fedora 14 machine, which will only try strong ciphers by default (cf. for example Red Hat/Fedora bug reports #652273 and #573968). First edit /etc/krb5.conf and set allow_weak_crypto = yes in the [libdefaults] section. Then make sure to add -e des-cbc-crc:normal to the ktadd command to export keys to the keytab for the NFS service keys (nfs/host@REALM). It is described in the NFSv4 Kerberos Setup Guide, as well as the mentioned howto, but something to be easily missed and hard to diagnose when new to the system. The NFSv4 Linux FAQ provides some tips for NFS problems. Another document describes common Kerberos issues. When googling the Ubuntu NFSv4 HOWTO frequently comes up, but it does not provide much useful additional information.

Once I had it running I was pointed to FreeIPA. It is an integrated solution that combines the 389 directory server, Kerberos, and the Dogtag Certificate System with nice console administration tools and a helpful WebUI. I went straight for version 2.0.0 for which a new Enterprise Identity Management Guide is currently work in progress. FreeIPA 2.0.0 is not perfect, yet, and I had to report a few bugs, but it makes the overall process much easier. You still need to allo weak cryptos if CentOS/RHEL 5 is involved by yourself, IPA won't do that automatically for you. If you run into problems that the authentication fails (add -vvvvv as RPC idmapd and gssd/svcgssd flags to see this), wipe out the keys on both server and client and get new ones with the -e des-cbc-crc:normal for the NFS service keys! The nice people behind the project are extremely helpful if you ask nicely via IRC.

Have fun and enjoy secure authentication and encrypted file sharing as we hopefully will once the system is deployed.

Fawkes in Google Summer of Code 2011 with Fedora

Wed, 23 Mar 2011 12:47:58 +0100

Fawkes participates in the Google Summer of Code 2011 under the umbrella of the Fedora Project.

The Fedora Robotics SIG has completed the Fedora Robotics Suite, a set of robotics related software packages that are readily available in Fedora Linux. Fawkes is one prominent member of this package set. The original idea also envisioned creating an educational application, where a user would learn step by step to control a robot, then instruct, and finally program it. The project could not be completed due to a developer shortage.

The project has now been proposed as one possible candidate for the Google Summer of Code 2011 with the Fedora Project as mentoring organization. If you are a student with a background in robotics and experience in C++ software development please consider applying for this project. The Fedora Robotics SIG comprises many developers of upstream software projects providing a good way to get in touch with those projects. It will also be a very visible feature of the Fedora Robotics effort providing a good show case for later applications.

To apply please read information on the Fedora GSoC 2011 page and contact Tim Niemueller of the Fedora Robotics SIG.

UniMensa RWTH Datenbank wieder verfügbar

Mon, 07 Feb 2011 11:04:03 +0100

Die wÃ¶chentlich aktualisierte RWTH Mensa Datenbank ist nun wieder verfÃ¼gbar. Vielen Dank an Holger Jeromin fÃ¼r das Beisteuern eines Scripts zum Auslesen der neuen Website des Studentenwerks.

Damit stehen die MensaplÃ¤ne wieder fÃ¼r Palm und per WAP zur VerfÃ¼gung, und auch andere Dienste wie das Aussenden per Skyper von der Amateurfunkgruppe der RWTH-Aachen sollten damit wieder funktionieren. Sollte es noch Probleme geben, bitte eine kurze Nachricht an mich.

Fedora now includes Fawkes

Tue, 16 Nov 2010 18:22:08 +0100

Recently we proposed Fawkes for inclusion into Fedora. The review has completed and the package has been accepted! Fawkes and its development environment can now be installed out of the box, see the download page for instructions.

On a related note Fawkes has been chosen by the Fedora Robotics SIG to play a major role in the Fedora Robotics Spin along with Player and Stage. The main idea of this spin is to have software on a LiveDVD providing a ready-to-use robotic simulation environment, with a demo application to introduce users step by step to the available software. Additionally as much robotics related software as possible, ranging from packages for hardware access (laser scanners, robotic arms) and robotic middleware frameworks, to full-blown simulation environments and libraries relevant for robotic applications like vision processing or task control will be added. The Robotics Spin has been proposed as a Fedora 15 Feature

For this LiveDVD Fawkes will be used in combination with Player and Stage to provide the environment for the demonstration application. In particular, the behavior engine based on Lua will be the tools used to guide the user to complete tasks of increasing complexity within the simulation environment. Among others like Player/Stage and the RoboCup 3D Soccer Server Fawkes will be one of the highlighted robot software packages.

Twitter annoyances (2 comments)

Thu, 02 Sep 2010 22:14:54 +0100

After Twitter disabled basic authentication yesterday, basically all Open Source Twitter clients ceased working (e.g. bug #629192). I tried some clients from the Fedora repositories, but none seems to work. I liked the simplicity of Pino, but now it's just dead wood. Mauku on the N900 is also broken. Not even the personal timeline RSS feed works anymore, although still advertised on the Twitter website.

ars technica has an article which describes the the problem with FOSS clients on page 2 and it doesn't look good so far. It seems finally to be time to switch to identi.ca (me).

Releases of Fawkes 0.4 and roslua

Tue, 17 Aug 2010 06:05:50 +0100

Yesterday I have released Fawkes 0.4, the robot software framework we have developed at RWTH Aachen University, with contributions from TU Graz and University of Cape Town. Various obstacles made it one year for the release to materialize, but finally it's here with lots of new stuff, check out the announcement. Many people have helped with this release, good to see. Fedora packages are on their way, now that we support system-wide installation.

Recently I have also been busy working on software for another robot software framework -- ROS. As part of my research stay with Dr. Siddharta Srinivasa from Intel Research Pittsburgh at the Carnegie Mellon University I have developed roslua, a ROS client library for Lua. This is only the first step in porting Fawkes' behavior engine to ROS. You can find the full announcement on the ROS website. So after "the discovery" I still can be kind of productive. More is cooking, stay tuned.

To sleep or not to sleep

Wed, 04 Aug 2010 20:15:26 +0100

Today looked for a problem for about an hour which I thought might be worth sharing. The following (most likely common) scenario was to be implemented: one process wants to start another and be notified if something happens to the child process. The proper way to achieve this is to fork(), then execl() the new program in the child, and have the parent call wait() to check for state changes of children.

Now, in I'd guess likewise common is to start with a test program that incorporates sleep(). Fail! After an hour it turns out that the sleep() call will block the SIGCHLD signal (cf. sleep.c). But it does not unblock the signal afterwards. Since the wait() call depends on SIGCHLD, no event would be received
and wait() would starve.

So either you need to unblock the signal after calling sleep(), or better call usleep() which does not suffer this problem (cf. usleep.c).

Dead routers which did not die

Sat, 13 Mar 2010 12:06:42 +0100

In the last few months three routers I maintained died on me. All where little home routers. Two of them Asus WL-500GP, and recently a Fonera (first generation). The routers appeared to be live, but could not be reached, neither over Wifi nor Ethernet. The LEDs were lit or sometimes flashing. On the second router, one of the Asus ones, on a closer look it had some strange look, where all LEDs were off shortly, then back to seemingly normal. Sniffing on the Ethernet revealed some strange packages which contained the hex string 0xDEADDEAD and a message to boot into failsafe mode.

Now, to make a long story short: all the routers were just fine, what was indeed broken were the power supplies - on all of those three. Replacing the power supply brought the device back to normal in all three cases. I haven't made an effort to measure what caused this problem, but it seems to be insufficient current.

So next time a (home) router dies on you, consider borrowing a compatible power supply first and test the device again, it might save some money and effort, besides reducing the electronic scrap on the planet.

Setup git tracking branch for existing branch (1 comment)

Sat, 20 Feb 2010 12:59:08 +0100

Something that I did many web searches is how to setup up tracking of a remote git branch for an existing local branch. That regularly happens for example if I start with a local topic branch, and then push it to the central repository. Now, git does neither automatically setup that this (new) remote branch should be tracked for the local branch nor did I find a convenient command or argument to do so. So here is what is required, assuming that the remote repository is named origin, the local branch is topicbr, and the remote branch is timn/topicbr, and you are in the local git repository:

git config branch.topicbr.remote origin
git config branch.topicbr.merge refs/heads/timn/topicbr

.
It's that easy and now I have a place to look it up next time I need it...