MythTV Build Journal

Here's a status report on my MythTV box.

As you may remember, I switched my main desktop from a PC running Debian GNU/Linux to a Mac G4--also running Debian, of course. As I posted earlier, I did this so that I could use the PC as my MythTV backend machine, thereby saving me some money because I didn't have to buy a whole new machine for the backend. It's been a few weeks since the switch, and now I feel comfortable moving ahead with the MythTV project, pretty sure that I've gotten everything I need off of the old machine.

Also, I finally received the new gigabit network switch and gigabit network cards that I ordered from Buy.com.

So, the other day, I installed one of the gigabit network cards and my Hauppauge TV capture card, that I've had for almost 3 months, into the PC.

On reboot, the network card was instantly recognized and working, but the kernel doesn't appear to have drivers for the Hauppauge card already. I think I may have known that at some point, but I've since forgotten. So, I'm hoping that this weekend I'll have time to get the card up and running and maybe even try installing MythTV. If I do get around to installing the MythTV software, it'll be in my already existing Debian installation on the PC. I haven't ordered the big hard drives yet, so at this point I'm just learning how it all works.

So, my current to-do list for the MythTV box is:

• Get Hauppauge drivers installed.
• Run an extra cable coax line
• Order the hard drives

I'll post again if I get anything working.

I posted this entry back in November explaining my thinking behind what I planned to use for storage on the MythTV box. This post is about my latest, updated, thinking on the matter.

As you may recall, my goal was to start out with a 1 terabyte RAID5 array handled by LVM. The RAID5 is for redundancy in case of drive failure, and the LVM was for ease of future management and growth.

Since then, I've decided that it would be wise to start smaller and then grow when necessary. Also, I've been doing some more reading and have changed my thinking somewhat about how it should be configured.

I've seen it suggested a few times to make smaller RAID arrays and group them together using LVM. For example (and a good example at that, since this is actually what I've decided to do), if I bought 2 300 gig drives, the first thought may be to make them into one 300 gig RAID1 array. Then, if you wanted to grow your storage space, you'd have to buy 2 new drives and make another RAID1 array and then add it to the logical volume. The suggestion that I've been reading is to split it up into, say, 3 100 gig RAID1 arrays (grouped using LVM, of course). It took me a while to realize the benefit in this. The benefit is flexibility. With smaller RAID arrays, it is easier to shuffle them around in the future.

Maybe just laying out what I'm planning in detail will help.

For now, I'm going to buy 2 300 gig drives (hda and hdb). I will partition each into 3 100 gig partitions (hdx0, hdx1, hdx2) which are then configured as 3 100 gig RAID1 arrays (md0, md1, md2). I've attempted to illustrate this in text below:

hda0 + hdb0 = md0
hda1 + hdb1 = md1
hda2 + hdb2 = md1

The total space I'll have available with this setup is 300 gigs.

Then, in the future, when I need more space, I can just buy one new drive and reshuffle. So, let's say I buy a 500 gig drive (hdc). I would then partition it into 5 100 gig partitions (hdc0, hdc1, etc.). Then what I would do, is (basically) break the existing RAID1 arrays by removing the partitions on hdb from the arrays. This wouldn't result in data loss because the data is mirrored on both drives, so losing one drive doesn't cause loss.

The next step would be to rebuild those RAID arrays using 3 of the new partitions on hdc. Basically, at this point it would like this:

hda0 + hdc0 = md0
hda1 + hdc1 = md1
hda2 + hdc2 = md2

That leaves me with 3 partitions on hdb (one of the original 300 gig drives) and 2 partitions on hdc (new 500 gig drive). So, I'd just make 2 new arrays like this:

hdb0 + hdc3 = md3
hdb1 + hdc4 = md4

After extending the logical volume onto the new arrays, I'd then have 500 gigs of usuable, RAIDed, space. I'd also have a 100 gig partition left unused (hdb2).

I can continue doing this as I need more space. After doing this twice, though, I'll have to start retiring drives. This shouldn't be any harder than what I've discussed above, though. The only difference is that when I break the RAID arrays, the drive that was removed from the arrays would also be physically removed and replaced with the bigger drive.

So, I finally got around to switching my main desktop machine to the Mac (original post. It did not go as smoothly as the other 3 machines mentioned in the original post, but is now up and running.

The first and main problem manifested itself during the first boot after the initial install. The install CD booted correctly and the install went smoothly. But, after it was complete and the system rebooted itself, it would hang after the bootloader with several lines of text starting with OPENPIC. After some googling, I found that it was a known bug. I was able to boot by telling the machine that it only had 768M of RAM, when in fact it had 1.5 gigs of RAM. After some messing around with the RAM, I discovered that it would work fine if I only had 1 gig of RAM installed. So, I just went with that. It kind of sucks that I'm limited to 1 gig, but my previous machine only had 512 megs, so I'm ok with it. This problem took me a couple weeks to figure out (well, a couple weeks of occasionally thinking about it and messing with it, then getting frustrated and going back to my functioning machine).

After doing a little reading up on the Linux Logical Volume Manager (LVM) for this post, I've basically decided that I'll use it every time I install a new Linux box. It may very well be that most of the time, I won't really need it or have cause to take advantage of it's features, but it doesn't hurt to have it there and when I do need it it'll be there. The Mac has a 20 gig drive in it, which I was going to use as the system drive. So, during the install, I partitioned the drive so that it had a 60 meg /boot partition, a 1 gig / partition (I have read that you want to keep your / and /boot directories on non-LVM partitions), an 800 meg swap partition, and the remainder was partitioned for use as an LVM physical volume (PV). So, I setup LVM to have one volume group (VG) that contained that PV. Within that VG, I setup the various directories (mount points really, but I figure if you care, you probably already know that) as logical volumes (LVs). The LVs I setup were for /usr /var /tmp /var/log, etc. I like to have them all split up so that I can assign different options to them (I'm pretty liberal with setting mount points to be noexec--files can be executed here--for example). I also setup a LV for /home.

Part of the install of the Mac was going to entail moving a 160 gig drive from my old machine to the new one. I wanted that drive to be added to the existing /home directory/LV, but first it had to be backed up and reformatted for various reasons. This was a perfect chance to make use of some of the LVM goodness I'd been reading about. So, the way it worked follows.

I backed up the stuff I wanted to keep off of the 160 gig drive to another drive and then shut down the machine. I yanked both drives from the PC, hooked them up to the Mac and booted it up. Once booted, I started up cfdisk to partition the 160 gig drive into another 800 meg swap partition and the remainder as one partition to be used by LVM. The actual LVM commands were incredibly simple. There's probably a nice graphical way to do it, but I'm a command-line guy, so here's the commands I ran:

First, define the new partition (/dev/hdb2) as a physical volume:

$ pvcreate /dev/hdb2

Next, add the new PV to the existing volume group (named vg0):

$ vgextend vg0 /dev/hdb2

Extend the logical volume (name lvHome) to use the newly available space. You have to tell it how much new space to use, so I used all that was available, which was about 158 gigs:

$ lvextend lvHome 158G

The only thing left to do was to grow the /home filesystem. I'm using XFS for my file system. One of the reasons I'm using XFS is that I've read that it is easy to change it's size. To grow /home, I ran this:

$ xfs_growfs /home

With no other options, xfs_growfs will just grow the filesystem to the largest available size. So now, df (command to show filesystem disk usage, shows this (this is after I'd copied all the backed up data to /home):

$ df -h /home

Filesystem             Size    Used  Avail Use% Mounted on

/dev/mapper/vg0-lvHome   160G   65G   95G  41%   /home

Other than those two things, the install went very smoothly. My cable internet connection was unusually fast that night, so the downloading and installing of software went very quickly. After that, I simply copied Cyndi and my config files over into our home directories and when I logged in, everything was just as it had been on the old machine (oh, the beauty of text-file based configuration!). The only difference was that now it was in an ugly Mac G4 box, and the alt key is in the wrong place. I'm thinking of mapping the dumb apple/command key to alt, so we don't have to remember that it's different.

Actually, there are a few other differences, but nothing major. Mostly, the differences are related to the different keyboard (F15 instead of Pause/Break, for example). Also, Macromedia/Adobe has never released a Linux PowerPC version of their Flash player. I'm fine with this, as I usually don't install Flash because it annoys me so, but Cyndi likes to have it. So, through the wonder of SSH X-forwarding, what I will do (haven't yet), is setup a menu item for her that will run a web browser on the kids machine (Linux PC, for which there is a Flash player plugin) and display it on her desktop. In this way, she'll be able to view sites with Flash. Oh, how I loathe Flash.

In working inside the Mac, I've reaffirmed that I do prefer the more standardized layout of PC hardware. The Mac is really intended to be bought, used, and discarded when you buy the next one--basically, a black box. It's not really intended to be upgraded. Even something as simple as adding another hard drive is more difficult than it typically is on a PC.

All in all, it was a pretty easy process, with the exceptions noted above, of course. Hopefully, now this means that I'll be able to start actually working on the MythTV box...

I'm somewhat shooting for a terabyte of storage for my MythTV box. You may be thinking that a terrabyte is a little extreme, and you may be a little right. But, in addition to being a Digital Video Recorder (DVR), I'm also planning on copying all of my DVDs to it, so that I have instant access to my entire collection. I'm looking into the space considerations of just ripping them directly to disk or transcoding them into a smaller video format. In either case, I'll probably forego the DVD extras.

So, this is a lot of data. A lot of data that I don't want to lose due to the effort it's going to take to get it on the disk. Also, it's going to be too big to back it up in a cost-effective manner. So, the decision of how to construct the underlying storage system is one of the more important ones in the entire process of constructing my MythTV box. The requirements are few (in no particular order):

• Big: a terabyte is big.
• Redundant: since it's too big to back up, it has to be able to handle a drive failure or two without losing all my data.
• Fast: This one isn't as important, but it will need to be able to handle writing or reading a video fast enough to stream over the network, as well as to serve as a file server for the rest of my computers. I'm not too worried about this, since the network is likely to be the bottleneck here.
• Able to get bigger: this is more of a nice-to-have than a requirement, actually. Drives are increasing in size all the time, and over time I imagine I'll end up needing more storage. So, if it can grow by adding drives, or by replacing smaller drives with bigger ones, that'd be good.

The rest of this entry will discuss the various options that are available to me, or at least the options that I know of.

RAID -- drive arrays

My plan all along has been to get enough drives to make a terabyte-sized RAID5 array with one live failover drive. Looking at the list above, this would satisfy all of the requirements: big, redundant, and fast. It doesn't satisfy the nice-to-have feature of expandability. RAID5 arrays (as far as I know) can't be grown without redoing the entire array. So, if I wanted to grow the array, I'd buy a new drive, and construct a new array using all of the disks. The problem is that I'd lose all the data since I can't back it up elsewhere while making the new array. I haven't really be worrying about that problem, though, since a terabyte is huge, and really should be enough for anyone.

Linux Logical Volume Manager

But, recently, I've been reading a little bit about the Linux Logical Volume Manager (LVM). LVM is a way to group multiple devices (drives or RAID arrays) into a logical volume that can be resized on the fly. So let's say, as a small example, that you buy a 10 gig drive and partition it into two partitions: a 5 gig / partition (/ is the top level in Linux) and a 5 gig /home partition. If you fill up your 5 gig /home partition, the only way to grow it is to put in another drive (another 10 gigger for the sake of this example), copy the data over, and tell the system to use the new space as the /home partition. This leaves you with a 5 gig / partition on the first drive, and a 10 gig /home partition on the second drive. You've also got 5 gigs of unused space on the first drive where the /home partition used to be. If you had setup that original drive as one big LVM volume group with the same two partitions as logical volumes (5 gig / and 5 gig /home), when /home filled up you'd have a lot more flexibility in how you could handle it. Assuming there was empty space on the / logical volume (LV), you could simply reallocate that space to the /home LV. Or, if you added in a second 10 gig drive, you could simply add that drive to the LVM volume group, and add it to the /home LV, giving /home 15 gigs.

So, LVM is cool and all, but scoring it against my list shows that it's a miserable failure for the purposes of my MythTV box. Yeah, it can be big and become bigger, but it's not redundant and it's not necessarily fast. But, as I mentioned, LVM can use either single hard drives or RAID arrays as the underlying storage devices. Instead of just having two 10 gig drives, as I said in the example, it could be RAID1 arrays.

Distributed file systems

On a side note, I've also thought about using a distributed file system that spreads the storage out amongst the various machines on the network. This would be really cool, but would make the usage of my MythTV box reliant on most or all of the other machines being up and running. This would also add another layer of technology that I know nothing about, adding to the complexity of the project. I do a lot of reading and tinkering with cluster technology, and this is one area that I'm really interested in, so I'll get to it eventually, just not for this project.

And the winner is...

So, what I think the solution that I've decided on is a combination of both. Starting from the bottom, I'll have several big hard drives configured into a RAID5 array. This will give me size, redundancy, and speed. I'll make that RAID5 array into a LVM volume group with one huge logical volume that encompasses the entire array. Then, in the future, if I need to add to the available storage, I can more easily do so using the LVM. I'd imagine that by the time I need more space, drives will be sufficiently large so that I can easily switch to one or a couple of RAID1 arrays. Why would I switch to RAID1, you may be asking? The reason I'm going RAID5 now, instead of a couple 500 gig RAID1 arrays made into one LV, is due to physical space and cost. 500 gig drives are currently running around $350, and I'd need 4. I could do a series of smaller RAID1 arrays, but the problem there is that my case can't handle that many drives (I may be looking into a larger case, though). But, with RAID5, I can achieve my goals using smaller drives. Coupled with LVM, RAID5 is the perfect solution for now, with room to grow in the future.

If anyone has any opinions on the matter, or information that I may not have considered, I would greatly appreciate it if you'd leave a comment. As I said above, this is one of the most important (and difficult) choices, so I'd like to get it right.

Further info

A few side notes and bits of further information:

First, I actually started this post to mention this incredible deal at NewEgg.com, but as is often the case, I digressed like a madman and ended up here. The deal is for 250 gig SATA drives for $110. If I used these drives, I could make my terabyte RAID5 array for $550. If you didn't care about RAID, you could get a terabyte of storage for $440. Absolute madness.

Second, some more info on LVM:

• LVM Howto -- in-depth instructions for using LVM
• Linux LVM - Is It Ready for Prime Time? -- slashdot article and discussion of LVM from a couple years ago

Third, various articles, blog entries, etc. related to setting up large storage systems using Linux:

• Notes on Building a Linux Storage Server -- this link is specifically about building a storage system for a MythTV box. I haven't read it, but it looks pretty detailed and looks like it addresses many of the issues I've discussed above.
• Building a Massive Single Volume Storage Solution? -- slashdot article and discussion about building a huge (25 terabytes to 1 petabyte) storage system. There is a lot of great info in the comments, as always.
• Ideas for a Home Grown Network Attached Storage? -- another /. article. Again, great info in the comments.
• Experiences w/ Software RAID 5 Under Linux? -- another /. article. I think I read this one when it was posted in October '04. Some good info in the comments about how to periodically check for signs of impending hard drive failure.
• A DVD Jukebox Without the DVDs? -- /. again. Comments include discussion on storing straight DVD rips vs. transcoded videos.

I've been reading a ton about setting up MythTV in preparation for building mine. I've decided to post links to things I read here. Yes, this is another instance of me trying to close some browser tabs. So, here's a couple I've got open right now:

A Beginning Look At MythTV

This seems to be a pretty good intro to MythTV. I'm not sure why exactly I have this one open, since I'm pretty well acquainted with the basics already...

O'reilly is doing an ongoing feature about building a MythTV box (somewhat similar to my own MythTV Build Journal, but way more informative):

Building My MythTV Box, Part 1: Hardware
Building My MythTV Box, Part 2: Software
MythTV Part 3: Getting Quiet Enough for the Living Room