Inside X - OS X Disk Maintenance Part 1


Disk maintenance is a complex and diverse subject for any computer user, encompassing more than what I can provide in an overview article or two. Hopefully my words of wisdom ;-) will get you on the right path, instill some good habits, and dispel odd myths that have popped up about Mac OS X storage volumes.



Preface:

Let me lay some terminology on you. The magnetic media drive you have hard wired to your computer is called a hard drive. Any hard drive can be formatted. There are dozens of different ways to format a hard drive, depending up how you want to use it.

Mac OS X allows users to install and run MOSX on two different disk formats. The most popular format is HFS+ which Apple introduced during the development of Mac OS 8. The other format is called UFS, or UNIX File System. There is no reason for a regular Mac OS X user to use the UFS format. Only experts who KNOW they require it will use it. Most likely they will be sharing the volume on a network with other flavors of UNIX.

It is possible with many operating systems, especially Macintosh, to create what are called partitions on a hard drive. This allows the user to break up the hard drive into convenient chunks of memory space. On the ye olde Mac OS platform you can install one blessed, or active, System Folder on each partition. With Mac OS X you can add an additional active System folder to run X. The result allows you to boot your Mac from any of the partitions you create. Each partition can be referred to as a 'volume,' as in volume of data. Volume is the term I shall be using henceforth.



With the arrival of Mac OS X, the need to repair disk volumes will theoretically decrease as it is far less likely for MOSX to crash than any previous version of MOS. Typically, disk crashes are when damage is done to volumes. However, despite everyone's best intentions, it is still possible to crash a Mac OS X system. If the crash is caused by a software problem, it usually shows up as what is called a Kernel Panic. A mild Kernel Panic lets you type the letter R to restart your Mac again. If this works you may find that your system volume has suffered no ill effects. A hard Kernel Panic ends with a lockup of the computer such that typing R does nothing, nor does anything else you can think of. This forces you to use a hard restart of your Mac, which is almost guaranteed to damage to your volume structures. The worst of all crashes is caused by hardware problems. They can result from a variety of problems including the use of the wrong type of RAM in your Mac, having faulty RAM, an overheating CPU, a faulty CPU, or having a bad solder joint on your motherboard or CPU daughter board. These crashes typically lock up your computer in the middle a random activity. Sometimes the cursor still works, sometimes not. It is unrecoverable, forcing you again to use a hard restart of your computer, most likely resulting in volume structure damage.

Volume structures are used by the systems of computers identify exactly what is going on at any one time on a volume. Think of it as a big database that knows everything about your hard drive volumes. If you have watched a disk utility program such as Disk First Aid crank away, checking out your volumes, you have seen obscure terms like 'B-Tree' and 'Mount Check.' If you really care what they mean you might want to look them up on the Web, look through the documentation for your disk repair utilities, or get a book on computer maintenance. (Gene Steinberg has an excellent computer maintenance book, which I will be reviewing soon).

Priority 1: Volume Backups

Rule Number One in computing is MAKE A BACKUP. There is no other rule number one. Anyone who says otherwise is none too serious about computing. I am so adamant about this rule that I believe no one should work with computers at all until they have a backup system in place. This may be as simple as copying your Documents folder over to another volume on a regular basis. This gives you have an extra copy of your most critical documents in preparation for the day your volume dies. Ideally, backups should be done to another hard drive, or CD, or zip disk, etc., not simply another volume. Alternatively, you could use dedicated backup software. All hard drives die. All your hard drives WILL die, sooner or later. When it happens it seems all too soon. Be prepared! How often you backup is dependent upon how active you are with your computer and how critical your files are. Just make sure you back up regularly, and that you have at least two backup sets you rotate over time. This means that you make one backup on day A to volume 1, then you make another backup on day B to volume 2, then you make backup C to volume 1 again, while hanging on to volume 2, later backup to volume 2 while holding onto volume 1, etc. It is also highly recommended that one of these backup sets be kept OFF SITE. For example, take volume 1 to a friend's on Mondays, Wednesdays, Fridays, and take volume 2 to your friend's on Tuesdays, Thursdays and Saturdays. That way, if your building burns down, you always have spare backup!



One big fat problem with Mac OS X is the inability to make a complete bootable backup by simply duplicating a volume to another drive. Why this does not work is related to the UNIX file system. Some files refuse to be copied if you drop them on another volume. They just sit there, leaving the resulting backup highly non-functional for complete restoration of a ruined volume. Even powerful backup programs like Retrospect 4 were stymied by some odd files it could see, would attempt to copy, but sadly failed to budge. Thus the need for a whole new set of backup programs just for Mac OS X. Retrospect 5 has recently been released and is Mac OS X native. The first couple versions proved to be buggy, so be sure you get the latest update. There is also a slew of shareware/freeware programs to consider as well, all of them costing much less than Retrospect, and all of them a lot less powerful. But they may be all you really need or want.



One very inexpensive and highly effective program is called Carbon Copy Cloner. At the time of this article, it does not allow backups over a network, to CDs or DVDs, and it is completely UFS volume illiterate. Oh well. But if you are crafty you can have it make a complete and functional copy of a volume into a disk image. You will find instructions for using this method with the software. You can move the disk image to other media to free up space on your hard drive. Be sure you pay attention to the provided instructions for restoring disk images. Disk images must be smaller than 4 GB in size, a size which is probably useful only to home users.



Priority 2: Virus Protection

Yes, virus checking is disk maintenance. It used to be that Mac users only had a few handfuls of viruses to worry about, and all of them could be squashed with freeware. Mac veterans will remember good old Disinfectant, free and functional. Those days are long gone. You MUST buy a virus protection program these days. There are a number of choices these days. My personal choice is the Virex by Network Associates.



Now that you have a virus checking program, you need to keep it up to date! You would be amazed how few people pay attention to virus updates, to their detriment. Virex and Norton Anti-Virus provide monthly update downloads you can apply and use for testing your volumes. Install them, then immediately check your volumes for new viruses.



How often should you perform a virus scan? Again it depends on what you do with your volumes. If you are constantly on the Internet it is a good idea to perform regular virus checks, at the very least once a month with the arrival of each new virus update. If you run Microsoft software on your Mac it is critical that you perform more frequent virus checks. Why? The actual number of Macintosh specific viruses hovers around 55 these days, and many of them are either extinct, or could not run even in Classic. However, every month there are loads (as in possibly 100s) of Microsoft specific viruses let loose on the world. Some of them are cross-platform for both Mac and PC! It is not unusual for professional Mac users to have a virus protection program running all day long, checking every single new file, no matter what the source. I have witnessed businesses come to a crushing halt because their computers failed to detect a virus infection. The most vicious of viruses can erase your entire hard drive! Computing can be cruel! Which leads us back to the importance of having a backup.

As a side note, you may find your virus checking program is a convenient way to identify bad files on your volumes. If your virus checker says it is unable to scan a particular file, chances are the file is corrupted. Of course verify that the problem is not simply an incompatibility between your virus checker and a particular file compression format. Older virus checkers won't have a clue what a .gz file is on a Mac OS X volume. You don't want to toss a file just because your virus program is clueless.

Priority 3: Volume Structure Maintenance.



Macintosh provides four convenient ways to check and repair your volume structures. The first is good old Disk First Aid, which you have to run from within Mac OS 9. The current version is 8.6.1. The equivalent program on Mac OS X is called Disk Utility. Sadly, Disk Utility is a wimp in comparison to Disk First Aid. (And you thought Disk First Aid was wimpy!). It cannot even verify its own MOSX volume, it refuses to check other volumes if they are being used for system purposes such as booted Classic volumes, and it as this pesky problem checking volumes with incompatible privileges. As you might guess, I consider Disk Utility to be next to useless. (Can you hear me Apple?) The third provided method is called 'fsck' or File System Check. Getting to it is very annoying to the average Mac user because you have to use a CLUI (Character Line User Interface) to get it to run. The fourth way to repair your volume is the automatic 'Checking Disk' routine that runs every time your MOSX system boots. NONE of these repair methods is entirely effective, and none of them do exactly the same thing, despite the myths you have heard. Let's throw a few of those myths into the trash can, shall we?

'Myth Descending Into A Trash Can'

Myth I: 'Every time I reboot MOSX the 'Checking Disk' routine repairs all the problems on my MOSX volume.' Wrong. A variant of this myth is: 'Every time I reboot MOSX the 'Checking Disk' routine runs fsck and repairs all the problems on my MOSX volume. Again, wrong. I have had an Apple marketing rep disagree with me, but I can't find any evidence that fsck is ever run during the 'Checking Disk' routine. One way you can check for yourself is to boot your MOSX system in what is called 'Verbose Mode.' You do this by holding down Command-V through the beginning of a system boot. What you get is a text list of all the processes the system goes through as it boots. Watch when the 'Checking Disk' routine runs. Have you ever seen fsck run during that routine? I haven't! Neither have I ever seen the 'Checking Disk' routine take as long as it takes to run fsck.

Result: I would like to know exactly what 'Checking Disk' actually does. (Some one out there in the readership must know!) There is evidence that it does perform some sort of verification and repair of volume structures when rebooting after a system crash. However, I have consistently found that even repeated reboots of MOSX, meaning several runs of 'Checking Disk,' fails to repair corrupted volume structures as effectively as fsck. Don't count on this boot routine doing much beyond a very superficial check of your boot volume.

Myth II: 'Whenever I run Disk First Aid it repairs all the problems on my MOSX volume.' Wrong. First off, Disk First Aid is what it's name infers. It is only 'First Aid.' Any commercial disk utility on the market, no matter how dreadful it may be, does a lot more than Disk First Aid. Also, if Disk First Aid does find a problem, and manages to repair it, you must run it again. Keep running Disk First Aid over and over until it comes up clean. The reason why is that the program is really quite lame. If it finds a problem, it focuses on fixing that problem, meaning that it ignores other problems that may be present. Run it again and it will find and repair the next problem. Run it again and it will find and repair the next problem, etc. Disk First Aid is at its worst when it goes into a loop that fixes one problem, finds another, repairs the other problem, but the first problem pops up again, so it repairs it, but the second problem pops up again, ad nauseam. At that point I suggest you give up and try something better. Even if Disk First Aid does come up clean, don't count on it having discovered or repaired every problem on a volume.

Myth III: 'Whenever I run fsck in Single User Mode it repairs all the problems on my MOSX volume.' Wrong again. Fsck is just as lame as Disk First Aid and still requires being run over and over until it comes up clean. Even then, count on fsck having missed something!

You may be surprised to learn this, but Disk First Aid and fsck are NOT the same program. They never were. They may appear similar, but that is all. They will find different problems with your volume. Running one does not preclude the necessity of running the other. Believe me! If you are going to run one, you might as well run the other.

fsck: I have had some NeXT users warn me about this disk repair utility. If your volume is pretty far gone, you may find fsck wreaking havoc on your file system. This can include having files completely disappear! I have had my Dock preference file disappear a number of times after an fsck 'repair.' (Good thing I made a backup!) I also believe fsck is a prime suspect in the infamous disappearance of Stickies note files. Another thing fsck can do is grab a bunch of application files and tell you they are bad, isolate them, leaving critical parts of your MOSX setup non-functional, leading to loads of fun restoring functionality again. Tip: If you are going to use fsck, use it frequently. Don't wait to let it loose on a badly corrupted volume. Or better yet, leave volume structure repair to a big gun disk repair utility.



The Big Guns: The big, expensive disk utility programs may well save your disk's life. Or, on some occasions, they may ruin your volume structures forever! (Uh Oh!) These utilities include TechTool Pro, Norton Disk Doctor, Drive 10, DiskWarrior, etc.

Let me begin by warning you that you must not use an old MOS version of a disk utility on Mac OS X volumes, or you will be sorry. The result is typically an unbootable volume! This is because there are a slew of oddball changes to files and file naming conventions in MOSX of which old MOS disk utilities have not-a-clue. Examples: In MOS it is illegal to have files start with periods. Not in MOSX! In MOS it is illegal for files to include null characters. Not in MOSX! MOS files typically have resource forks. Not MOSX files! When your ye olde disk utility program runs into these oddball files it will think they are broken and will try to repair them. It is actually damaging them! This is why it is absolutely critical that you know your disk utility is certified to run with the particular version of MOSX you are running!!! I cannot emphasize this enough! You can find this information on the web site of any of the disk utility manufacturers. If their site says nothing about MOSX compatibility, then count on it not being compatible.



At this point in time my personal recommendation is that everyone obtain a copy of Disk Warrior 2.1 or higher. Like me, you may think the price is a bit high. But when you see the results, you will calm down again. Also note that if you order the CD version of DiskWarrior from the web you will receive a free copy of PlusOptimizer, a wonderful utility I will describe ahead. If you have a large volume for your MOSX installation, (I recommend 4 GB), then you will find DiskWarrior can take a long time doing its work. So run it during lunch, at the end of the work day, or while you have nothing better to do but snooze awhile. Disk Warrior will offer to let you look at the new volume it has created. Take advantage of this offer and take a look. Very rarely DiskWarrior will create a monster disk that makes no sense at all. It is rather obvious that something is wrong when every single one of your files is dumped at the root level of your volume. If this happens, reject the new volume structures, and start DiskWarrior over again from scratch. Chances are that the next pass will get it right. In other words, never simply agree to let DiskWarrior overwrite your old volume structures. Take the time to verify it has done a good job.



Another nice thing DiskWarrior will do is identify bad files on your volume, then set them aside into a folder at the root of the volume. Look these files over after you overwrite the old volume structures. If they are only 'temp' files you can trash them. If the files look important, you need to replace them.



I suggest running those other utilities, TechTool Pro, Norton Disk Doctor, etc., after you have run DiskWarrior. Yes, they may claim to do what DiskWarrior does, but I have yet to find any of them actually do. Much as I love the folks at Micromat, TechTool's ability to recreate volume structures remains lame, and Drive 10 is still not mature. Norton Disk Doctor has in the past been notorious for inventing volume structure problems, screwing up volumes to the point where they are worse than when they started. So beware!



Instead use these other utilities to check the integrity of your volume media, and check files. Checking your media should be done rarely. Once a month is more than enough. This test finds where data has dropped off your disk, indicating bad blocks. A good disk utility program will mark off these bad blocks in order to prevent the system from trying to use them again. With regards to file checking, as long as the utility is MOSX savvy, it will report and repair improperly set bundle bits and dates. These programs are also typically good at identifying corrupted files.

Priority 4: Rebuild The Desktop

Mac OS X itself keeps track of files using a different system from the MOS desktop database files. There is no way I am aware of to rebuild the desktop on a MOSX partition during or after booting. However, you will find that if you boot to MOS9 you can rebuild desktop database files on a MOSX volume using the usual Option-Command keys. MOSX does use the desktop database files of other volumes to find files. So, it is good practice to rebuild all your volume desktop database files on a regular basis. While you are booted in MOSX you can tell it is time to rebuild the desktop files on other volumes when their application icons turn generic, looking like white pages with a big letter A on them made out of little tools, just as you see on your MOSX Applications folder.

You can rebuild the desktop of MOS volumes from inside the Classic System Preferences interface. Note that a volume has to have a blessed MOS 9.1+ System Folder in order to be selected as a Classic startup volume. Begin inside the Start/Stop tab and select the volume you want to rebuild as your startup volume.



Next, hit the Advanced tab and go to the bottom of its window. Push the "Rebuild Desktop" button. The volume you selected will now rebuild. You can then go back to the Start/Stop tab, pick another volume, return to the Advanced tab and rebuild that volume's desktop, etc.



If you do not have MOS 9.1+ installed on a volume you wish to rebuild, you will have to boot to a MOS9 system to rebuild its desktop.

Note that the effect of your rebuilds may not appear immediately while you are booted in MOSX. Try logging out of your account then logging in again to see the effect. Occasionally I have had to restart MOSX to see the results.

As ever, it can be extremely useful to entirely wipe out your desktop databases and rebuild them from scratch. Over time a desktop database file can become corrupt. Rebuilding a corrupted desktop database file may mean keeping the corruption while only the uncorrupted part of the file rebuilds. The simplest way to wipe out the desktop is to boot to MOS 9 and use TechTool Lite. Start up the application, double-click the section of the interface for rebuilding the desktop and tell it what volume's files to wipe out. TechTool Lite will then remove the desktop database for that volume, restart your Mac, then rebuild the desktop database from scratch. I have found it best to wipe out one volume's desktop database at a time. I have seen TechTool Lite choke if I choose to wipe more than one desktop database at a time. Remember to rebuild the desktop database on your MOSX volume as well as your MOS9 volumes!



Priority 5: Disk Optimization

There is an excellent discussion of this subject at Macintouch. <http://www.macintouch.com/> You may even find a few comments from me listed there. The fact is that Mac OS X installations leave your disk rather significantly un-optimized. Yes, there are UNIX routines used during installations on MOSX that 'optimize' files for access by the system. But this does not mean disk optimization. An optimized volume has all its files tightly written together, in an ideal order so as to minimize the time the system needs to access them. When system files are scattered all over your volume your system will slow down while it thrashes around your hard drive trying to access everything it needs. Also, if you have files that are broken into pieces, and these pieces are scattered all over the volume, the system will be further slowed down trying to piece them back together into memory. Add to that the fact that MOSX virtual memory will run faster if all the data it has written to disk is in one neat uncluttered location.



There have been many reports that the version of Speed Disk with Norton Utilities 6.x is not particularly MOSX savvy. Sadly, Symantec shipped exactly the same old version of Speed Disk with Norton Utilities 7.0 as well. (Watch for an updated version). I have not heard of Speed Disk 6 actually corrupting a MOSX volume, but I can verify that it will most likely cause system access to your files to slow down, which is the exact opposite of what you want. Apparently this is due to Speed Disk 6 optimizing your volume according to what is the best file arrangement for MOS9, not MOSX. The only optimizer I have personally found worth using with MOSX is PlusOptimizer 1.3 or above from Alsoft, the same folks who brought you DiskWarrior. Again note, PlusOptimizer if free if you order the DiskWarrior CD from the Alsoft Website. <http://www.alsoft.com/>. PlusOptimizer is swift and sure. I have run it with both MOSX 10.1.5 and 10.1.5 Server, and obtained excellent results in every case. System speed is at least as fast as it was before the optimization. Ideally it is faster.

______

There is more to come! Watch for Mac OS X Disk Maintenance, Part II.

The second half of this article we will get into some more obscure aspects of MOSX disk maintenance like prebinding and garbage collection, as well as frequently asked questions. If you have questions about MOSX disk maintenance, please post them and I do my best to answer them.

*Special thanks to Jesper Oersted for assisting with this article.*

Share and enjoy!