How not to break your root mirror (and then how to fix it)

_The statements and opinions expressed here are my own_
_ and do not necessarily represent those of Oracle Corporation._ 
  _This post applies to: Solaris 8-10 systems using disksuite (solaris volume manager) to mirror root disks_

Let's say you're planning to patch your solaris system. If things went wrong, it would be nice to be able to roll-back the system to it's un-patched state. Here's one way to do it.
If you have mirrored system disks, we can break the mirror and patch one side. Another (better) way to do this is by using Live Upgrade.
To break the mirror, you will need to identify the system disks (typically / and /var ), for example d0 (a mirror of d10 and d20) and d3 (a mirror of d13 and d23 ). Let's also assume you have OBP device aliases set up for 'rootdisk' and 'rootmirror', pointing to each of your two (mirrored) system disks.
Let's say you quiesce the system, then break the mirrors using metadetach for each of your mirrored devices listed above, thus breaking the "right side" off, leaving the "left side" to be modified by our maintenance activity (e.g. patching).
We do the maintenance activity we were planning, and it goes horribly wrong -- let's say your system fails to reboot after patching. We then decide to boot from your mirror (d20) using "ok> boot rootmirror". **This is where things go wrong. **In fact, our system seems to have the same problem on the mirrored device which you haven't patched.
Here's what your system is doing:
ok> bootmirror

  • The OBP now looks up the alias for bootmirror, finds that it's a disk and (simplistically speaking) loads the solaris kernel from this disk
  • Disksuite gets initialised.
  • The kernel looks at the /etc/system file and finds a "rootdev" entry which tells it to mount the root filesystem from the "d0" disksuite mirror (which contains "d10")
  • The system looks at the /etc/vfstab file on the d10 device, and loads /var from d3.
  • The system is now running on the "left side"The trick here is that that the "right side" mirror contains references to the "left side" by way of the /etc/system file and the /etc/vfstab file on the "right side" disk's root filesystem.

To avoid this: make sure you edit /etc/system and /etc/vfstab on the "alternate" disk so that it only boots from itself, not the other mirror.

To fix this unfortunate situation, we need to modify the /etc/system file and /etc/vfstab file, however that leads us to our next problem: we can't boot. In order to resolve this, we have two choices:
1: Boot from the network, cdrom or some other bootable media, mount our root filesystem and make our corrections
2: Boot using "ok> boot -as" -- this causes the kernel to ask for various parameters, including the system file. See boot-flags in "man -s 1M kernel", specifically:


         Asks the user for  configuration  information,  such  as
         where  to find the system file, where to mount root, and
         even override the name of  the  kernel  itself.  Default
         responses  will  be  contained in square brackets ([ ]),
         and the user may simply enter RETURN to use the  default
         response (note that RETURN is labeled ENTER on some key-
         boards). To help  repair  a  damaged  /etc/system  file,
         enter /dev/null at the prompt that asks for the pathname
         of the system configuration file. See system(4).
So, just as the man page suggests, we can use "dev/null" as a temporary replacement for the /etc/system file, which will at least allow us to boot to single user mode (albeit with a read-only root filesystem).

If root is now mounted read-only, we can re-mount it read-write with the following command (replace c0t0d0s0 with the correct disk name):

mount -o remount,rw /dev/dsk/c0t0d0s0 /

... we can now effect whatever repairs we need to make. I'd very much recommend starting by modifying the /etc/system and /etc/vfstab files so that the system will boot from the underlying root disk device rather than a disksuite mirror (use /dev/dsk/c0t0d0s0 rather than /dev/md/dsk/d0)