Cloning LDOM using ZFS

Walkthrough of the steps used to configure zfs on a server, setup your GOLDEN domain and clone from it using zfs. Some useful links too:
http://blogs.sun.com/vreality/entry/ldom_with_zfs http://blogs.sun.com/timthomas/entry/a_simple_vnc_server_and

Step 1 : Setup zfs pool
check disks you want to add to pool

bash-3.00# format
Searching for disks...done
AVAILABLE DISK SELECTIONS:
0. c1t0d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424>
/pci@7c0/pci@0/pci@1/pci@0,2/LSILogic,sas@2/sd@0,0
1. c1t1d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424>
/pci@7c0/pci@0/pci@1/pci@0,2/LSILogic,sas@2/sd@1,0
2. c1t2d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424>
/pci@7c0/pci@0/pci@1/pci@0,2/LSILogic,sas@2/sd@2,0
3. c1t3d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424>
/pci@7c0/pci@0/pci@1/pci@0,2/LSILogic,sas@2/sd@3,0

add to pool:
zpool -f create vpool c1t1d0 c1t2d0 c1t3d0
doesnt work because /export/home is mounted.

1. umount /export/home

remove entry from /etc/vfstab
zpool -f create vpool c1t1d0 c1t2d0 c1t3d0

bash-3.00# zfs list
NAME USED AVAIL REFER MOUNTPOINT
vpool 110K 201G 18K /vpool

Step 2 : Create 20 gig area in pool for Golden LDOM
zfs create vpool/goldenimage
zfs create -V 20g vpool/goldenimage/b111a

Step 3 : Create LDOM and point to the above area.
bash-3.00# /usr/sbin/svcadm enable vntsd

bash-3.00# /usr/bin/svcs vntsd
STATE STIME FMRI
online 11:29:18 svc:/ldoms/vntsd:default
bash-3.00# /opt/SUNWldm/bin/ldm create b111a
bash-3.00# /opt/SUNWldm/bin/ldm set-mau 1 b111a
bash-3.00# /opt/SUNWldm/bin/ldm set-vcpu 4 b111a
bash-3.00# /opt/SUNWldm/bin/ldm set-mem 1024M b111a
bash-3.00# /opt/SUNWldm/bin/ldm add-vnet vnet1 primary-vsw0 b111a
bash-3.00# ls /dev/zvol/dsk/vpool/goldenimage
b111a
/opt/SUNWldm/bin/ldm add-vdsdev /dev/zvol/dsk/vpool/goldenimage/b111a b111a@primary-vds0
/opt/SUNWldm/bin/ldm add-vdisk vdisk1 b111a@primary-vds0 b111a
bash-3.00# /opt/SUNWldm/bin/ldm set-variable auto-boot?=false b111a
bash-3.00# /opt/SUNWldm/bin/ldm set-variable boot-device=/virtual-devices@100/channel-devices@200/disk@0 b111a
bash-3.00# /opt/SUNWldm/bin/ldm bind b111a
bash-3.00# /opt/SUNWldm/bin/ldm start b111a
LDom b111a started
bash-3.00# /opt/SUNWldm/bin/ldm list
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv SP 4 1G 6.2% 23h 10m
b111a active t-- 5000 4 1G 25% 9s
opensolaris-b111a inactive ----- 4 1G
opensolaris-b111a_local inactive ----- 4 1G
Step 4 : Install LDOM
use boot net:dhcp
(There will be another post on this wiki for setting up the opensolaris install server)
Step 5 :
Configure root user:
su root (passwd is opensolaris by default)
TERM=dtterm
export TERM
vi /etc/user_attr
remove "type=role;" from the root line.
reboot
log in again as root.

1. sys-unconfig

Sample output:
opensolaris console login: root
Password:
May 12 04:42:37 opensolaris login: ROOT LOGIN /dev/console
Sun Microsystems Inc. SunOS 5.11 snv_111a November 2008
root@opensolaris:~# sys-unconfig
WARNING
This program will unconfigure your system. It will cause it
to revert to a "blank" system - it will not have a name or know
about other systems or networks.
This program will also halt the system.
Do you want to continue (y/n) ? y
sys-unconfig started Tue May 12 04:42:47 2009
rm: cannot remove `//etc/vfstab.sys-u': No such file or directory
sys-unconfig completed Tue May 12 04:42:48 2009
Halting system...
svc.startd: The system is coming down. Please wait.
svc.startd: 83 system services are now being stopped.
May 12 04:42:57 opensolaris syslogd: going down on signal 15
svc.startd: Killing user processes.
May 12 04:43:07 The system is down. Shutdown took 16 seconds.
syncing file systems... done
Program terminated
Sun Fire T200, No Keyboard
Copyright 2008 Sun Microsystems, Inc. All rights reserved.
OpenBoot 4.28.9, 1024 MB memory available, Serial #66639941.
Ethernet address 0:14:4f:f8:d8:45, Host ID: 83f8d845.

Unknown macro: {0}

ok
Step 6:
Stop and unbind the LDOM
bash-3.00# /opt/SUNWldm/bin/ldm stop b111a
bash-3.00# /opt/SUNWldm/bin/ldm unbind b111a
Step 7:
Take a snapshot of the GOLDEN LDOM disk image:
bash-3.00# zfs snapshot vpool/goldenimage/b111a@sysunconfig
bash-3.00# zfs list
NAME USED AVAIL REFER MOUNTPOINT
vpool 25.3G 175G 1.66M /vpool
vpool/goldenimage 25.3G 175G 18K /vpool/goldenimage
vpool/goldenimage/b111a 25.3G 195G 5.32G -
vpool/goldenimage/b111a@sysunconfig 0 - 5.32G -
Create a new area for the new domain:
bash-3.00# zfs create vpool/domain1
bash-3.00# zfs list
NAME USED AVAIL REFER MOUNTPOINT
vpool 25.3G 175G 1.66M /vpool
vpool/domain1 18K 175G 18K /vpool/domain1
vpool/goldenimage 25.3G 175G 18K /vpool/goldenimage
vpool/goldenimage/b111a 25.3G 195G 5.32G -
vpool/goldenimage/b111a@sysunconfig 0 - 5.32G -
Clone the GOLDEN LDOM disk to the new domain:
zfs clone vpool/goldenimage/b111a@sysunconfig vpool/domain1/disk_image
bash-3.00# zfs list
NAME USED AVAIL REFER MOUNTPOINT
vpool 25.3G 175G 1.66M /vpool
vpool/domain1 18K 175G 18K /vpool/domain1
vpool/domain1/disk_image 0 175G 5.32G -
vpool/goldenimage 25.3G 175G 18K /vpool/goldenimage
vpool/goldenimage/b111a 25.3G 195G 5.32G -
vpool/goldenimage/b111a@sysunconfig 0 - 5.32G -
Step 8:
Create a new guest LDOM and point to this newly cloned disk
bash-3.00# /opt/SUNWldm/bin/ldm create domain1
bash-3.00# /opt/SUNWldm/bin/ldm set-mau 1 domain1
bash-3.00# /opt/SUNWldm/bin/ldm set-vcpu 4 domain1
bash-3.00# /opt/SUNWldm/bin/ldm set-mem 1024M domain1
bash-3.00# /opt/SUNWldm/bin/ldm add-vnet vnet1 primary-vsw0 domain1
bash-3.00# ls /dev/zvol/dsk/vpool/domain1
disk_image
bash-3.00# /opt/SUNWldm/bin/ldm add-vdsdev /dev/zvol/dsk/vpool/domain1/disk_image vol2@primary-vds0
bash-3.00# /opt/SUNWldm/bin/ldm add-vdisk vdisk1 vol2@primary-vds0 domain1
bash-3.00# /opt/SUNWldm/bin/ldm set-variable auto-boot?=false domain1
bash-3.00# /opt/SUNWldm/bin/ldm bind domain1
bash-3.00# /opt/SUNWldm/bin/ldm start domain1
LDom domain1 started
Step 9 : Log into the newly cloned LDOM
telnet localhost <port number>
from ok prompt : boot
Answer the sys-unconfig questions;
terminal type: dtterm
select dhcp,
domain:
nameserver:
ip of nameserver:
Sample output after answering questions:
System identification is completed.
rebooting system due to change(s) in /etc/default/init
syncing file systems... done
rebooting...
Resetting...
Sun Fire T200, No Keyboard
Copyright 2008 Sun Microsystems, Inc. All rights reserved.
OpenBoot 4.28.9, 1024 MB memory available, Serial #66694113.
Ethernet address 0:14:4f:f9:ab:e1, Host ID: 83f9abe1.
Boot device: disk File and args:
SunOS Release 5.11 Version snv_111a 64-bit
Copyright 1983-2009 Sun Microsystems, Inc. All rights reserved.
Use is subject to license terms.
Hostname:
NIS domain name is
Reading ZFS config: done.
Mounting ZFS filesystems: (6/6)
console login: root
Password:
Last login: Tue May 12 12:42:37 on console
Sun Microsystems Inc. SunOS 5.11 snv_111a November 2008
root@jdub65:~#
Step 10 : Check zfs on domain
root@jdub65:~# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
rpool 19.9G 4.70G 15.2G 23% ONLINE -
Thats it. You have created a GOLDEN LDOM that you can clone from multiple times.