Mirrored vdevs
For the full story on why to use mirrored vdevs, check this.
How to do so?
Let’s create 6 files to use as disks:
1❯ truncate -s 100M zfs_drive{1,2,3,4,5,6}.img
2
3❯ ls -l
4.rw-rw-r-- gsac gsac 100 MB Sat Jan 22 10:32:26 2022 zfs_drive1.img
5.rw-rw-r-- gsac gsac 100 MB Sat Jan 22 10:32:26 2022 zfs_drive2.img
6.rw-rw-r-- gsac gsac 100 MB Sat Jan 22 10:32:26 2022 zfs_drive3.img
7.rw-rw-r-- gsac gsac 100 MB Sat Jan 22 10:32:26 2022 zfs_drive4.img
8.rw-rw-r-- gsac gsac 100 MB Sat Jan 22 10:32:26 2022 zfs_drive5.img
9.rw-rw-r-- gsac gsac 100 MB Sat Jan 22 10:32:26 2022 zfs_drive6.img
Now, we create a zpool of 3x2 mirrored vdevs:
1❯ sudo zpool create mypool mirror /home/gsac/tmp/zfs/zfs_drive1.img /home/gsac/tmp/zfs/zfs_drive2.img mirror /home/gsac/tmp/zfs/zfs_drive3.img /home/gsac/tmp/zfs/zfs_drive4.img mirror /home/gsac/tmp/zfs/zfs_drive5.img /home/gsac/tmp/zfs/zfs_drive6.img
Let’s look how it looks like:
1❯ zpool status
2 pool: mypool
3 state: ONLINE
4config:
5
6 NAME STATE READ WRITE CKSUM
7 mypool ONLINE 0 0 0
8 mirror-0 ONLINE 0 0 0
9 /home/gsac/tmp/zfs/zfs_drive1.img ONLINE 0 0 0
10 /home/gsac/tmp/zfs/zfs_drive2.img ONLINE 0 0 0
11 mirror-1 ONLINE 0 0 0
12 /home/gsac/tmp/zfs/zfs_drive3.img ONLINE 0 0 0
13 /home/gsac/tmp/zfs/zfs_drive4.img ONLINE 0 0 0
14 mirror-2 ONLINE 0 0 0
15 /home/gsac/tmp/zfs/zfs_drive5.img ONLINE 0 0 0
16 /home/gsac/tmp/zfs/zfs_drive6.img ONLINE 0 0 0
17
18errors: No known data errors
19
20❯ zpool list
21NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
22mypool 240M 117K 240M - - 1% 0% 1.00x ONLINE -
How to expand?
To expand a current mirrored vdev, you need to take the three steps:
- add the drive and let the resilvering go
- detach the old drive
- expand the pool
Adding the new drive
Let’s create first the fake drives:
1❯ zpool status
2 pool: mypool
3 state: ONLINE
4config:
5
6 NAME STATE READ WRITE CKSUM
7 mypool ONLINE 0 0 0
8 mirror-0 ONLINE 0 0 0
9 /tmp/1.raw ONLINE 0 0 0
10 /tmp/2.raw ONLINE 0 0 0
11 mirror-1 ONLINE 0 0 0
12 /tmp/4.raw ONLINE 0 0 0
13 /tmp/3.raw ONLINE 0 0 0
14
15errors: No known data errors
16
17❯ zpool list
18NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
19mypool 1.88G 110K 1.87G - - 0% 0% 1.00x ONLINE -
20
21❯ truncate -s 2G /tmp/1{0,1}.raw
Now, we can add the drives to the pool and to match to a specific file:
1❯ sudo zpool attach mypool -f /tmp/1.raw /tmp/10.raw
2
3❯ zpool status
4 pool: mypool
5 state: ONLINE
6 scan: resilvered 90K in 00:00:00 with 0 errors on Mon Jan 24 17:45:57 2022
7config:
8
9 NAME STATE READ WRITE CKSUM
10 mypool ONLINE 0 0 0
11 mirror-0 ONLINE 0 0 0
12 /tmp/1.raw ONLINE 0 0 0
13 /tmp/2.raw ONLINE 0 0 0
14 /tmp/10.raw ONLINE 0 0 0
15 mirror-1 ONLINE 0 0 0
16 /tmp/4.raw ONLINE 0 0 0
17 /tmp/3.raw ONLINE 0 0 0
18
19errors: No known data errors
20
21❯ sudo zpool attach mypool -f /tmp/2.raw /tmp/11.raw
22
23❯ zpool status
24 pool: mypool
25 state: ONLINE
26 scan: resilvered 158K in 00:00:00 with 0 errors on Mon Jan 24 17:45:57 2022
27config:
28
29 NAME STATE READ WRITE CKSUM
30 mypool ONLINE 0 0 0
31 mirror-0 ONLINE 0 0 0
32 /tmp/1.raw ONLINE 0 0 0
33 /tmp/2.raw ONLINE 0 0 0
34 /tmp/10.raw ONLINE 0 0 0
35 /tmp/11.raw ONLINE 0 0 0
36 mirror-1 ONLINE 0 0 0
37 /tmp/4.raw ONLINE 0 0 0
38 /tmp/3.raw ONLINE 0 0 0
39
40errors: No known data errors
Detach the disk from the pool
Now that the resilvering has been done, we can detach the old drives:
1❯ sudo zpool detach mypool /tmp/1.raw
2
3❯ sudo zpool detach mypool /tmp/2.raw
4
5❯ sudo zpool status
6 pool: mypool
7 state: ONLINE
8 scan: resilvered 158K in 00:00:00 with 0 errors on Mon Jan 24 17:47:51 2022
9config:
10
11 NAME STATE READ WRITE CKSUM
12 mypool ONLINE 0 0 0
13 mirror-0 ONLINE 0 0 0
14 /tmp/10.raw ONLINE 0 0 0
15 /tmp/11.raw ONLINE 0 0 0
16 mirror-1 ONLINE 0 0 0
17 /tmp/4.raw ONLINE 0 0 0
18 /tmp/3.raw ONLINE 0 0 0
19
20errors: No known data errors
Now, let’s see the size of the pool:
1❯ sudo zpool list
2NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
3mypool 1.88G 266K 1.87G - - 0% 0% 1.00x ONLINE -
Wait what? No size increase???
Hahahaha, you fool! You need to expand the pool!
Expand the pool
1❯ sudo zpool online -e mypool /tmp/10.raw
2
3❯ sudo zpool online -e mypool /tmp/11.raw
4
5❯ zpool list
6NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
7mypool 2.88G 236K 2.87G - - 0% 0% 1.00x ONLINE -
Now, we can see the increase! Yayyy!