flyinweb's blog

Following are few situations where you may be interested in performing a filesystem benchmarking.

=> Deploying a new application that is very read and write intensive.
=> Purchased a new storage system and would like to measure the performance.
=> Changing the RAID level and would like to measure the performance of the new RAID.
=> Changing the storage parameters and would like to know the performance impact of this change

This article gives you a jumpstart on performing benchmark on filesystem using iozone a free Filesystem Benchmark utility.

1. Download and Install iozone software

Go to iozone and download the iozone for your appropriate platform. I downloaded the "Linux i386 RPM". Install the iozone from the RPM as shown below. By default this installs the iozone under /opt/iozone/bin
# cd /tmp
# wget http://www.iozone.org/src/current/iozone-3-303.i386.rpm
# rpm -ivh iozone-3-303.i386.rpm
Sample output:

Preparing...                ########################################### [100%]
   1:iozone                 ########################################### [100%]

Note: You can install iozone under any UNIX / Linux or Windows operating system.

2. Start the performance test

Execute the following command in the background to begin the performance test.

# /opt/iozone/bin/iozone -R -l 5 -u 5 -r 4k -s 100m -F /home/f1 /home/f2 /home/f3 /home/f4 /home/f5 | tee -a /tmp/iozone_results.txt &

Let us review all the individual parameter passed to iozone command.

• -R : Instructs the iozone to generate excel compatible text output.
• -l : This is the lower limit on how many process/threads needs to be started by iozone during execution. In this example, iozone will start 5 threads.
• -u : This is the upper limit on how many process/threads needs to be started by iozone during execution. In this example, iozone will not exceed maximum of 5 threads. If you set -l and -u to the same value, it will run exactly those many number of process/threads. In this example, this will execute exactly 5 threads.
• -r : This specifies the record size. In this example, the record size for benchmark testing is 4k. This is an important parameter to be set appropriately depending on the purpose of your filesystem performance testing. For e.g. If you are performing benchmark on a filesystem that will host a database, it is appropriate to set this value to the DB block size of the database.
• -s : This specifies the size of the file that needs to be tested. In this example, iozone will try to perform test on 100Mb file.
• -F : Specify the temporary filename that should be used by the iozone during testing. The total number of files specified here should match the value specified in -l and -u parameter.

3. Analyze the output of iozone file.

The first part of the output will contain the details about every individual filesystem performance metrics that was tested. for e.g. Initial write, rewrite etc as shown below.

        Iozone: Performance Test of File I/O
                Version $Revision: 3.303 $
                Compiled for 32 bit mode.
                Build: linux
        Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
                     Al Slater, Scott Rhine, Mike Wisner, Ken Goss
                     Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
                     Randy Dunlap, Mark Montague, Dan Million,
                     Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy,
                     Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root.
        Run began: Thu Jun  22 00:08:51 2008
        Excel chart generation enabled
        Record Size 4 KB
        File size set to 102400 KB
        Command line used: /opt/iozone/bin/iozone -R -l 5 -u 5 -r  4k -s 100m -F /home/f1 /home/f2 /home/f3 /home/f4 /home/f5
        Output is in Kbytes/sec
        Time Resolution = 0.000001 seconds.
        Processor cache size set to 1024 Kbytes.
        Processor cache line size set to 32 bytes.
        File stride size set to 17 * record size.
        Min process = 5
        Max process = 5
        Throughput test with 5 processes
        Each process writes a 102400 Kbyte file in 4 Kbyte records
        Children see throughput for  2 initial writers  =   60172.28 KB/sec
        Parent sees throughput for  2 initial writers   =   45902.89 KB/sec
        Min throughput per process                        =   28564.52 KB/sec
        Max throughput per process                      =   31607.76 KB/sec
        Avg throughput per process                      =   30086.14 KB/sec
        Min xfer                                                  =   92540.00 KB
        Children see throughput for  2 rewriters        =   78658.92 KB/sec
        Parent sees throughput for  2 rewriters         =   34277.52 KB/sec
        Min throughput per process                      =   35743.92 KB/sec
        Max throughput per process                      =   42915.00 KB/sec
        Avg throughput per process                      =   39329.46 KB/sec
        Min xfer                                        =   85296.00 KB

Similar values like above will be generated for readers, re-readers, reverse readers, stride readers, random readers, mixed workload, random writers, pwrite writers, pread readers. The last part of the iozone output will contain the Throughput summary for different metrics as shown below.

Throughput report Y-axis is type of test X-axis is number of processes
Record size = 4 Kbytes
Output is in Kbytes/sec
Initial write       60172.28
Rewrite           78658.92
Read              2125613.88
Re-read          1754367.31
Reverse Read 1603521.50
Stride read      1633166.38
Random read   1583648.75
Mixed workload 1171437.78
Random write    5365.59
Pwrite               26847.44
Pread               2054149.00

(Fig.01: iozone in action)

Iozone does a benchmarking on different types of file system performance metrics. for e.g. Read, Write, Random read. Depending on the application that you are planning to deploy on that particular filesystem, pay attention to the appropriate items. for e.g. If the filesystem hosts an read intensive OLTP database, then pay attention to Random Read, Random write and Mixed workload. If the application does lot of streaming media content, pay attention to the Sequential Read. On a final note, you can generate graphs using the Generate_Graphs and gengnuplot.sh located under /opt/iozone/bin, based on the iozone output.

References:

• Iozone PDF documentation – Full documentation from iozone.org explaining all the iozone command line options and more.
• Linux Iozone graph example - This is a sample *.xls file from iozone that shows the kind of excel output that can be generated from iozone.

This article was written by Ramesh Natarajan. At the The Geek Stuff blog he shares his knowledge and experience on Linux and other Geeky stuff. He has more than 15 years of experience in IT industry and has performed very intensive work on Linux system administration, DBA, Hardware and Storage. nixCraft welcomes readers' tips / howtos.

=============================================================================

If someone is complaining that a database (or any application) running on one server (with certain filesystem, or RAID configuration) is running faster than the same database or application running on another server, you might want to make sure that the performance at the disk level is same on both the server. You can use iozone for this situation.

If you are running your database (or any application) on certain SAN or NAS environment, and would like to migrate it to different SAN or NAS environment, you should perform filesystem benchmakring on both the systems and compare it. You can use iozone for this situation.

If you know how to use iozone, you can pretty much use it for various filesystem benchmarking purpose.

Download and Install IOZone

Iozone is an open source file system benchmarking utility.

Follow the steps below to download and install iozone on your system.

wget http://www.iozone.org/src/current/iozone3_394.tar
tar xvf iozone3_394.tar 
cd iozone3_394/src/current
make
make linux

What does IOzone utility measure?

IOzone performs the following 13 types of test. If you are executing iozone test on a database server, you can focus on the 1st 6 tests, as they directly impact the database performance.

1. Read – Indicates the performance of reading a file that already exists in the filesystem.
2. Write – Indicates the performance of writing a new file to the filesystem.
3. Re-read – After reading a file, this indicates the performance of reading a file again.
4. Re-write – Indicates the performance of writing to an existing file.
5. Random Read – Indicates the performance of reading a file by reading random information from the file. i.e this is not a sequential read.
6. Random Write – Indicates the performance of writing to a file in various random locations. i.e this is not a sequential write.
7. Backward Read
8. Record Re-Write
9. Stride Read
10. Fread
11. Fwrite
12. Freread
13. Frewrite

10 IOZone Examples

1. Run all IOZone tests using default values

-a option stands for automatic mode. This creates temporary test files from sizes 64k to 512MB for performance testing. This mode also uses 4k to 16M of record sizes for read and write (more on this later) testing.

-a option will also execute all the 13 types of tests.

$ ./iozone -a

The first setion of the iozone output contains the header information, which displays information about the iozone utility, and all the iozone options that are used to generate this report, as shown below.

Iozone: Performance Test of File I/O
        Version $Revision: 3.394 $
	Compiled for 32 bit mode.
	Build: linux 
Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
             Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Run began: Sat Apr 23 12:25:34 2011
Auto Mode
Command line used: ./iozone -a
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.

The second section of the output contains the output values (in per second) of various tests.

• 1st column KB: Indicates the file size that was used for the testing.
• 2nd column reclen: Indicates the record length that was used for the testing.
• 3rd column until the last column: Indicates the various tests that are performed and its output values in per second.

                                                random  random    bkwd   record   stride
  KB  reclen   write rewrite    read    reread    read   write    read  rewrite     read   fwrite frewrite   fread  freread
  64       4  495678  152376  1824993  2065601 2204215  875739  582008   971435   667351   383106   363588  566583   889465
  64       8  507650  528611  1051124  1563289 2071399 1084570 1332702  1143842  2138827  1066172  1141145 1303442  2004783
  64      16  587283 1526887  2560897  2778775 2366545 1122734 1254016   593214  1776132   463919  1783085 3214531  3057782
  64      32  552203  402223  1121909  1388380 1162129  415722  666360  1163351  1637488  1876728  1685359  673798  2466145
  64      64  551580 1122912  2895401  4911206 2782966 1734491 1825933  1206983  2901728  1207235  1781889 2133506  2780559
 128       4  587259 1525366  1801559  3366950 1600898 1391307 1348096   547193   666360   458907  1486461 1831301  1998737
 128       8  292218 1175381  1966197  3451829 2165599 1601619 1232122  1291619  3273329  1827104  1162858 1663987  1937151
 128      16  650008  510099  4120180  4003449 2508627 1727493 1560181  1307583  2203579  1229980   603804 1911004  2669183
 128      32  703200 1802599  2842966  2974289 2777020 1331977 3279734  1347551  1152291   684197   722704  907518  2466350
 128      64  848280 1294308  2288112  1377038 1345725  659686 1997031  1439349  2903100  1267322  1968355 2560063  1506623
 128     128  902120  551579  1305206  4727881 3046261 1405509 1802090  1085124  3649539  2066688  1423514 2609286  3039423
...

2. Save the output to a spreadsheet using iozone -b

To save the iozone output to a spreadsheet, use the -b option as shown below. -b stands for binary, and it instructs iozone to write the test output in binary format to a spreadsheet.

$ ./iozone -a -b output.xls

Note: The -b option can be used with any of the examples mentioned below.

From the data that is saved in the spreadsheet, you can use the create some pretty graphs using the graph functionality of the spreadsheet tool. The following is a sample graph that was created from iozone output.

Fig: IOZone graph

3. Run only a specific type of test using iozone -i

If you are interested in running only a specific type of test, use the -i option.

Syntax:

iozone -i [test-type]

The test-type is a numeric value. The following are the various available test types and its numeric value.

• 0=write/rewrite
• 1=read/re-read
• 2=random-read/write
• 3=Read-backwards
• 4=Re-write-record
• 5=stride-read
• 6=fwrite/re-fwrite
• 7=fread/Re-fread,
• 8=random mix
• 9=pwrite/Re-pwrite
• 10=pread/Re-pread
• 11=pwritev/Re-pwritev
• 12=preadv/Re-preadv

The following example will run only the write tests (i.e both write and rewrite). As you see from the output the other columns are empty.

$ ./iozone -a -i 0
                                                random  random    bkwd   record   stride
  KB  reclen   write rewrite    read    reread    read   write    read  rewrite     read   fwrite frewrite   fread  freread
  64       4  353666  680969
  64       8  477269  744768
  64      16  429574  326442
  64      32  557029  942148
  64      64  680844  633214
 128       4  187138  524591

Combine multiple iozone test types

You can also combine multiple test types by specifying multiple -i in the command line.

For example, the following example will test both read and write test types.

$ ./iozone -a -i 0 -i 1
                                                random  random    bkwd   record   stride
  KB  reclen   write rewrite    read    reread    read   write    read  rewrite     read   fwrite frewrite   fread  freread
  64       4  372112  407456  1520085   889086
  64       8  385574  743960  3364024  2553333
  64      16  496011  397459  3748273  1330586
  64      32  499600  876631  2459558  4270078

4. Specify the file size using iozone -s

By default, iozone will automatically create temporary files of size from 64k to 512M, to perform various testing.

The 1st column in the iozone output (with the column header KB) indicates the file size. As you saw from the previous output, it starts with 64KB file, and will keep increasing until 512M (by doubling the file size every time).

Instead of running the test for all the file sizes, you can specific the file size using option -s.

The following example will perform write test only for file size 1MB (i.e 1024KB).

$ ./iozone -a -i 0 -s 1024
                                                random  random    bkwd   record   stride
  KB  reclen   write rewrite    read    reread    read   write    read  rewrite     read   fwrite frewrite   fread  freread
1024       4  469710  785882
1024       8  593621 1055581
1024      16  745286 1110539
1024      32  610585 1030184
1024      64  929225 1590130
1024     128 1009859 1672930
1024     256 1042711 2039603
1024     512  941942 1931895
1024    1024 1039504  706167

5. Specify the record size for testing using iozone -r

When you run a test, for a specific file size, it tests with different record sizes ranging from 4k to 16M.

If you like to do I/O performance testing of an I/O subsystem that hosts oracle database, you might want to set the record size in the iozone to the same value of the DB block size. The database reads and writes based on the DB block size.

reclen stands for Record Length. In the previous example, the 2nd column (with the column header “reclen”) indicates the record length that should be used for testing IOzone. In the previous example outout, for the file size of 1024KB, the iozone testing used various record sizes ranging from 4k to 16M to perform the write test.

Instead of using all these default record length sizes, you can also specify the record size you would like to test.

The example below will run write test only for record length of 32k. In the output, the 2nd column will now only display 32.

$ ./iozone -a -i 0 -r 32
                                                random  random    bkwd   record   stride
  KB  reclen   write rewrite    read    reread    read   write    read  rewrite     read   fwrite frewrite   fread  freread
  64      32  566551  820553
 128      32  574098 1000000
 256      32  826044  948043
 512      32  801282 1560624
1024      32  859116  528901
2048      32  881206 1423096

6. Combine file size with record size

You can also using both -s and -r option to specific a exact temporary file size, and exact record length that needs to be tested.

For example, the following will run the write test using a 2M file with a record length of 1M

$ ./iozone -a -i 0 -s 2048 -r 1024
                                                random  random    bkwd   record   stride
  KB  reclen   write rewrite    read    reread    read   write    read  rewrite     read   fwrite frewrite   fread  freread
2048    1024 1065570 1871841

7. Throughput test using iozone -t

To execute the iozone in throughput mode, use -t option. You should also specify the number of threads that needs to be active during this test.

The following example will execute the iozone throughput test for writes using 2 threads. Please note that you cannot combine -a option with -t option.

$ ./iozone -i 0 -t 2
Children see throughput for  2 initial writers 1=  433194.53 KB/sec
Parent sees throughput for  2 initial writers 	=    7372.12 KB/sec
Min throughput per process 			=       0.00 KB/sec
Max throughput per process 			=  433194.53 KB/sec
Avg throughput per process 			=  216597.27 KB/sec
Min xfer 					=       0.00 KB
Children see throughput for  2 rewriters 	=  459924.70 KB/sec
Parent sees throughput for  2 rewriters 	=   13049.40 KB/sec
Min throughput per process 			=  225610.86 KB/sec
Max throughput per process 			=  234313.84 KB/sec
Avg throughput per process 			=  229962.35 KB/sec
Min xfer 					=     488.00 KB

To perform throughput for all the test types, remove the “-i 0″ from the above example, as shown below.

$ ./iozone -t 2

8. Include CPU Utilization using iozone -+u

While performing the iozone testing, you can also instruct iozone to collect the CPU utilization using -+u option.

The -+ in front of the option might look little strange. But, you have to give the whole -+u (not just -u, or +u) for this to work properly.

The following example will execute all the test, and include the CPU utilization report as part of the excel spreadsheet output it generates.

$ ./iozone -a -+u -b output.xls

Note: This will display separate CPU utilization for each and every test it performs.

9. Increase the file size using iozone -g

This is important. If your system has more than 512MB of RAM, you should increase the temporary file size that iozone uses for testing. If you don’t, you might not get accurate results, as the system buffer cache will play a role in it.

For accurate disk performance, it is recommended to have the temporary file size 3 times the size of your system buffer cache.

The following example will run the iozone by increasing the maximum file size to 2GB, and run the automatic iozone testing for write tests.

$ ./iozone -a -g 2G -i 0
                                                random  random    bkwd   record   stride
  KB  reclen   write rewrite    read    reread    read   write    read  rewrite     read   fwrite frewrite   fread  freread
  64       4  556674 1230677
  64       8  278340  441320
  64      16  608990 1454053
  64      32  504125 1085411
  64      64  571418 1279331
 128       4  526602  961764
 128       8  714730  518219
...

10. Test multiple mount points together using iozone -F

By combining several iozone options, you can perform disk I/O testing on multiple mount points as shown below.

If you have 2 mounts points, you can start 2 different iozone threads to create temporary files on both these mount points for testing as shown below.

$ ./iozone -l 2 -u 2 -r 16k -s 512M -F /u01/tmp1 /u02/tmp2

• -l indicates the minimum number of iozone processes that should be started
• -u indicates the maximum number of iozone processes that should be started
• -F should contain multiple values. i.e If we specify 2 in both -l and -u, we should have two filenames here. Please note that only the mount points need to exists. The file specified in the -F option doesn’t need to exists, as iozone will create this temporary file during the testing. In the above example, the mount points are /u01, and /u02. The file tmp1 and tmp2 will be automatically created by iozone for testing purpose.