Linux-Manual

MySQL 测试

mysqlslap 工具

进行基准测试:

测试结果含义解释:

对自己的数据库进行测试:

其他一些参数:


sysbench 工具

安装

开始测试

测试报告

Running the test with following options: Number of threads: 15 Report intermediate results every 10 second(s) Initializing random number generator from current time

Initializing worker threads…

Threads started!

[ 10s ] thds: 15 tps: 337.43 qps: 6773.72 (r/w/o: 4745.03/1351.92/676.76) lat (ms,95%): 73.13 err/s: 0.40 reconn/s: 0.00 [ 20s ] thds: 15 tps: 340.12 qps: 6813.82 (r/w/o: 4772.12/1361.06/680.63) lat (ms,95%): 71.83 err/s: 0.40 reconn/s: 0.00 [ 30s ] thds: 15 tps: 344.78 qps: 6897.36 (r/w/o: 4828.36/1379.23/689.77) lat (ms,95%): 71.83 err/s: 0.20 reconn/s: 0.00 [ 40s ] thds: 15 tps: 343.32 qps: 6876.75 (r/w/o: 4815.15/1374.47/687.14) lat (ms,95%): 71.83 err/s: 0.60 reconn/s: 0.00 [ 50s ] thds: 15 tps: 342.80 qps: 6864.76 (r/w/o: 4806.67/1371.89/686.20) lat (ms,95%): 73.13 err/s: 0.50 reconn/s: 0.00 [ 60s ] thds: 15 tps: 347.90 qps: 6960.74 (r/w/o: 4873.93/1390.71/696.10) lat (ms,95%): 70.55 err/s: 0.30 reconn/s: 0.00 [ 70s ] thds: 15 tps: 346.70 qps: 6942.39 (r/w/o: 4859.29/1389.30/693.80) lat (ms,95%): 70.55 err/s: 0.40 reconn/s: 0.00 [ 80s ] thds: 15 tps: 345.60 qps: 6914.88 (r/w/o: 4841.48/1382.00/691.40) lat (ms,95%): 70.55 err/s: 0.20 reconn/s: 0.00 [ 90s ] thds: 15 tps: 341.10 qps: 6830.31 (r/w/o: 4782.20/1365.40/682.70) lat (ms,95%): 74.46 err/s: 0.50 reconn/s: 0.00 [ 100s ] thds: 15 tps: 341.20 qps: 6829.33 (r/w/o: 4782.12/1364.41/682.80) lat (ms,95%): 74.46 err/s: 0.40 reconn/s: 0.00 [ 110s ] thds: 15 tps: 343.40 qps: 6875.79 (r/w/o: 4812.29/1376.50/687.00) lat (ms,95%): 71.83 err/s: 0.20 reconn/s: 0.00 [ 120s ] thds: 15 tps: 347.00 qps: 6943.51 (r/w/o: 4862.40/1386.70/694.40) lat (ms,95%): 71.83 err/s: 0.40 reconn/s: 0.00 SQL statistics: queries performed: read: 577836 –读总数 write: 164978 –写总数 other: 82503 –其他操作(CURD之外的操作,例如COMMIT) total: 825317 –全部总数 transactions: 41229 (343.51 per sec.) –总事务数(每秒事务数,这个每秒事务数也就是:TPS 吞吐量) queries: 825317 (6876.33 per sec.) –查询总数(查询次数) ignored errors: 45 (0.37 per sec.) –总忽略错误总数(每秒忽略错误次数) reconnects: 0 (0.00 per sec.) –重连总数(每秒重连次数)

General statistics: total time: 120.0214s –总耗时 total number of events: 41229 –共发生多少事务数

Latency (ms): min: 7.19 –最小耗时 avg: 43.66 –平均耗时 max: 162.82 –最长耗时 95th percentile: 71.83 –超过95%平均耗时 sum: 1799860.45

Threads fairness: events (avg/stddev): 2748.6000/132.71 –总处理事件数/标准偏差 execution time (avg/stddev): 119.9907/0.00 –总执行时间/标准偏差


QPS 和 TPS 和说明

基本概念

QPS、TPS和性能的关系

结论


Percona TPCC-MySQL 测试工具(优先推荐)

TPC-C 是专门针对联机交易处理系统(OLTP系统)的规范,一般情况下我们也把这类系统称为业务处理系统。
TPC-C是TPC(Transaction Processing Performance Council)组织发布的一个测试规范,用于模拟测试复杂的在线事务处理系统。其测试结果包括每分钟事务数(tpmC),以及每事务的成本(Price/tpmC)。
在进行大压力下MySQL的一些行为时经常使用。

安装

git clone https://github.com/Percona-Lab/tpcc-mysql
cd tpcc-mysql/src
make

如果make没报错,就会在tpcc-mysql 根目录文件夹下生成tpcc二进制命令行工具tpcc_load、tpcc_start

如果要同时支持 PgSQL 可以考虑:https://github.com/Percona-Lab/sysbench-tpcc

测试的几个表介绍

tpcc-mysql的业务逻辑及其相关的几个表作用如下:
New-Order:新订单,主要对应 new_orders 表
Payment:支付,主要对应 orders、history 表
Order-Status:订单状态,主要对应 orders、order_line 表
Delivery:发货,主要对应 order_line 表
Stock-Level:库存,主要对应 stock 表

其他相关表:
客户:主要对应customer表
地区:主要对应district表
商品:主要对应item表
仓库:主要对应warehouse表

准备

ping rm-wz9v0vej02ys79jbj.mysql.rds.aliyuncs.com

mysql -h rm-wz9v0vej02ys79jbj.mysql.rds.aliyuncs.com -P 3306 -u myaccount -p

输入密码:Aa123456
创库,名字为:TPCC:
CREATE DATABASE TPCC DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;


导入项目中的出初始化数据脚本:
创建表:create_table.sql
/usr/bin/mysql -h rm-wz9v0vej02ys79jbj.mysql.rds.aliyuncs.com -u myaccount -p tpcc < /root/tpcc-mysql/create_table.sql

创建索引和外键:add_fkey_idx.sql
/usr/bin/mysql -h rm-wz9v0vej02ys79jbj.mysql.rds.aliyuncs.com -u myaccount -p tpcc < /root/tpcc-mysql/add_fkey_idx.sql

测试

cd /opt/tpcc-mysql
./tpcc_load -h rm-wz9v0vej02ys79jbj.mysql.rds.aliyuncs.com -P 3306 -d TPCC -u myaccount -p Aa123456 -w 80
-w 80 表示创建 80 个仓库数据
这个过程花费时间还是挺长的,建议测试机是高性能计算型。2CPU 差不多要 8h,你自己估量下。
我这边 RDS 监控中,曲线上每秒 insert 差不多在 2W 差不多,如果你没有这个数,速度可能就很慢了。
我这边差不多用了 2.5h 完成数据准备。


插入过程 RDS-2C4G 的监控情况:
CPU利用率 24%
内存 30% ~ 40% (随着数据增加而增大)
连接数:1%
IOPS:9%
已使用存储空间:5.5G ~ 10G

要模拟出够真实的数据,仓库不要太少,一般要大于 100,
下面是基于 80 个库的最终数据:

select count(*) from customer;
    2400000
select count(*) from district;
    800    
select count(*) from history;
    2400000
select count(*) from item;
    100000
select count(*) from new_orders;
    720000
select count(*) from order_line;
    23996450
select count(*) from orders;
    2400000
select count(*) from stock;
    8000000
select count(*) from warehouse;
    80

./tpcc_start -h rm-wz9v0vej02ys79jbj.mysql.rds.aliyuncs.com -P 3306 -d TPCC -u myaccount -p Aa123456 -w 80 -c 200 -r 300 -l 1800 -f /opt/mysql_tpcc_100_20190325

-w 100 表示 100 个仓库数据
-c 200 表示并发 200 个线程
-r 300 表示预热 300 秒
-l 1800 表示持续压测 1800 秒

报表

<TpmC>
188.000 TpmC
TpmC结果值(每分钟事务数,该值是第一次统计结果中的新订单事务数除以总耗时分钟数,例如本例中是:372/2=186)
tpmC值在国内外被广泛用于衡量计算机系统的事务处理能力
1780, trx: 979, 95%: 1849.535, 99%: 2402.613, max_rt: 3401.947, 986|3248.772, 98|698.821, 103|4202.110, 101|4547.416
1790, trx: 1021, 95%: 1898.903, 99%: 2700.936, max_rt: 3848.142, 999|3150.117, 100|500.740, 102|3600.104, 100|5551.834
1800, trx: 989, 95%: 1899.472, 99%: 2847.899, max_rt: 4455.064, 989|3049.921, 101|699.144, 97|3599.021, 102|5151.141

STOPPING THREADS........................................................................................................................................................................................................

<Raw Results>
  [0] sc:2 lt:174378  rt:0  fl:0 avg_rt: 1192.8 (5)
  [1] sc:253 lt:173935  rt:0  fl:0 avg_rt: 542.7 (5)
  [2] sc:4726 lt:12712  rt:0  fl:0 avg_rt: 144.7 (5)
  [3] sc:0 lt:17435  rt:0  fl:0 avg_rt: 3029.8 (80)
  [4] sc:0 lt:17435  rt:0  fl:0 avg_rt: 3550.7 (20)
 in 1800 sec.

<Raw Results2(sum ver.)>
  [0] sc:2  lt:174378  rt:0  fl:0
  [1] sc:254  lt:174096  rt:0  fl:0
  [2] sc:4726  lt:12712  rt:0  fl:0
  [3] sc:0  lt:17437  rt:0  fl:0
  [4] sc:0  lt:17435  rt:0  fl:0

<Constraint Check> (all must be [OK])
 [transaction percentage]
        Payment: 43.45% (>=43.0%) [OK]
   Order-Status: 4.35% (>= 4.0%) [OK]
       Delivery: 4.35% (>= 4.0%) [OK]
    Stock-Level: 4.35% (>= 4.0%) [OK]
 [response time (at least 90% passed)]
      New-Order: 0.00%  [NG] *
        Payment: 0.15%  [NG] *
   Order-Status: 27.10%  [NG] *
       Delivery: 0.00%  [NG] *
    Stock-Level: 0.00%  [NG] *

<TpmC>
                 5812.667 TpmC
1780, trx: 2303, 95%: 796.121, 99%: 1099.640, max_rt: 1596.883, 2293|2249.288, 232|256.393, 230|1694.050, 235|2550.775
1790, trx: 2336, 95%: 798.030, 99%: 1093.403, max_rt: 1547.840, 2338|2803.739, 234|305.185, 232|1799.869, 228|2453.748
1800, trx: 2305, 95%: 801.381, 99%: 1048.528, max_rt: 1297.465, 2306|1798.565, 229|304.329, 227|1649.609, 233|2549.599

STOPPING THREADS........................................................................................................................................................................................................

<Raw Results>
  [0] sc:7 lt:406567  rt:0  fl:0 avg_rt: 493.7 (5)
  [1] sc:10485 lt:395860  rt:0  fl:0 avg_rt: 240.1 (5)
  [2] sc:24615 lt:16045  rt:0  fl:0 avg_rt: 49.4 (5)
  [3] sc:0 lt:40651  rt:0  fl:0 avg_rt: 1273.6 (80)
  [4] sc:0 lt:40656  rt:0  fl:0 avg_rt: 1665.3 (20)
 in 1800 sec.

<Raw Results2(sum ver.)>
  [0] sc:7  lt:406569  rt:0  fl:0
  [1] sc:10487  lt:396098  rt:0  fl:0
  [2] sc:24615  lt:16045  rt:0  fl:0
  [3] sc:0  lt:40655  rt:0  fl:0
  [4] sc:0  lt:40659  rt:0  fl:0

<Constraint Check> (all must be [OK])
 [transaction percentage]
        Payment: 43.46% (>=43.0%) [OK]
   Order-Status: 4.35% (>= 4.0%) [OK]
       Delivery: 4.35% (>= 4.0%) [OK]
    Stock-Level: 4.35% (>= 4.0%) [OK]
 [response time (at least 90% passed)]
      New-Order: 0.00%  [NG] *
        Payment: 2.58%  [NG] *
   Order-Status: 60.54%  [NG] *
       Delivery: 0.00%  [NG] *
    Stock-Level: 0.00%  [NG] *

<TpmC>
                 13552.467 TpmC
1780, trx: 4502, 95%: 398.131, 99%: 501.634, max_rt: 772.128, 4473|740.073, 446|183.361, 448|1042.264, 442|1302.569
1790, trx: 4465, 95%: 398.489, 99%: 541.424, max_rt: 803.659, 4476|845.313, 448|152.917, 450|997.319, 454|1250.160
1800, trx: 4506, 95%: 397.774, 99%: 501.334, max_rt: 747.074, 4508|701.625, 453|108.619, 450|1052.293, 451|1107.277

STOPPING THREADS........................................................................................................................................................................................................

<Raw Results>
  [0] sc:20 lt:803738  rt:0  fl:0 avg_rt: 240.5 (5)
  [1] sc:13844 lt:789535  rt:0  fl:0 avg_rt: 128.5 (5)
  [2] sc:54560 lt:25817  rt:0  fl:0 avg_rt: 22.1 (5)
  [3] sc:0 lt:80372  rt:0  fl:0 avg_rt: 739.8 (80)
  [4] sc:0 lt:80378  rt:0  fl:0 avg_rt: 771.1 (20)
 in 1800 sec.

<Raw Results2(sum ver.)>
  [0] sc:20  lt:803747  rt:0  fl:0
  [1] sc:13845  lt:789916  rt:0  fl:0
  [2] sc:54561  lt:25817  rt:0  fl:0
  [3] sc:0  lt:80377  rt:0  fl:0
  [4] sc:0  lt:80381  rt:0  fl:0

<Constraint Check> (all must be [OK])
 [transaction percentage]
        Payment: 43.47% (>=43.0%) [OK]
   Order-Status: 4.35% (>= 4.0%) [OK]
       Delivery: 4.35% (>= 4.0%) [OK]
    Stock-Level: 4.35% (>= 4.0%) [OK]
 [response time (at least 90% passed)]
      New-Order: 0.00%  [NG] *
        Payment: 1.72%  [NG] *
   Order-Status: 67.88%  [NG] *
       Delivery: 0.00%  [NG] *
    Stock-Level: 0.00%  [NG] *

<TpmC>
                 26791.934 TpmC
1780, trx: 8413, 95%: 203.560, 99%: 279.322, max_rt: 451.010, 8414|441.849, 841|92.900, 839|583.340, 843|644.276
1790, trx: 8269, 95%: 204.599, 99%: 282.602, max_rt: 444.075, 8262|412.414, 827|91.551, 831|665.421, 824|616.396
1800, trx: 8395, 95%: 202.285, 99%: 255.026, max_rt: 436.136, 8404|446.292, 839|87.081, 839|609.221, 842|697.509

STOPPING THREADS........................................................................................................................................................................................................

<Raw Results>
  [0] sc:37 lt:1532893  rt:0  fl:0 avg_rt: 124.8 (5)
  [1] sc:36091 lt:1496111  rt:0  fl:0 avg_rt: 68.5 (5)
  [2] sc:105738 lt:47555  rt:0  fl:0 avg_rt: 11.4 (5)
  [3] sc:0 lt:153285  rt:0  fl:0 avg_rt: 404.6 (80)
  [4] sc:0 lt:153293  rt:0  fl:0 avg_rt: 389.5 (20)
 in 1800 sec.

<Raw Results2(sum ver.)>
  [0] sc:37  lt:1532918  rt:0  fl:0
  [1] sc:36093  lt:1496868  rt:0  fl:0
  [2] sc:105739  lt:47556  rt:0  fl:0
  [3] sc:0  lt:153297  rt:0  fl:0
  [4] sc:0  lt:153298  rt:0  fl:0

<Constraint Check> (all must be [OK])
 [transaction percentage]
        Payment: 43.47% (>=43.0%) [OK]
   Order-Status: 4.35% (>= 4.0%) [OK]
       Delivery: 4.35% (>= 4.0%) [OK]
    Stock-Level: 4.35% (>= 4.0%) [OK]
 [response time (at least 90% passed)]
      New-Order: 0.00%  [NG] *
        Payment: 2.36%  [NG] *
   Order-Status: 68.98%  [NG] *
       Delivery: 0.00%  [NG] *
    Stock-Level: 0.00%  [NG] *

<TpmC>
                 51097.668 TpmC
select count(*) from customer;
    2400000
select count(*) from district;
    800    
select count(*) from history;
    5779395
select count(*) from item;
    100000
select count(*) from new_orders;
    764970
select count(*) from order_line;
    57453708
select count(*) from orders;
    5745589
select count(*) from stock;
    8000000
select count(*) from warehouse;
    80

资料