mysqlslap: Error when connecting to server: Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2)
ln -s /usr/program/mysql/data/mysql.sock /tmp/mysql.sock,主要是我的 sock 文件位置是自己的配置的,跟 mysqlslap 默认去读的地方不一样。ln -s /usr/program/mysql/bin/mysqlslap /usr/binmysqlslap --defaults-file=/etc/my.cnf -a --auto-generate-sql-load-type=mixed --auto-generate-sql-add-autoincrement --engine=innodb --concurrency=50,100 --number-of-queries=1000 --iterations=2 --debug-info -uroot -p123456
-a,表示自动生成要测试的数据,等同于:--auto-generate-sql--debug-info,代表要额外输出 CPU 以及内存的相关信息。mysqlslap --defaults-file=/etc/my.cnf --concurrency=50,100,200 --iterations=2 --number-int-cols=7 --number-char-cols=13 --auto-generate-sql --auto-generate-sql-add-autoincrement --auto-generate-sql-load-type=mixed --engine=innodb --number-of-queries=1000 --debug-info -S /tmp/mysql.sock -uroot -p123456
-number-int-cols=7 表示生成的表中必须有 7 个 int 类型的列-number-char-cols=13 表示生成的表中必须有 13 个 char 类型的列mysqlslap --defaults-file=/etc/my.cnf --concurrency=50,100,200,500,1000 --iterations=10 --number-int-cols=7 --number-char-cols=13 --auto-generate-sql --auto-generate-sql-add-autoincrement --auto-generate-sql-load-type=mixed --engine=innodb --number-of-queries=10000 --debug-info -S /tmp/mysql.sock -uroot -p123456youmeek_navmysqlslap --defaults-file=/etc/my.cnf --create-schema=youmeek_nav --query="SELECT * FROM nav_url;" --debug-info -uroot -p123456SELECT * FROM sys_user;SELECT * FROM nav_column;SELECT * FROM nav_url;
mysqlslap --defaults-file=/etc/my.cnf --create-schema=youmeek_nav --query="/opt/test.sql" --delimiter=";" --debug-info -uroot -p123456--delimiter=”;” 表示文件中不同 sql 的分隔符是什么mysqlslap --help 查看所有参数--auto-generate-sql-load-type=XXX,XXX 代表要测试的是读还是写还是两者混合,该值分别有:read,write,update,mixed,默认是 mixed--auto-generate-sql-add-autoincrement 代表对生成的表自动添加 auto_increment 列--debug-info 代表要额外输出 CPU 以及内存的相关信息。--only-print 打印压力测试的时候 mysqlslap 到底做了什么事,通过 sql 语句方式告诉我们。yum -y install automake libtoolsysbench-1.0.3.zip/usr/program/mysql
/usr/program/mysql/include/usr/program/mysql/libexport LD_LIBRARY_PATH=/usr/program/mysql/lib/unzip sysbench-1.0.3.zipcd sysbench-1.0.3./autogen.sh./configure --with-mysql-includes=/usr/program/mysql/include --with-mysql-libs=/usr/program/mysql/lib/makemake installsysbench --versioncd /usr/local/share/sysbench,等下测试的时候需要指定这些脚本位置,用这些脚本测试的。vim /usr/local/share/sysbench/oltp_common.lua 文件。常修改的参数:
tables,生成多少张表table_size,每张表多少记录数sbtestsysbench /usr/local/share/sysbench/oltp_point_select.lua --threads=15 --report-interval=10 --time=120 --mysql-user=root --mysql-password=123456 --mysql-host=127.0.0.1 --mysql-port=3306 preparesysbench /usr/local/share/sysbench/oltp_point_select.lua --threads=15 --report-interval=10 --time=120 --mysql-user=root --mysql-password=123456 --mysql-host=127.0.0.1 --mysql-port=3306 runsysbench /usr/local/share/sysbench/oltp_point_select.lua --threads=15 --report-interval=10 --time=120 --mysql-user=root --mysql-password=123456 --mysql-host=127.0.0.1 --mysql-port=3306 cleanupMySQL error: 1461 "Can't create more than max_prepared_stmt_count statements,那你需要在 MySQL 中执行这句临时设置 SQL:SET GLOBAL max_prepared_stmt_count=100000;sysbench /usr/local/share/sysbench/oltp_read_write.lua --threads=100 --report-interval=10 --time=100 --mysql-user=root --mysql-password=123456 --mysql-host=127.0.0.1 --mysql-port=3306 preparesysbench /usr/local/share/sysbench/oltp_read_write.lua --threads=100 --report-interval=10 --time=100 --mysql-user=root --mysql-password=123456 --mysql-host=127.0.0.1 --mysql-port=3306 runsysbench /usr/local/share/sysbench/oltp_read_write.lua --threads=100 --report-interval=10 --time=100 --mysql-user=root --mysql-password=123456 --mysql-host=127.0.0.1 --mysql-port=3306 cleanup--threads=15 表示发起 15 个并发连接--report-interval=10 表示控制台每 10 秒输出一次测试进度报告--time=120 总的测试时长为 120 秒--max-requests=0 表示总请求数为 0,因为上面已经定义了总执行时长,所以总请求数可以设定为 0;也可以只设定总请求数,不设定执行时长--percentile=99 表示设定采样比例,即丢弃1%的长请求,在剩余的99%里取最大值。默认是 95%,Running the test with following options: Number of threads: 15 Report intermediate results every 10 second(s) Initializing random number generator from current time
Initializing worker threads…
Threads started!
[ 10s ] thds: 15 tps: 337.43 qps: 6773.72 (r/w/o: 4745.03/1351.92/676.76) lat (ms,95%): 73.13 err/s: 0.40 reconn/s: 0.00 [ 20s ] thds: 15 tps: 340.12 qps: 6813.82 (r/w/o: 4772.12/1361.06/680.63) lat (ms,95%): 71.83 err/s: 0.40 reconn/s: 0.00 [ 30s ] thds: 15 tps: 344.78 qps: 6897.36 (r/w/o: 4828.36/1379.23/689.77) lat (ms,95%): 71.83 err/s: 0.20 reconn/s: 0.00 [ 40s ] thds: 15 tps: 343.32 qps: 6876.75 (r/w/o: 4815.15/1374.47/687.14) lat (ms,95%): 71.83 err/s: 0.60 reconn/s: 0.00 [ 50s ] thds: 15 tps: 342.80 qps: 6864.76 (r/w/o: 4806.67/1371.89/686.20) lat (ms,95%): 73.13 err/s: 0.50 reconn/s: 0.00 [ 60s ] thds: 15 tps: 347.90 qps: 6960.74 (r/w/o: 4873.93/1390.71/696.10) lat (ms,95%): 70.55 err/s: 0.30 reconn/s: 0.00 [ 70s ] thds: 15 tps: 346.70 qps: 6942.39 (r/w/o: 4859.29/1389.30/693.80) lat (ms,95%): 70.55 err/s: 0.40 reconn/s: 0.00 [ 80s ] thds: 15 tps: 345.60 qps: 6914.88 (r/w/o: 4841.48/1382.00/691.40) lat (ms,95%): 70.55 err/s: 0.20 reconn/s: 0.00 [ 90s ] thds: 15 tps: 341.10 qps: 6830.31 (r/w/o: 4782.20/1365.40/682.70) lat (ms,95%): 74.46 err/s: 0.50 reconn/s: 0.00 [ 100s ] thds: 15 tps: 341.20 qps: 6829.33 (r/w/o: 4782.12/1364.41/682.80) lat (ms,95%): 74.46 err/s: 0.40 reconn/s: 0.00 [ 110s ] thds: 15 tps: 343.40 qps: 6875.79 (r/w/o: 4812.29/1376.50/687.00) lat (ms,95%): 71.83 err/s: 0.20 reconn/s: 0.00 [ 120s ] thds: 15 tps: 347.00 qps: 6943.51 (r/w/o: 4862.40/1386.70/694.40) lat (ms,95%): 71.83 err/s: 0.40 reconn/s: 0.00 SQL statistics: queries performed: read: 577836 –读总数 write: 164978 –写总数 other: 82503 –其他操作(CURD之外的操作,例如COMMIT) total: 825317 –全部总数 transactions: 41229 (343.51 per sec.) –总事务数(每秒事务数,这个每秒事务数也就是:TPS 吞吐量) queries: 825317 (6876.33 per sec.) –查询总数(查询次数) ignored errors: 45 (0.37 per sec.) –总忽略错误总数(每秒忽略错误次数) reconnects: 0 (0.00 per sec.) –重连总数(每秒重连次数)
General statistics: total time: 120.0214s –总耗时 total number of events: 41229 –共发生多少事务数
Latency (ms): min: 7.19 –最小耗时 avg: 43.66 –平均耗时 max: 162.82 –最长耗时 95th percentile: 71.83 –超过95%平均耗时 sum: 1799860.45
Threads fairness: events (avg/stddev): 2748.6000/132.71 –总处理事件数/标准偏差 execution time (avg/stddev): 119.9907/0.00 –总执行时间/标准偏差
TPC-C 是专门针对联机交易处理系统(OLTP系统)的规范,一般情况下我们也把这类系统称为业务处理系统。
TPC-C是TPC(Transaction Processing Performance Council)组织发布的一个测试规范,用于模拟测试复杂的在线事务处理系统。其测试结果包括每分钟事务数(tpmC),以及每事务的成本(Price/tpmC)。
在进行大压力下MySQL的一些行为时经常使用。
yum install mysql-develgit clone https://github.com/Percona-Lab/tpcc-mysql
cd tpcc-mysql/src
make
如果make没报错,就会在tpcc-mysql 根目录文件夹下生成tpcc二进制命令行工具tpcc_load、tpcc_start
如果要同时支持 PgSQL 可以考虑:https://github.com/Percona-Lab/sysbench-tpcc
tpcc-mysql的业务逻辑及其相关的几个表作用如下:
New-Order:新订单,主要对应 new_orders 表
Payment:支付,主要对应 orders、history 表
Order-Status:订单状态,主要对应 orders、order_line 表
Delivery:发货,主要对应 order_line 表
Stock-Level:库存,主要对应 stock 表
其他相关表:
客户:主要对应customer表
地区:主要对应district表
商品:主要对应item表
仓库:主要对应warehouse表
ping rm-wz9v0vej02ys79jbj.mysql.rds.aliyuncs.com
mysql -h rm-wz9v0vej02ys79jbj.mysql.rds.aliyuncs.com -P 3306 -u myaccount -p
输入密码:Aa123456
创库,名字为:TPCC:
CREATE DATABASE TPCC DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
导入项目中的出初始化数据脚本:
创建表:create_table.sql
/usr/bin/mysql -h rm-wz9v0vej02ys79jbj.mysql.rds.aliyuncs.com -u myaccount -p tpcc < /root/tpcc-mysql/create_table.sql
创建索引和外键:add_fkey_idx.sql
/usr/bin/mysql -h rm-wz9v0vej02ys79jbj.mysql.rds.aliyuncs.com -u myaccount -p tpcc < /root/tpcc-mysql/add_fkey_idx.sql
根据测试,不同的 ECS 测试机,不同的 RDS 测试结果有时候差距挺大的,这个很蛋疼。
cd /opt/tpcc-mysql
./tpcc_load -h rm-wz9v0vej02ys79jbj.mysql.rds.aliyuncs.com -P 3306 -d TPCC -u myaccount -p Aa123456 -w 80
-w 80 表示创建 80 个仓库数据
这个过程花费时间还是挺长的,建议测试机是高性能计算型。2CPU 差不多要 8h,你自己估量下。
我这边 RDS 监控中,曲线上每秒 insert 差不多在 2W 差不多,如果你没有这个数,速度可能就很慢了。
我这边差不多用了 2.5h 完成数据准备。
插入过程 RDS-2C4G 的监控情况:
CPU利用率 24%
内存 30% ~ 40% (随着数据增加而增大)
连接数:1%
IOPS:9%
已使用存储空间:5.5G ~ 10G
要模拟出够真实的数据,仓库不要太少,一般要大于 100,
下面是基于 80 个库的最终数据:
select count(*) from customer;
2400000
select count(*) from district;
800
select count(*) from history;
2400000
select count(*) from item;
100000
select count(*) from new_orders;
720000
select count(*) from order_line;
23996450
select count(*) from orders;
2400000
select count(*) from stock;
8000000
select count(*) from warehouse;
80
./tpcc_start -h rm-wz9v0vej02ys79jbj.mysql.rds.aliyuncs.com -P 3306 -d TPCC -u myaccount -p Aa123456 -w 80 -c 200 -r 300 -l 1800 -f /opt/mysql_tpcc_100_20190325
-w 100 表示 100 个仓库数据
-c 200 表示并发 200 个线程
-r 300 表示预热 300 秒
-l 1800 表示持续压测 1800 秒
<TpmC>
188.000 TpmC
TpmC结果值(每分钟事务数,该值是第一次统计结果中的新订单事务数除以总耗时分钟数,例如本例中是:372/2=186)
tpmC值在国内外被广泛用于衡量计算机系统的事务处理能力
1780, trx: 979, 95%: 1849.535, 99%: 2402.613, max_rt: 3401.947, 986|3248.772, 98|698.821, 103|4202.110, 101|4547.416
1790, trx: 1021, 95%: 1898.903, 99%: 2700.936, max_rt: 3848.142, 999|3150.117, 100|500.740, 102|3600.104, 100|5551.834
1800, trx: 989, 95%: 1899.472, 99%: 2847.899, max_rt: 4455.064, 989|3049.921, 101|699.144, 97|3599.021, 102|5151.141
STOPPING THREADS........................................................................................................................................................................................................
<Raw Results>
[0] sc:2 lt:174378 rt:0 fl:0 avg_rt: 1192.8 (5)
[1] sc:253 lt:173935 rt:0 fl:0 avg_rt: 542.7 (5)
[2] sc:4726 lt:12712 rt:0 fl:0 avg_rt: 144.7 (5)
[3] sc:0 lt:17435 rt:0 fl:0 avg_rt: 3029.8 (80)
[4] sc:0 lt:17435 rt:0 fl:0 avg_rt: 3550.7 (20)
in 1800 sec.
<Raw Results2(sum ver.)>
[0] sc:2 lt:174378 rt:0 fl:0
[1] sc:254 lt:174096 rt:0 fl:0
[2] sc:4726 lt:12712 rt:0 fl:0
[3] sc:0 lt:17437 rt:0 fl:0
[4] sc:0 lt:17435 rt:0 fl:0
<Constraint Check> (all must be [OK])
[transaction percentage]
Payment: 43.45% (>=43.0%) [OK]
Order-Status: 4.35% (>= 4.0%) [OK]
Delivery: 4.35% (>= 4.0%) [OK]
Stock-Level: 4.35% (>= 4.0%) [OK]
[response time (at least 90% passed)]
New-Order: 0.00% [NG] *
Payment: 0.15% [NG] *
Order-Status: 27.10% [NG] *
Delivery: 0.00% [NG] *
Stock-Level: 0.00% [NG] *
<TpmC>
5812.667 TpmC
1780, trx: 2303, 95%: 796.121, 99%: 1099.640, max_rt: 1596.883, 2293|2249.288, 232|256.393, 230|1694.050, 235|2550.775
1790, trx: 2336, 95%: 798.030, 99%: 1093.403, max_rt: 1547.840, 2338|2803.739, 234|305.185, 232|1799.869, 228|2453.748
1800, trx: 2305, 95%: 801.381, 99%: 1048.528, max_rt: 1297.465, 2306|1798.565, 229|304.329, 227|1649.609, 233|2549.599
STOPPING THREADS........................................................................................................................................................................................................
<Raw Results>
[0] sc:7 lt:406567 rt:0 fl:0 avg_rt: 493.7 (5)
[1] sc:10485 lt:395860 rt:0 fl:0 avg_rt: 240.1 (5)
[2] sc:24615 lt:16045 rt:0 fl:0 avg_rt: 49.4 (5)
[3] sc:0 lt:40651 rt:0 fl:0 avg_rt: 1273.6 (80)
[4] sc:0 lt:40656 rt:0 fl:0 avg_rt: 1665.3 (20)
in 1800 sec.
<Raw Results2(sum ver.)>
[0] sc:7 lt:406569 rt:0 fl:0
[1] sc:10487 lt:396098 rt:0 fl:0
[2] sc:24615 lt:16045 rt:0 fl:0
[3] sc:0 lt:40655 rt:0 fl:0
[4] sc:0 lt:40659 rt:0 fl:0
<Constraint Check> (all must be [OK])
[transaction percentage]
Payment: 43.46% (>=43.0%) [OK]
Order-Status: 4.35% (>= 4.0%) [OK]
Delivery: 4.35% (>= 4.0%) [OK]
Stock-Level: 4.35% (>= 4.0%) [OK]
[response time (at least 90% passed)]
New-Order: 0.00% [NG] *
Payment: 2.58% [NG] *
Order-Status: 60.54% [NG] *
Delivery: 0.00% [NG] *
Stock-Level: 0.00% [NG] *
<TpmC>
13552.467 TpmC
1780, trx: 4502, 95%: 398.131, 99%: 501.634, max_rt: 772.128, 4473|740.073, 446|183.361, 448|1042.264, 442|1302.569
1790, trx: 4465, 95%: 398.489, 99%: 541.424, max_rt: 803.659, 4476|845.313, 448|152.917, 450|997.319, 454|1250.160
1800, trx: 4506, 95%: 397.774, 99%: 501.334, max_rt: 747.074, 4508|701.625, 453|108.619, 450|1052.293, 451|1107.277
STOPPING THREADS........................................................................................................................................................................................................
<Raw Results>
[0] sc:20 lt:803738 rt:0 fl:0 avg_rt: 240.5 (5)
[1] sc:13844 lt:789535 rt:0 fl:0 avg_rt: 128.5 (5)
[2] sc:54560 lt:25817 rt:0 fl:0 avg_rt: 22.1 (5)
[3] sc:0 lt:80372 rt:0 fl:0 avg_rt: 739.8 (80)
[4] sc:0 lt:80378 rt:0 fl:0 avg_rt: 771.1 (20)
in 1800 sec.
<Raw Results2(sum ver.)>
[0] sc:20 lt:803747 rt:0 fl:0
[1] sc:13845 lt:789916 rt:0 fl:0
[2] sc:54561 lt:25817 rt:0 fl:0
[3] sc:0 lt:80377 rt:0 fl:0
[4] sc:0 lt:80381 rt:0 fl:0
<Constraint Check> (all must be [OK])
[transaction percentage]
Payment: 43.47% (>=43.0%) [OK]
Order-Status: 4.35% (>= 4.0%) [OK]
Delivery: 4.35% (>= 4.0%) [OK]
Stock-Level: 4.35% (>= 4.0%) [OK]
[response time (at least 90% passed)]
New-Order: 0.00% [NG] *
Payment: 1.72% [NG] *
Order-Status: 67.88% [NG] *
Delivery: 0.00% [NG] *
Stock-Level: 0.00% [NG] *
<TpmC>
26791.934 TpmC
1780, trx: 8413, 95%: 203.560, 99%: 279.322, max_rt: 451.010, 8414|441.849, 841|92.900, 839|583.340, 843|644.276
1790, trx: 8269, 95%: 204.599, 99%: 282.602, max_rt: 444.075, 8262|412.414, 827|91.551, 831|665.421, 824|616.396
1800, trx: 8395, 95%: 202.285, 99%: 255.026, max_rt: 436.136, 8404|446.292, 839|87.081, 839|609.221, 842|697.509
STOPPING THREADS........................................................................................................................................................................................................
<Raw Results>
[0] sc:37 lt:1532893 rt:0 fl:0 avg_rt: 124.8 (5)
[1] sc:36091 lt:1496111 rt:0 fl:0 avg_rt: 68.5 (5)
[2] sc:105738 lt:47555 rt:0 fl:0 avg_rt: 11.4 (5)
[3] sc:0 lt:153285 rt:0 fl:0 avg_rt: 404.6 (80)
[4] sc:0 lt:153293 rt:0 fl:0 avg_rt: 389.5 (20)
in 1800 sec.
<Raw Results2(sum ver.)>
[0] sc:37 lt:1532918 rt:0 fl:0
[1] sc:36093 lt:1496868 rt:0 fl:0
[2] sc:105739 lt:47556 rt:0 fl:0
[3] sc:0 lt:153297 rt:0 fl:0
[4] sc:0 lt:153298 rt:0 fl:0
<Constraint Check> (all must be [OK])
[transaction percentage]
Payment: 43.47% (>=43.0%) [OK]
Order-Status: 4.35% (>= 4.0%) [OK]
Delivery: 4.35% (>= 4.0%) [OK]
Stock-Level: 4.35% (>= 4.0%) [OK]
[response time (at least 90% passed)]
New-Order: 0.00% [NG] *
Payment: 2.36% [NG] *
Order-Status: 68.98% [NG] *
Delivery: 0.00% [NG] *
Stock-Level: 0.00% [NG] *
<TpmC>
51097.668 TpmC
select count(*) from customer;
2400000
select count(*) from district;
800
select count(*) from history;
5779395
select count(*) from item;
100000
select count(*) from new_orders;
764970
select count(*) from order_line;
57453708
select count(*) from orders;
5745589
select count(*) from stock;
8000000
select count(*) from warehouse;
80