优化阶段拦截写事务-j9九游会登录
功能介绍
在mysql中,单个事务如果涉及大规模数据操作(如插入、更新或删除数百万行),会长时间持有锁并大量消耗i/o资源,严重影响后续业务的正常执行。用户往往不得不强制终止会话,触发回滚操作。回滚过程通常与原事务执行时间相当,期间仍会占用数据库锁和i/o资源,进一步阻塞其他工作负载。
为避免此类性能瓶颈,华为云rds for mysql在此类大规模写入事务执行前进行拦截,防止因数据变更过大导致系统整体性能下降,并及时提醒客户关注相关风险,从而保障数据库的稳定运行和服务的高可用性。此外,该特性开启后对数据库写性能会有轻微影响。
前提条件
- 对于rds for mysql 5.7,要求内核版本大于等于5.7.44.241207。
- 对于rds for mysql 8.0,要求内核版本大于等于8.0.41.250900。
配置参数
|
参数名称 |
是否动态参数 |
数据类型 |
取值范围 |
描述 |
|---|---|---|---|---|
|
rds_max_data_mutation_num_per_dml |
是 |
integer |
[0, 10000-9999999999999] |
insert/update/delete可影响的行数的最大值,超过了该配置值,则提前拦截报错。 该参数是会话级参数。 默认值为0,表示不拦截。可配置0,以及[10000, 9999999999999]。 |
|
rds_data_mutation_counting_mode |
是 |
enum |
[precise, estimated] |
insert/update/delete影响的行数评估模式。在无法获取结果集行数的相对精确值时,estimated模式下依赖优化器估算值,precise模式下则不做拦截。 该参数是会话级参数。 默认值为estimated。 |
使用限制
当前仅支持语句级的限制,以下语句在执行前会提前风险识别并拦截:
|
sql语句 |
约束条件 |
|---|---|
|
insert ... values |
仅对sql语句values增加的行数进行限制,非实际插入的行数。 |
|
replace ... values |
仅对sql语句values增加的行数进行限制,非实际插入的行数。 |
|
insert ... select |
|
|
replace ... select |
|
|
create table ... select |
|
|
update table ... |
|
|
delete from table ... |
|
由于sql尚未实际执行,系统只能依赖优化器的估算值进行拦截判断,因此无法做到绝对精确。然而,在以下几类场景中,可以获取到相对准确的行数信息:
- 对表执行无限制条件的insert ... values操作。
- insert/replace ... select或create table ... select语句,其中 select 的结果在插入前需要缓存(比如物化临时表、sort缓存等)。
- update/delete语句在执行前需要缓存所有待更新行的row id(比如临时文件、sort 缓存等)。
- 单表查询可直接使用索引访问(如ref或者range),无额外过滤等操作。
在上述场景中,无论当前配置为precise模式还是estimated模式,系统都会基于获取到的行数来决定是否进行拦截处理。
使用示例
以下是优化阶段拦截写事务的使用示例:
- 表结构。
mysql> show create table t1\g *************************** 1. row *************************** table: t1 create table: create table `t1` ( `id` int not null auto_increment, `category` int not null, `value` int not null, `flag` enum('active','inactive') default 'active', `created_at` timestamp not null default '2025-05-05 01:01:01', primary key (`id`), key `idx_category` (`category`), key `idx_value` (`value`), key `idx_flag` (`flag`), key `idx_category_value` (`category`,`value`) ) engine=innodb auto_increment=120109 default charset=utf8mb4 collate=utf8mb4_0900_ai_ci 1 row in set (0.00 sec) mysql> create table t2 like t1; query ok, 0 rows affected (0.15 sec) - insert ... select语句拦截示例。
mysql> explain format=tree insert into t2 select * from t1 where category=1; ------------------------------------------------------------------------------------------------------ | explain | ------------------------------------------------------------------------------------------------------ | -> insert into t2 -> index lookup on t1 using idx_category (category=1) (cost=6276 rows=60112) | ------------------------------------------------------------------------------------------------------ 1 row in set (0.02 sec) mysql> set rds_max_data_mutation_num_per_dml=60111; query ok, 0 rows affected (0.00 sec) # 单表走ref索引无过滤条件,估算值超过门限,'estimated'模式下会拦截 mysql> set rds_data_mutation_counting_mode='estimated'; query ok, 0 rows affected (0.00 sec) mysql> insert into t2 select * from t1 where category=1; error 4167 (hy000): the number [60112] of data mutations in the sql statement has exceeded the maximum limit. please check the configuration parameter 'rds_max_data_mutation_num_per_dml'. mysql> # 单表走ref索引无过滤条件,估算值超过门限,'precise'模式下会拦截 mysql> set rds_data_mutation_counting_mode='precise'; query ok, 0 rows affected (0.00 sec) mysql> insert into t2 select * from t1 where category=1; error 4167 (hy000): the number [60112] of data mutations in the sql statement has exceeded the maximum limit. please check the configuration parameter 'rds_max_data_mutation_num_per_dml'. mysql> mysql> explain format=tree insert into t2 select * from t1 where category=1 and flag='active'; ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | explain | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -> insert into t2 -> filter: (t1.flag = 'active') (cost=3270 rows=30056) -> index lookup on t1 using idx_category (category=1) (cost=3270 rows=60112) | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1 row in set (0.00 sec) mysql> set rds_max_data_mutation_num_per_dml=30055; query ok, 0 rows affected (0.00 sec) # 存在无法走索引的过滤条件,且估算值超过门限,'estimated'模式下会拦截 mysql> set rds_data_mutation_counting_mode='estimated'; query ok, 0 rows affected (0.00 sec) mysql> insert into t2 select * from t1 where category=1 and flag='active'; error 4167 (hy000): the number [30056] of data mutations in the sql statement has exceeded the maximum limit. please check the configuration parameter 'rds_max_data_mutation_num_per_dml'. mysql> mysql> set rds_data_mutation_counting_mode='precise'; query ok, 0 rows affected (0.00 sec) # 存在无法走索引的过滤条件,且估算值超过门限,'precise'模式下不拦截 mysql> insert into t2 select * from t1 where category=1 and flag='active'; query ok, 32275 rows affected (37.11 sec) records: 32275 duplicates: 0 warnings: 0 mysql> - update语句拦截示例。
mysql> explain format=tree select * from t1 where category=1; -------------------------------------------------------------------------------- | explain | -------------------------------------------------------------------------------- | -> index lookup on t1 using idx_category (category=1) (cost=4710 rows=44448) | -------------------------------------------------------------------------------- 1 row in set (0.00 sec) mysql> set rds_max_data_mutation_num_per_dml=44447; query ok, 0 rows affected (0.00 sec) # 单表走ref索引无过滤条件,估算值超过门限,'estimated'模式下会拦截 mysql> set rds_data_mutation_counting_mode='estimated'; query ok, 0 rows affected (0.00 sec) mysql> update t1 set value=value 100 where category=1; error 4167 (hy000): the number [44448] of data mutations in the sql statement has exceeded the maximum limit. please check the configuration parameter 'rds_max_data_mutation_num_per_dml'. mysql> # 单表走ref索引无过滤条件,估算值超过门限,'precise'模式下会拦截 mysql> set rds_data_mutation_counting_mode='precise'; query ok, 0 rows affected (0.00 sec) mysql> update t1 set value=value 100 where category=1; error 4167 (hy000): the number [44448] of data mutations in the sql statement has exceeded the maximum limit. please check the configuration parameter 'rds_max_data_mutation_num_per_dml'. mysql> mysql> explain format=tree select * from t1 where category=1 and flag='active'; -------------------------------------------------------------------------------------------------------------------------------------------- | explain | -------------------------------------------------------------------------------------------------------------------------------------------- | -> filter: (t1.flag = 'active') (cost=2487 rows=22224) -> index lookup on t1 using idx_category (category=1) (cost=2487 rows=44448) | -------------------------------------------------------------------------------------------------------------------------------------------- 1 row in set (0.01 sec) mysql> mysql> set rds_max_data_mutation_num_per_dml=22223; query ok, 0 rows affected (0.00 sec) mysql> # 存在无法走索引的过滤条件,且估算值超过门限,'estimated'模式下会拦截 mysql> set rds_data_mutation_counting_mode='estimated'; query ok, 0 rows affected (0.00 sec) mysql> update t1 set value=value 100 where category=1 and flag='active'; error 4167 (hy000): the number [22224] of data mutations in the sql statement has exceeded the maximum limit. please check the configuration parameter 'rds_max_data_mutation_num_per_dml'. mysql> # 存在无法走索引的过滤条件,且估算值超过门限,'precise'模式下不拦截 mysql> set rds_data_mutation_counting_mode='precise'; query ok, 0 rows affected (0.00 sec) mysql> update t1 set value=value 100 where category=1 and flag='active'; query ok, 32275 rows affected (43.55 sec) rows matched: 32275 changed: 32275 warnings: 0 - delete语句拦截示例。
mysql> explain format=tree select * from t1 where category=2; -------------------------------------------------------------------------------- | explain | -------------------------------------------------------------------------------- | -> index lookup on t1 using idx_category (category=2) (cost=6276 rows=60112) | -------------------------------------------------------------------------------- 1 row in set (0.00 sec) mysql> mysql> set rds_max_data_mutation_num_per_dml=60111; query ok, 0 rows affected (0.00 sec) # 单表走ref索引无过滤条件,估算值超过门限,'estimated'模式下会拦截 mysql> set rds_data_mutation_counting_mode='estimated'; query ok, 0 rows affected (0.00 sec) mysql> delete from t1 where category=2; error 4167 (hy000): the number [60112] of data mutations in the sql statement has exceeded the maximum limit. please check the configuration parameter 'rds_max_data_mutation_num_per_dml'. mysql> # 单表走ref索引无过滤条件,估算值超过门限,'precise'模式下会拦截 mysql> set rds_data_mutation_counting_mode='precise'; query ok, 0 rows affected (0.00 sec) mysql> delete from t1 where category=2; error 4167 (hy000): the number [60112] of data mutations in the sql statement has exceeded the maximum limit. please check the configuration parameter 'rds_max_data_mutation_num_per_dml'. mysql> mysql> explain format=tree select * from t1 where category=2 and flag='active'; -------------------------------------------------------------------------------------------------------------------------------------------- | explain | -------------------------------------------------------------------------------------------------------------------------------------------- | -> filter: (t1.flag = 'active') (cost=3270 rows=30056) -> index lookup on t1 using idx_category (category=2) (cost=3270 rows=60112) | -------------------------------------------------------------------------------------------------------------------------------------------- 1 row in set (0.12 sec) mysql> set rds_max_data_mutation_num_per_dml=30055; query ok, 0 rows affected (0.00 sec) # 存在无法走索引的过滤条件,且估算值超过门限,'estimated'模式下会拦截 mysql> set rds_data_mutation_counting_mode='estimated'; query ok, 0 rows affected (0.00 sec) mysql> delete from t1 where category=2 and flag='active'; error 4167 (hy000): the number [30056] of data mutations in the sql statement has exceeded the maximum limit. please check the configuration parameter 'rds_max_data_mutation_num_per_dml'. mysql> # 存在无法走索引的过滤条件,且估算值超过门限,'precise'模式下不拦截 mysql> set rds_data_mutation_counting_mode='precise'; query ok, 0 rows affected (0.00 sec) mysql> delete from t1 where category=2 and flag='active'; query ok, 32040 rows affected (41.72 sec)
相关文档
意见反馈
文档内容是否对您有帮助?
如您有其它疑问,您也可以通过华为云社区问答频道来与我们联系探讨