1. Redis内存管理
支持配置 Redis
服务器能够使用的最大内存量,确保 Redis
使用内存不会无限制增长,从而消耗掉整个系统的所有可用内存,导致系统不稳定或崩溃。
redis.conf
配置文件中对应的配置项如下:
# maxmemory <bytes>
<bytes>
的单位是字节,可以配置为kb
、mb
、gb
,示例:
maxmemory 1gb
注意事项 :
- 如果设置的太小,可能会频繁地进行触发内存淘汰,导致性能下降,数据丢失,如果设置的太大,可能会占用过多系统资源,影响其他应用的运行。因此,需要根据实际的应用场景和硬件资源做出合理的选择。
- 在主从复制环境中,这个限制不包含复制时的输出缓冲区,所以配置时,需要给缓冲区预留内存。
2. 内存淘汰
2.1 策略
当 Redis
使用内存达到限制时,会根据相关的淘汰策略尝试移除 key
来清理内存。如果根据策略无法移除 key
,或者策略设置为
noeviction
(不清理内存)时, Redis
会进入只读模式,SET
、LPUSH
等写入命令会返回错误。
内存淘汰策略的配置项为:
# maxmemory-policy noeviction
可配置内容说明如下:
volatile-lru
:使用近似的LRU
算法来移除设置了过期时间的键。会优先移除那些长时间未被访问的、设置了过期时间的键。allkeys-lru
::使用近似的LRU
算法来移除任何键,无论它们是否设置了过期时间。会考虑所有键的访问频率,移除那些最久未被访问的键。volatile-lfu
:使用近似的LFU
算法来移除设置了过期时间的键。allkeys-lfu
:使用近似的LFU
算法来移除任何键。volatile-random
: 随机移除设置了过期时间的键。allkeys-random
: 随机移除任意键。volatile-ttl
:移除那些设置了过期时间,并且即将过期的键。noeviction
:不进行任何键的移除操作,仅对写操作返回错误。
LRU
(The Least Recently Used
)是最经典的一种淘汰算法,其核心原理是淘汰最近最少使用的数据。如果一个数据在最近一段时间没有被访问到,那么在将来它被访问的可能性也很低。
示例,当配置为 allkeys-lru
时,Redis
在每次访问 key
时,会更新其最近一次的访问时间,当触发了淘汰策略时,假设当前存在以下几个键:
key | 访问时间 |
---|---|
key001 | 2024-09-02 08:00:00 |
key002 | 2024-08-02 09:00:00 |
key003 | 2024-09-02 10:00:00 |
如果必须淘汰一个,key003
是最近未被访问时间最长的,根据 LRU
算法 Redis
会删除这个 key
。如果 key003
是热点数据,只是在最近这段时间未被访问,却被删除了,由此 LRU
存在一定的缺陷。
LFU
(Least Frequently Used
)算法的核心原理是淘汰最近使用频率最少的数据。如果一个数据在近期被访问地频率很低,那么在将来它被访问的可能性也很低。
示例,当配置为 allkeys-lfu
时,Redis
在每次访问 key
时,会更新其最近一次的访问时间,并记录其访问频率。当触发了淘汰策略时,假设当前存在以下几个键:
key | 访问时间 | 在某个时间段被访问的次数 |
---|---|---|
key001 | 2024-09-02 08:00:00 | 10 |
key002 | 2024-09-02 09:00:00 | 8 |
key003 | 2024-09-02 10:00:00 | 8 |
key002
和 key003
的访问频率一致,但是 key002
的最后一次的访问时间更小,根据 LFU
算法还是会删除key003
。
2.2 样本数量
LRU
、LFU
、TTL
都不是很精确算法,而是近似算法(为了节省内存),可以根据需要调整其速度或精度。默认情况下,Redis
会检查五个键,并选择最近最少使用的那个。
可以使用以下配置修改检查样本的数量:
# maxmemory-samples 5
默认为5个样本,已经可以产生足够好的效果。选择 10
个样本可以非常接近真实的 LRU
效果,但会消耗更多的 CPU
资源。选择 3
个样本则更快,但精度不是很高。
2.3 逐出因子
当 key
被逐出时,可以设置延时,取值为 [0-100]
,默认为 10
。
# maxmemory-eviction-tenacity 10
若降低该值,可能会降低延时,但会影响数据逐出的有效性。当写入流量特别大时,可增加该值,但延时会增大。100
表示忽略延时,不停逐出直到内存降至maxmemory
以下或者没有可逐出 Key
。
3. 副本的内存限制
在主从关系中,副本默认会忽略自身的 maxmemory
配置,除非在故障转移后或手动操作升级为主节点。副本的内存淘汰仅由主节点处理,由其发送 DEL
命令到副本。这种方式确保了主节点和副本之间的一致性,但是在部署副本时,一定要确保其所在服务器的内存是够用的。
可以通过以下配置修改默认行为(不推荐):
# replica-ignore-maxmemory yes
4. 回收过期键
Redis
通过两种方式回收过期键:
- 主动删除:定期扫描,检查过期时间并删除已过期的
key
- 被动删除:用户访问某个
key
的时候,如果过期了就立即删除
回收过期键可以释放那些已过期,且短时间内不会再被访问的键所占用的内存。在主动删除进行回收的过程中,可以配置查找的频率和深度,值越大会越积极地寻找并删除过期键。
# active-expire-effort 1
默认值为 1
,Redis
会以一种相对较为平衡的方式在内存使用、CPU
消耗和系统延迟之间做出权衡。可配置的最大值为10
,Redis
会更积极的收过期键以释放内存,使用更多的 CPU
资源,周期会更长,适用于很多键频繁过期的场景。
5. 附录
redis.conf
配置文件内存管理部分:
############################## MEMORY MANAGEMENT ################################
# Set a memory usage limit to the specified amount of bytes.
# When the memory limit is reached Redis will try to remove keys
# according to the eviction policy selected (see maxmemory-policy).
#
# If Redis can't remove keys according to the policy, or if the policy is
# set to 'noeviction', Redis will start to reply with errors to commands
# that would use more memory, like SET, LPUSH, and so on, and will continue
# to reply to read-only commands like GET.
#
# This option is usually useful when using Redis as an LRU or LFU cache, or to
# set a hard memory limit for an instance (using the 'noeviction' policy).
#
# WARNING: If you have replicas attached to an instance with maxmemory on,
# the size of the output buffers needed to feed the replicas are subtracted
# from the used memory count, so that network problems / resyncs will
# not trigger a loop where keys are evicted, and in turn the output
# buffer of replicas is full with DELs of keys evicted triggering the deletion
# of more keys, and so forth until the database is completely emptied.
#
# In short... if you have replicas attached it is suggested that you set a lower
# limit for maxmemory so that there is some free RAM on the system for replica
# output buffers (but this is not needed if the policy is 'noeviction').
#
# maxmemory <bytes>
# MAXMEMORY POLICY: how Redis will select what to remove when maxmemory
# is reached. You can select one from the following behaviors:
#
# volatile-lru -> Evict using approximated LRU, only keys with an expire set.
# allkeys-lru -> Evict any key using approximated LRU.
# volatile-lfu -> Evict using approximated LFU, only keys with an expire set.
# allkeys-lfu -> Evict any key using approximated LFU.
# volatile-random -> Remove a random key having an expire set.
# allkeys-random -> Remove a random key, any key.
# volatile-ttl -> Remove the key with the nearest expire time (minor TTL)
# noeviction -> Don't evict anything, just return an error on write operations.
#
# LRU means Least Recently Used
# LFU means Least Frequently Used
#
# Both LRU, LFU and volatile-ttl are implemented using approximated
# randomized algorithms.
#
# Note: with any of the above policies, when there are no suitable keys for
# eviction, Redis will return an error on write operations that require
# more memory. These are usually commands that create new keys, add data or
# modify existing keys. A few examples are: SET, INCR, HSET, LPUSH, SUNIONSTORE,
# SORT (due to the STORE argument), and EXEC (if the transaction includes any
# command that requires memory).
#
# The default is:
#
# maxmemory-policy noeviction
# LRU, LFU and minimal TTL algorithms are not precise algorithms but approximated
# algorithms (in order to save memory), so you can tune it for speed or
# accuracy. By default Redis will check five keys and pick the one that was
# used least recently, you can change the sample size using the following
# configuration directive.
#
# The default of 5 produces good enough results. 10 Approximates very closely
# true LRU but costs more CPU. 3 is faster but not very accurate.
#
# maxmemory-samples 5
# Eviction processing is designed to function well with the default setting.
# If there is an unusually large amount of write traffic, this value may need to
# be increased. Decreasing this value may reduce latency at the risk of
# eviction processing effectiveness
<h4 id="0minimumlatency10default100processwithoutregardtolatency">0 = minimum latency, 10 = default, 100 = process without regard to latency</h4>
#
# maxmemory-eviction-tenacity 10
# Starting from Redis 5, by default a replica will ignore its maxmemory setting
# (unless it is promoted to master after a failover or manually). It means
# that the eviction of keys will be just handled by the master, sending the
# DEL commands to the replica as keys evict in the master side.
#
# This behavior ensures that masters and replicas stay consistent, and is usually
# what you want, however if your replica is writable, or you want the replica
# to have a different memory setting, and you are sure all the writes performed
# to the replica are idempotent, then you may change this default (but be sure
# to understand what you are doing).
#
# Note that since the replica by default does not evict, it may end using more
# memory than the one set via maxmemory (there are certain buffers that may
# be larger on the replica, or data structures may sometimes take more memory
# and so forth). So make sure you monitor your replicas and make sure they
# have enough memory to never hit a real out-of-memory condition before the
# master hits the configured maxmemory setting.
#
# replica-ignore-maxmemory yes
# Redis reclaims expired keys in two ways: upon access when those keys are
# found to be expired, and also in background, in what is called the
# "active expire key". The key space is slowly and interactively scanned
# looking for expired keys to reclaim, so that it is possible to free memory
# of keys that are expired and will never be accessed again in a short time.
#
# The default effort of the expire cycle will try to avoid having more than
# ten percent of expired keys still in memory, and will try to avoid consuming
# more than 25% of total memory and to add latency to the system. However
# it is possible to increase the expire "effort" that is normally set to
# "1", to a greater value, up to the value "10". At its maximum value the
# system will use more CPU, longer cycles (and technically may introduce
# more latency), and will tolerate less already expired keys still present
# in the system. It's a tradeoff between memory, CPU and latency.
#
# active-expire-effort 1