跳转到主要内容

将易混淆字符规范化为安全文本

规范化易混淆字符,就是把每个仿冒字符替换为它所模仿的普通字符,使看起来相同的两个字符串在比较时也相等。在比较用户名、匹配黑名单或对标识符去重之前进行这一步尤其有用。

实例解析

输入
Cоnfig file: аdmin2, Noёl, café
检测到的文字系统
拉丁文 (Latn), 西里尔文 (Cyrl)
已标记字符
7
位置字符码位文字系统替换为规则
0CU+FF23拉丁文CNFKC 规范化
1оU+043E西里尔文o易混淆字符映射表
7fiU+FB01拉丁文fiNFKC 规范化
10:U+FF1A通用:易混淆字符映射表
12аU+0430西里尔文a易混淆字符映射表
172U+FF12通用2易混淆字符映射表
22ёU+0451西里尔文ë易混淆字符映射表
保留可读的 Unicode
Config file: admin2, Noël, café
严格的 ASCII 后备
Config file: admin2, Noel, café

工作原理

  • 首先按内置映射表替换已知的仿冒字符。映射表之外的字符使用 NFKC 规范化,把全角形式、连字和其他兼容字符转换为标准等价字符。
  • “保留可读的 Unicode”会在映射表定义了带重音字母时保留它,例如西里尔字母 ё → ë。“严格的 ASCII 后备”则改用普通 ASCII 字母(ё → e)。
  • 既不在映射表中、也不会被 NFKC 改变的字母会原样保留,因此 café 在两种模式下都保留重音。规范化后的文本只是比较用的键,而不是安全结论:请同时保存原文,并检查被标记的字符。

转换是尽力而为:映射的易混淆项和 NFKC 折叠是确定性的,但某些合法的 Unicode 不会被标记。

您的文字

粘贴或键入 — 结果会在您键入时更新(对于长输入会稍微去抖)。

已扫描 30 个字符
7 个可疑项
严格的 ASCII 后备
转换方式

严格 ASCII 模式积极用 ASCII 替换已知的相似字符。当字符位于捆绑地图或 NFKC 折叠中时,它可以更改合法的非拉丁文本。

原文(可疑字符已标记)

原始视图中的可疑字符带有下划线并标记为“可疑”。除了突出颜色。

suspicious character Csuspicious character оnfig suspicious character filesuspicious character : suspicious character аdminsuspicious character 2, Nosuspicious character ёl, café
清理后的输出
字符分析
索引(从0开始)原字符替换为代码点原因
0CCU+FF23NFKC normalization changed this character (compatibility or width folding).
1оoU+043ENon-ASCII confusable mapped to a safer Latin ASCII equivalent.
2nnU+006ENot flagged as a confusable or compatibility character.
3ffU+0066Not flagged as a confusable or compatibility character.
4iiU+0069Not flagged as a confusable or compatibility character.
5ggU+0067Not flagged as a confusable or compatibility character.
6 U+0020Not flagged as a confusable or compatibility character.
7fifiU+FB01NFKC normalization changed this character (compatibility or width folding).
8llU+006CNot flagged as a confusable or compatibility character.
9eeU+0065Not flagged as a confusable or compatibility character.
10::U+FF1ANon-ASCII confusable mapped to a safer Latin ASCII equivalent.
11 U+0020Not flagged as a confusable or compatibility character.
12аaU+0430Non-ASCII confusable mapped to a safer Latin ASCII equivalent.
13ddU+0064Not flagged as a confusable or compatibility character.
14mmU+006DNot flagged as a confusable or compatibility character.
15iiU+0069Not flagged as a confusable or compatibility character.
16nnU+006ENot flagged as a confusable or compatibility character.
1722U+FF12Non-ASCII confusable mapped to a safer Latin ASCII equivalent.
18,,U+002CNot flagged as a confusable or compatibility character.
19 U+0020Not flagged as a confusable or compatibility character.
20NNU+004ENot flagged as a confusable or compatibility character.
21ooU+006FNot flagged as a confusable or compatibility character.
22ёeU+0451Non-ASCII confusable mapped to a safer Latin ASCII equivalent.
23llU+006CNot flagged as a confusable or compatibility character.
24,,U+002CNot flagged as a confusable or compatibility character.
25 U+0020Not flagged as a confusable or compatibility character.
26ccU+0063Not flagged as a confusable or compatibility character.
27aaU+0061Not flagged as a confusable or compatibility character.
28ffU+0066Not flagged as a confusable or compatibility character.
29ééU+00E9Not flagged as a confusable or compatibility character.

更多指南