uniq
相关命令：暂无相关命令
用法：uniq [选项]... [文件]
从输入文件或者标准输入中筛选相邻的匹配行并写入到输出文件或标准输出。
不附加任何选项时匹配行将在首次出现处被合并。
长选项必须使用的参数对于短选项时也是必需使用的。
-c, --count 在每行前加上表示相应行目出现次数的前缀编号
-d, --repeated 只输出重复的行
-D, --all-repeated[=delimit-method 显示所有重复的行
delimit-method={none(default),prepend,separate}
以空行为界限
-f, --skip-fields=N 比较时跳过前N 列
-i, --ignore-case 在比较的时候不区分大小写
-s, --skip-chars=N 比较时跳过前N 个字符
-u, --unique 只显示唯一的行
-z, --zero-terminated 使用'\0'作为行结束符，而不是新换行
-w, --check-chars=N 对每行第N 个字符以后的内容不作对照
--help 显示此帮助信息并退出
--version 显示版本信息并退出
若域中为先空字符(通常包括空格以及制表符)，然后非空字符，域中字符前的空字符将被跳过。
提示：uniq 不会检查重复的行，除非它们是相邻的行。
如果您想先对输入排序，使用没有uniq 的"sort -u"。
同时，比较服从"LC_COLLATE" 变量所指定的规则。
例1
[root@localhost ~]# cat uniqtest #测试文件
this is a test
this is a test
this is a test
i am tank
i love tank
i love tank
this is a test
whom have a try
WhoM have a try
you have a try
i want to abroad
those are good men
we are good men
[zhangy@BlackGhost mytest]$ uniq -c uniqtest #uniq的一个特性，检查重复行的时候，只会检查相邻的行。重复数据，肯定有很多不是相邻在一起的
3 this is a test
1 i am tank
2 i love tank
1 this is a test #和第一行是重复的
1 whom have a try
1 WhoM have a try
1 you? have a try
1 i want to abroad
1 those are good men
1 we are good men
[zhangy@BlackGhost mytest]$ sort uniqtest |uniq -c #这样就可以解决上个例子中提到的问题
1 WhoM have a try
1 i am tank
2 i love tank
1 i want to abroad
4 this is a test
1 those are good men
1 we are good men
1 whom have a try
1 you have a try
[zhangy@BlackGhost mytest]$ uniq -d -c uniqtest #uniq -d 只显示重复的行
3 this is a test
2 i love tank
[zhangy@BlackGhost mytest]$ uniq -D uniqtest #uniq -D 只显示重复的行，并且把重复几行都显示出来。他不能和-c一起使用
this is a test
this is a test
this is a test
i love tank
i love tank
[zhangy@BlackGhost mytest]$ uniq -f 1 -c uniqtest #在这里those只有一行，显示的却是重复了，这是因为，-f 1 忽略了第一列，检查重复从第二字段开始的。
3 this is a test
1 i am tank
2 i love tank
1 this is a test
2 whom have a try
1 you have a try
1 i want to abroad
2 those are good men #只有一行，显示二行
[zhangy@BlackGhost mytest]$ uniq -i -c uniqtest #检查的时候，不区分大小写
3 this is a test
1 i am tank
2 i love tank
1 this is a test
2 whom have a try #一个大写，一个小写
1 you have a try
1 i want to abroad
1 those are good men
1 we are good men
[zhangy@BlackGhost mytest]$ uniq -s 4 -c uniqtest #检查的时候，不考虑前4个字符，这样whom have a try 就和 you have a try 就一样了。
3 this is a test
1 i am tank
2 i love tank
1 this is a test
3 whom have a try #根上一个例子有什么不同
1 i want to abroad
1 those are good men
1 we are good men
[zhangy@BlackGhost mytest]$ uniq -u uniqtest #去重复的项，然后全部显示出来
i am tank
this is a test
whom have a try
WhoM have a try
you have a try
want to abroad
those are good men
we are good men
[zhangy@BlackGhost mytest]$ uniq -w 2 -c uniqtest #对每行第2个字符以后的内容不作检查，所以i am tank 根 i love tank就一样了。
3 this is a test
3 i am tank
1 this is a test
1 whom have a try
1 WhoM have a try
1 you have a try
1 i want to abroad
1 those are good men
1 we are good men
例2
# grep -oE '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' /var/log/nginx/access.log |sort |uniq -c #查看nginx访问IP数
1 101.200.78.64
2 103.41.52.94
1 106.185.47.161
2 113.240.250.155
260 13.0.782.215
2 185.130.5.231
26 192.168.10.16
6 192.168.10.17
148 192.168.10.2
189 192.168.10.202
270 192.168.10.222
25 192.168.10.235
291 192.168.10.3
12 192.168.10.5
2 23.251.63.45
20 7.0.11.0
评论
