1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
|
uniq(P) uniq(P)
NAME
uniq - report or filter out repeated lines in a file
SYNOPSIS
uniq [-c|-d|-u][-f fields][-s char][input_file [out-
put_file]]
DESCRIPTION
The uniq utility shall read an input file comparing
adjacent lines, and write one copy of each input line on
the output. The second and succeeding copies of
repeated adjacent input lines shall not be written.
Repeated lines in the input shall not be detected if
they are not adjacent.
OPTIONS
The uniq utility shall conform to the Base Definitions
volume of IEEE Std 1003.1-2001, Section 12.2, Utility
Syntax Guidelines.
The following options shall be supported:
-c Precede each output line with a count of the num-
ber of times the line occurred in the input.
-d Suppress the writing of lines that are not
repeated in the input.
-f fields
Ignore the first fields fields on each input line
when doing comparisons, where fields is a posi-
tive decimal integer. A field is the maximal
string matched by the basic regular expression:
[[:blank:]]*[^[:blank:]]*
If the fields option-argument specifies more fields than
appear on an input line, a null string shall be used for
comparison.
-s chars
Ignore the first chars characters when doing com-
parisons, where chars shall be a positive decimal
integer. If specified in conjunction with the -f
option, the first chars characters after the
first fields fields shall be ignored. If the
chars option-argument specifies more characters
than remain on an input line, a null string shall
be used for comparison.
-u Suppress the writing of lines that are repeated
in the input.
OPERANDS
The following operands shall be supported:
input_file
A pathname of the input file. If the input_file
operand is not specified, or if the input_file is
'-' , the standard input shall be used.
output_file
A pathname of the output file. If the output_file
operand is not specified, the standard output
shall be used. The results are unspecified if the
file named by output_file is the file named by
input_file.
STDIN
The standard input shall be used only if no input_file
operand is specified or if input_file is '-' . See the
INPUT FILES section.
INPUT FILES
The input file shall be a text file.
ENVIRONMENT VARIABLES
The following environment variables shall affect the
execution of uniq:
LANG Provide a default value for the internationaliza-
tion variables that are unset or null. (See the
Base Definitions volume of IEEE Std 1003.1-2001,
Section 8.2, Internationalization Variables for
the precedence of internationalization variables
used to determine the values of locale cate-
gories.)
LC_ALL If set to a non-empty string value, override the
values of all the other internationalization
variables.
LC_COLLATE
Determine the locale for ordering rules.
LC_CTYPE
Determine the locale for the interpretation of
sequences of bytes of text data as characters
(for example, single-byte as opposed to multi-
byte characters in arguments and input files) and
which characters constitute a <blank> in the cur-
rent locale.
LC_MESSAGES
Determine the locale that should be used to
affect the format and contents of diagnostic mes-
sages written to standard error.
NLSPATH
Determine the location of message catalogs for
the processing of LC_MESSAGES .
ASYNCHRONOUS EVENTS
Default.
STDOUT
The standard output shall be used only if no output_file
operand is specified. See the OUTPUT FILES section.
STDERR
The standard error shall be used only for diagnostic
messages.
OUTPUT FILES
If the -c option is specified, the output file shall be
empty or each line shall be of the form:
"%d %s", <number of duplicates>, <line>
otherwise, the output file shall be empty or each line
shall be of the form:
"%s", <line>
EXTENDED DESCRIPTION
None.
EXIT STATUS
The following exit values shall be returned:
0 The utility executed successfully.
>0 An error occurred.
CONSEQUENCES OF ERRORS
Default.
The following sections are informative.
APPLICATION USAGE
The sort utility can be used to cause repeated lines to
be adjacent in the input file.
EXAMPLES
The following input file data (but flushed left) was
used for a test series on uniq:
#01 foo0 bar0 foo1 bar1
#02 bar0 foo1 bar1 foo1
#03 foo0 bar0 foo1 bar1
#04
#05 foo0 bar0 foo1 bar1
#06 foo0 bar0 foo1 bar1
#07 bar0 foo1 bar1 foo0
What follows is a series of test invocations of the uniq
utility that use a mixture of uniq options against the
input file data. These tests verify the meaning of adja-
cent. The uniq utility views the input data as a
sequence of strings delimited by '\n' . Accordingly, for
the fieldsth member of the sequence, uniq interprets
unique or repeated adjacent lines strictly relative to
the fields+1th member.
This first example tests the line counting option, com-
paring each line of the input file data starting from
the second field:
uniq -c -f 1 uniq_0I.t
1 #01 foo0 bar0 foo1 bar1
1 #02 bar0 foo1 bar1 foo0
1 #03 foo0 bar0 foo1 bar1
1 #04
2 #05 foo0 bar0 foo1 bar1
1 #07 bar0 foo1 bar1 foo0
The number '2' , prefixing the fifth line of output,
signifies that the uniq utility detected a pair of
repeated lines. Given the input data, this can only be
true when uniq is run using the -f 1 option (which shall
cause uniq to ignore the first field on each input
line).
The second example tests the option to suppress unique
lines, comparing each line of the input file data start-
ing from the second field:
uniq -d -f 1 uniq_0I.t
#05 foo0 bar0 foo1 bar1
This test suppresses repeated lines, comparing each line
of the input file data starting from the second field:
uniq -u -f 1 uniq_0I.t
#01 foo0 bar0 foo1 bar1
#02 bar0 foo1 bar1 foo1
#03 foo0 bar0 foo1 bar1
#04
#07 bar0 foo1 bar1 foo0
This suppresses unique lines, comparing each line of the
input file data starting from the third character:
uniq -d -s 2 uniq_0I.t
In the last example, the uniq utility found no input
matching the above criteria.
RATIONALE
Some historical implementations have limited lines to be
1080 bytes in length, which does not meet the implied
{LINE_MAX} limit.
FUTURE DIRECTIONS
None.
SEE ALSO
comm , sort
COPYRIGHT
Portions of this text are reprinted and reproduced in
electronic form from IEEE Std 1003.1, 2003 Edition,
Standard for Information Technology -- Portable Operat-
ing System Interface (POSIX), The Open Group Base Speci-
fications Issue 6, Copyright (C) 2001-2003 by the Insti-
tute of Electrical and Electronics Engineers, Inc and
The Open Group. In the event of any discrepancy between
this version and the original IEEE and The Open Group
Standard, the original IEEE and The Open Group Standard
is the referee document. The original Standard can be
obtained online at http://www.open-
group.org/unix/online.html .
POSIX 2003 uniq(P)
|
