Remove Duplicate Lines
Remove duplicate lines while preserving order, with case-sensitive and whitespace-aware comparison and a report of what was removed.
How it works
- Paste or type a list of lines.
- Choose how lines are compared: case-sensitivity, whitespace handling, and whether to keep the first or last occurrence.
- The deduplicated output appears live as you type, with a summary of how many lines were removed and which values repeated.
Frequently Asked Questions
Does it keep the first or the last occurrence?
The first, by default, and the original order of the survivors is preserved. There is an option to keep the last occurrence instead, which still restores the original ordering rather than reversing it.
Does case-insensitive matching change my text?
No. The comparison ignores case, but the line written out is the original text exactly as you typed it. The same applies to the trim and whitespace-normalise options: they affect only how lines are COMPARED, never how the survivor is written. Rewriting the survivor would be a silent data change you did not ask for.
Can I see which lines were duplicated?
Yes. The result lists each duplicated value with how many times it appeared, sorted by frequency, so you can check the removals were the ones you expected.
Does it sort the output?
No. Ordering is untouched, which is what distinguishes this from a sort-and-unique pipeline. Use the sort-lines tool if you want both.