Duplicate Line Remover
Remove repeated lines from a list, with options to ignore case, trim and sort
Runs entirely in your browser. Nothing you type or upload is sent to us or stored.
Results are for general guidance. Please check anything important before relying on it. Terms of use
How to Use
- Paste your list into Your list or text, one item per line. Try a sample list loads a few repeated email addresses.
- Pick the options you need: Ignore case treats “Pune” and “pune” as the same line, Trim spaces ignores spaces at the start and end, and Remove empty lines drops blank lines.
- Choose whether to keep the First copy or the Last copy of a repeated line, and whether to keep the original order or sort A to Z or Z to A.
- Check the counts: lines in, unique lines and lines removed. The Most repeated lines table shows which entries appeared more than once.
- Copy the Result, save it as a .txt file, or click Use as input to work on the cleaned list again.
Remove duplicate lines from any list
A duplicate line remover takes a list with repeats and gives back each line once. It's handy for the lists that pile up in everyday work: phone numbers collected from three WhatsApp groups, email IDs merged from two Google Forms, roll numbers copied from different sections, GSTINs of your suppliers, or keywords for a blog post. Paste the list, and the duplicates are gone as you type. Nothing is uploaded; the work happens in your browser.
How the tool decides two lines are the same
By default two lines are duplicates only if they match exactly after spaces at the start and end are trimmed. Three options change that:
| Option | What it does | Example |
|---|---|---|
| Ignore case | Capital and small letters count as the same | “Rahul@Example.com” and “rahul@example.com” become one line |
| Trim spaces | Spaces and tabs at either end are ignored and removed | “ 98XXXXXXXX” and “98XXXXXXXX ” become one line |
| Remove empty lines | Blank lines are dropped instead of being kept once | A list with gaps comes out as one solid block |
Spaces in the middle of a line still matter. “Ravi Kumar” and “Ravi Kumar” (two spaces) are different lines. If your data has that problem, run it through a remove-extra-spaces tool first.
First copy or last copy?
Keeping the first copy is what most people want: the list stays in its original order, and each item sits where it first appeared. Keeping the last copy is useful when the order tells you when something happened. In a list of visitors in the order they signed in, keeping the last copy puts each person where they signed in most recently.
When you keep the last copy with “Ignore case” on, the result also uses the capitals of the last copy. For example “Mumbai” followed later by “MUMBAI” gives “MUMBAI”.
A worked example
Suppose you paste these eight lines, with “Ignore case”, “Trim spaces” and “Remove empty lines” on:
priya@example.com
Rahul@example.com
amit@example.com
(blank line)
priya@example.com
sneha@example.com
amit@example.com
The tool reports 8 lines in, 4 unique lines and 4 lines removed: three duplicates plus one empty line. The result is rahul, priya, amit and sneha, in the order they first appeared. Turn off “Ignore case” and “Rahul@example.com” stays as a separate line, because email addresses are often typed with a capital letter by mistake. For email lists, leaving “Ignore case” on is usually right; for codes where case matters (some coupon codes, passwords, file names on Linux), turn it off.
Sorting the result
You can keep the original order or sort A to Z or Z to A. The sort is “natural”, so numbers inside lines are compared by value: “Class 2” comes before “Class 10”, which a plain alphabetical sort gets wrong. Capitals don't affect the order.
Tips
- Phone numbers written as “+91 98XXX XXXXX” and “98XXXXXXXX” are different text, so they won't be caught as duplicates. Make the format consistent first (Find and Replace can strip “+91” and spaces).
- The tool works on whole lines. To find repeated words inside a paragraph, use a word frequency counter instead.
- Long lists are fine: a list of a lakh lines takes only a moment.
- Spreadsheet users: copy one column from Excel or Google Sheets, clean it here, and paste it back. Each cell becomes one line.