Text Sorter
Order any list A-Z, Z-A, by number, by line length, or simply reverse it.
Text Tools
Use tool
Remove Duplicates from a List of Lines
Strip repeated lines from a list and keep only the unique ones.
Remove Duplicate Lines tool
0
Original lines
0
Unique lines
0
Duplicates removed
About the Remove Duplicate Lines
Remove Duplicate Lines takes a list where some entries appear more than once and returns a version where each entry appears exactly once. The first occurrence of every line is kept in its original position, so the order of your list is preserved rather than sorted.
Two options change how matching works. Case-insensitive matching treats Apple and apple as the same entry, which is usually what you want for email addresses, tags and keyword lists. Ignoring surrounding whitespace means that a line with a trailing space still matches the same line without one, a common problem in lists copied out of spreadsheets.
After processing, the tool reports how many lines went in, how many unique lines came out and how many duplicates were removed, so you can sanity-check the result before using it.
How to use the Remove Duplicate Lines
- Paste your listPut one entry per line into the input box.
- Pick your matching rulesDecide whether case and surrounding spaces should matter, and whether empty lines should be dropped.
- Press Remove duplicatesThe deduplicated list appears below along with a count of what changed.
- Copy or downloadTake the clean list with the copy button, or save it as a .txt file.
Features
- Keeps the first occurrence and preserves original order
- Case-sensitive or case-insensitive matching
- Optional trimming of leading and trailing spaces
- Optional removal of blank lines
- Reports original, unique and removed counts
- Copy and download the cleaned list
Who uses this tool
The remove duplicate lines is used by writers, developers, students, marketers and anyone else who needs the job done once without installing software. Common cases include:
- Cleaning an email or subscriber list before import
- Deduplicating keyword lists for SEO research
- Tidying URL lists collected from a crawl
- Removing repeated rows pasted out of several spreadsheets
- Preparing a unique list of IDs or SKUs
Frequently asked questions
Which copy of a duplicate line is kept?
The first one. Later copies are removed, so the surviving entry keeps its original position in the list. This matters when your list is already in a meaningful order.
Does the tool sort my list?
No. Order is preserved exactly. If you also want the result sorted, run it through the Text Sorter afterwards.
What does case-insensitive matching change?
With it switched on, lines that differ only in capitalisation are treated as the same entry, so only one survives. The version that is kept is the first one as it was typed, with its original capitalisation intact.
Are lines with trailing spaces treated as duplicates?
Only if you enable the trim option. Without trimming, a trailing space makes a line technically different, which is why near-identical entries sometimes survive deduplication elsewhere.
Is there a limit on list size?
There is no hard limit. Lists of several hundred thousand lines process in well under a second because the work happens locally in your browser.
Does my list get uploaded anywhere?
No. Processing is done in the page itself, which is why the tool is safe to use with customer lists and other data you cannot send to a third party.
Related tools
If the remove duplicate lines is not quite what you need, these other free tools solve closely related problems.
Text Cleaner
Tidy messy pasted text: extra spaces, empty lines, breaks and special characters.
Text Tools
Use tool
Word Counter
Live word, character, sentence and paragraph counts with a reading-time estimate.
Text Tools
Use tool
Text Reverser
Flip text backwards at character, word or line level in one click.
Text Tools
Use tool
Character Counter
Character, word, line and paragraph counts that update as you type.
Text Tools
Use tool