🧹 Remove Duplicate Lines
Paste a list with one item per line and get only the unique lines back. Great for cleaning up spreadsheet columns, email lists and keyword lists.
How to remove duplicate lines
- Paste your list with one item per line. Copying a column from Excel or Google Sheets gives you exactly that.
- Choose what counts as "the same". By default case and leading/trailing spaces are ignored, so
Appleandappleare treated as one item. - Optionally sort the result, then copy it and paste it back into your spreadsheet. The number of removed lines appears below the output.
Matching options
| Option | Treated as the same | When to use it |
|---|---|---|
| Ignore case | Apple = apple | Email addresses, keywords, domains |
| Ignore leading/trailing spaces | banana = banana | Data with stray spaces from copy and paste |
| Ignore all spaces | (555) 123 4567 = (555)1234567 | Phone numbers, codes and IDs where spacing doesn't matter |
| Remove empty lines | Blank lines are dropped | Columns with empty rows in between |
When a space-ignoring option is on, the output also uses the trimmed value. Turn both space options off to keep lines exactly as they were. When an item appears several times, the first occurrence is kept by default; "Keep the last occurrence" keeps the latest one instead, at its position.
What to show
- Unique lines — every item once. Same as SQL
SELECT DISTINCT, Excel's "Remove Duplicates" or thesort -ucommand. - Duplicates only — only items that appear two or more times. Useful for finding double sign-ups or repeated survey responses.
- Lines that appear once — items that occur exactly once. Combine two lists and use this to find items that are in only one of them.
- Prefix each line with its count adds the number of occurrences and a tab. Paste into a spreadsheet to get two columns, and sort by "Most frequent" for an instant frequency table.
Sorting
A → Z uses natural sorting, so item2 comes before item10 and accented letters sort next to their plain versions. "By length" puts the shortest lines first and sorts lines of equal length alphabetically. "Most frequent" puts the most repeated items at the top.
Common uses
- Email and contact lists — merge attendee lists from several events and count the real number of people. Email addresses are not case-sensitive, so keep "Ignore case" on.
- Keyword and hashtag lists — combine keyword research exports and remove overlaps; "Most frequent" + counts shows which terms came up most.
- Logs and code — group identical error messages, or find repeated imports and dependencies.
Lines are compared as a whole. To dedupe items within one comma-separated line, put each item on its own line first. If you paste several spreadsheet columns, the entire row (including tabs) must match to count as a duplicate. Your data stays in your browser.