Remove Duplicate Numbers

Eight numbers, four survivors

Remove duplicate numbers when a second copy adds nothing. The list already in the box is the worked example.

Eight tokens. Four survive.

1. 4 is new. It stays.
2. 7 is new. It stays.
3. The next 4 is a copy. It leaves.
4. 1.0 converts to 1. That value is new, so 1 stays.
5. The next 7 leaves.
6. 9 stays.
7. 01 converts to 1, already kept, so it leaves.
8. The last 4 leaves.

Kept, in first-sighting order: 4, 7, 1, 9.

The ledger does not print every discarded copy on its own row. It names each repeated value once.

• 4 showed up 3 times, so 2 extras left.
• 7 showed up 2 times, so 1 extra left.
• 1 showed up twice, once written 1.0 and once written 01, so 1 extra left.

Extras add up to 4. Parsed count is 8. Repeat share is 50%.

4, 7, 4, 1.0, 7, 9, 01, 4
kept: 4, 7, 1, 9
extras dropped: 4

That percent is not a grade. It only says how much of the paste was a copy.

The first copy is the only one that survives

Duplicate removal looks like tidying. On some lists it deletes the only interesting fact.

A score of 82 written six times is six students. Drop the repeats and one 82 remains, which is a different class. A log line that prints 71.2 twelve times in a row is a flat stretch. Strip the copies and the stretch is gone. One reading remains, with no sign the sensor sat still.

Use this page when a second copy adds no new fact. Port numbers gathered from three servers. SKU codes pasted from two exports. Invoice IDs before a join, after you have checked the double is a paste error and not two charges.

Leave when the count is the fact. Grades, votes, time series, and any column where "how often" changes the conclusion. The Number Frequency Counter keeps every copy and prints the tally. Sort Numbers lines copies up beside each other, and drops them only if you turn that option on. This page drops them as the main job, then names what left.

Three questions, three pages

If the question is how many times 7 appeared, stop. The drop erases the answer. Open the frequency counter with the original paste.

Spreadsheet UNIQUE also keeps the first row. It does not name the extras. Stay in the sheet when the list already lives there and later formulas need the column. Use this page when you want the dropped values named before a shorter list goes back into a ticket, a config, or a mail.

1, 1.0, and 01 collapse into one value

Comparison runs after each token becomes a number. The conversion is the same one a browser applies when it reads a numeric string. After that step, 1, 1.0, 1.00, 01, and 1e0 are all the number 1. They match. The kept line prints 1. The original spelling is not stored, so the output does not record whether the survivor arrived as 1.0 or as 01.

A thousands comma splits the other way. Every comma is a gap between values. 3,200 becomes 3 and 200. 1,000e3 becomes 1 and 000e3, and 000e3 becomes 0. The exponent never attaches to the 1. Paste 3200 or 1000 when the comma was only grouping digits.

A token has to start with a digit, a dot, or a minus sign. $42.50 fails that test, so the token is ignored. 42kg passes, the letters fall off, and the stored value is 42. -12.5 stays negative. .5 becomes 0.5. 0 and -0 count as one value, so a signed zero after a plain zero is a repeat.

0.3 and 0.30 collapse too. Both read as 0.3. A measurement logged to two decimals and the same measurement logged to one decimal become duplicates. That is right for arithmetic and wrong for a lab notebook that treats trailing precision as part of the record.

There is no switch labeled "keep the text as typed." If the characters around a number matter, strip them in the sheet, then paste bare values here.

Order is a second choice

First sighting keeps survivors in the sequence they arrived. Low to high and high to low reorder only the kept values. Neither option puts dropped copies back, and neither sorts the raw paste.

The ledger follows the same choice. Under first sighting, a repeated value shows up in the order it was first stored. For the sample, that is 4, then 7, then 1. Under low to high, both the kept line and the ledger run from small to large.

Seen

Full count of that value in the paste, including the copy you kept.

Extras dropped

Seen minus one. The copies that left.

Extras dropped, the stat

Sum of the extras column, not the number of ledger rows. Three rows and four removed copies are both correct for the sample. Mixing those two numbers is the usual misread.

A repeat is noise only in some lists

Firewall ports. 443 copied from three host exports is still one port. The shorter list is the one you want in the config.

Test IDs. A run file with the same ID twice usually means a double paste. Trust the shorter list only after the source shows two identical rows, not two cases that happen to share a number.

Catalog codes. An import often wants each product code once. Two rows with the same code and two prices are a conflict. This page never sees the price. Two matching codes become one number. If a code collision is hiding a price change, the drop is the wrong first move. Open the rows in the sheet.

Poll answers, exam scores, lap times, daily close totals. Here the repeat is the measurement. Six votes for 4 are not one vote. Removing them makes the chart look tidy and makes the result false. Frequency is the page for that paste. Number List Statistics is the page when you need the mean of every copy, not the mean of the set.

The largest survivor matches the largest raw value. The smallest does too. The sum does not, and neither does the average or the median. Find Maximum on the kept list agrees with the raw list. An average computed after the drop does not. The Even Odd Filter is a later pass, for splitting survivors by parity once you are sure the copies were noise.

The tab gives out before the arithmetic does

The drop itself is one pass over the list. What slows the page is drawing a long kept line and a long ledger in the browser. A paste in the tens of thousands of values makes the tab sluggish well before a comparison comes out wrong. Split a huge export into chunks, or dedupe in the spreadsheet that already holds the file.

Parsing, the drop, the ledger, and both copy buttons run in the tab. Nothing is uploaded. Still skip values you would not put in a notes app on a shared computer. Salaries, medical readings, student IDs, account numbers. Local execution is not a private room.