How to Remove Duplicates from a List in Python

Published Oct 25, 2020  ∙  Updated May 2, 2022

We want to remove duplicates from a list, or in other words, keep only unique values in our list.

``````duplicates = [0, 0, 0, 1, 1, 1, 2, 2, 2]
``````

Using a `for` Loop

The brute force, inefficient way to remove duplicates is using a `for` loop, which might look something like this.

``````unique = []
for num in duplicates:
if num not in unique:
unique.append(num)
# [0, 1, 2]
``````

Using `set()`

We can avoid this loop by simply converting the `duplicates` list into a set.

By definition, a set will only contain unique items.

We can then convert the set back into a list if needed.

``````unique = list(set(duplicates))
# [0, 1, 2]
``````

The issue here is that a set is unordered, so the original list order is not guaranteed after the set-to-list conversion.

So, how can we preserve order?

Using `dict.fromkeys()` to Preserve Order

`dict.fromkeys(list)` will return a dictionary with the list items as keys. Dictionaries do not allow duplicate keys, so the returned dictionary will remove the duplicates for us while preserving order.

``````dict.fromkeys(duplicates)
# {0: None, 1: None, 2: None}
``````

We can then convert back to a list.

``````unique = list(dict.fromkeys(duplicates))
# [0, 1, 2]
``````