Here is an example of the typical simplest approach to filtering out missing values from a pandas DataFrame.

One small detail: In Pandas the missing values (NAs) are represented as NaN (not a number).

import pandas as pd
import numpy as np

df = pd.DataFrame({
    "brand": ["iphone", "samsung", "motorola", "huawei"],
    "price": [1099, 899, np.NaN, np.NaN]
})
df

Here are some potential use cases. You need to make sure you are not removing “good” data.

Filter out all missing values

df.dropna()

Filter NAs based on one column

An alternative approach I generally use is select which column I want to remove missing values for.

In this case, I’m filtering out the rows where df.price has missing values.

df[df.price.notna()]

Finally, another option is using the following, the output is the same.

df[~df.price.isnull()]

I hope you enjoyed this short post about missing values in Python. If you want to learn more, please check out my youtube channel!


Leave a Reply

Your email address will not be published. Required fields are marked *