How to Inspect Data in the Data Analysis Lifecycle

A simple data-analysis lifecycle may include:

1. Define the business problem
2. Collect or receive the data
3. Inspect the data
4. Clean and prepare the data
5. Analyze the data
6. Visualize the findings
7. Communicate recommendations
8. Validate and monitor the results

Imagine receiving a box of customer application forms.

Before using the information, you would probably check:

• How many forms are in the box?
• Are all the forms using the same format?
• Are important answers missing?
• Has anyone submitted the same form twice?
• Are dates written consistently?
• Do any ages, prices or other values look impossible?

Inspecting a dataset follows the same idea. You examine its structure and condition before trusting it.

Before opening the dataset, clarify what the analysis is expected to accomplish.

This question suggests that the dataset may need:

• A customer identifier
• Subscription type
• Customer tenure
• Monthly charges
• Support activity
• Churn status

A CSV file can be loaded with Pandas:

import pandas as pd

customers = pd.read_csv("customer_data.csv")

The variable customers now contains a Pandas DataFrame. A DataFrame is a table-like structure containing rows and columns.

Display the first five rows:

customers.head()

Display the first ten rows:

customers.head(10)

You can also inspect the last five rows:

customers.tail()

This preview helps you examine:

• Column names
• Example values
• General formatting
• Possible missing values
• Unexpected categories
• Whether the file loaded correctly

One row might represent:

• One customer
• One order
• One product
• One support ticket
• One shipment
• One monthly account record

Suppose one row represents an order. The same customer may correctly appear several times because one customer can place multiple orders. Repeated customer IDs would not automatically be duplicate records.

customers.shape

An output such as (5000, 12) means:

• 5,000 rows
• 12 columns

You can store the values separately:

rows, columns = customers.shape

print("Number of rows:", rows)
print("Number of columns:", columns)

customers.columns

Or display them as a list:

customers.columns.tolist()

Look for:

• Extra spaces
• Unclear abbreviations
• Inconsistent capitalization
• Duplicate column names
• Misspelled names
• Names that do not match the data dictionary

These names may later be standardized, but inspection should document the issue before changing anything.

customers.info()

info() displays:

• Column names
• Number of non-null values
• Data types
• Dataset memory usage

Look for problems such as:

• Monthly charges stored as text
• Dates stored as ordinary strings
• Identifiers incorrectly treated as measures
• Numbers containing currency symbols
• Columns containing both numbers and text

customers.describe()

For numeric columns, this commonly displays:

• Count
• Mean
• Standard deviation
• Minimum
• 25th percentile
• Median
• 75th percentile
• Maximum

Ask:

• Does the minimum age appear reasonable?
• Are any monthly charges negative?
• Is the maximum value unusually high?
• Are the mean and median very different?
• Do the numeric ranges make business sense?

To include categorical columns:

customers.describe(include="all")

Count missing values:

customers.isna().sum()

Calculate missing percentages:

missing_percentage = (
    customers.isna().mean() * 100
).round(2)

missing_percentage

Create a reusable report:

missing_report = pd.DataFrame({
    "Missing_Count": customers.isna().sum(),
    "Missing_Percentage": (
        customers.isna().mean() * 100
    ).round(2)
})

missing_report.sort_values(
    "Missing_Percentage",
    ascending=False
)

Interpret missing values in context:

• Missing age might be manageable.
• Missing customer ID could affect record identification.
• Missing churn status could directly affect churn analysis.
• Missing payment date could affect time-based reporting.

customers.duplicated().sum()

Display every copy of duplicated rows:

customers[
    customers.duplicated(keep=False)
]

Inspect an identifier where appropriate:

customers["Customer_ID"].duplicated().sum()

Before calling a record a duplicate, ask:

• Should this identifier be unique?
• Can one customer have multiple transactions?
• Do repeated records contain different dates or products?
• Are all column values identical?
• Could the repetition represent a valid business event?

customers["Subscription_Type"].value_counts(
    dropna=False
)

customers["Subscription_Type"].unique()

Look for:

• Unexpected categories
• Blank strings
• Misspellings
• Different capitalization
• Leading or trailing spaces
• Very rare categories

Inspect the largest values:

customers.sort_values(
    "Monthly_Charges",
    ascending=False
).head(10)

Inspect the smallest values:

customers.sort_values(
    "Monthly_Charges"
).head(10)

Apply relevant business rules:

customers[
    customers["Age"] < 0
]

customers[
    customers["Monthly_Charges"] < 0
]

A very large value could represent:

• A valuable corporate customer
• A data-entry mistake
• A currency problem
• A duplicated transaction
• A legitimate seasonal event

customers["Join_Date"] = pd.to_datetime(
    customers["Join_Date"],
    errors="coerce"
)

Inspect the date range:

customers["Join_Date"].min()
customers["Join_Date"].max()

Check values that could not be converted:

customers["Join_Date"].isna().sum()

Return to the original business question. For customer-churn analysis, confirm that the dataset contains:

• A customer identifier
• A reliable churn indicator
• Relevant customer characteristics
• Enough records for useful comparisons
• An appropriate time period
• Clearly defined categories

Keeping these stages separate makes the analysis easier to explain, validate and reproduce.

• Looking only at the first five rows
• Cleaning before recording the original condition
• Treating every repeated identifier as a duplicate
• Calculating averages for identifiers
• Assuming every object column is correctly stored
• Ignoring blank strings and placeholder values
• Removing unusual values without investigation
• Inspecting data without understanding the business question
• Assuming that a large dataset must be useful
• Forgetting to document limitations

import pandas as pd

df = pd.read_csv("dataset.csv")

print("First five rows:")
display(df.head())

print("Shape:")
print(df.shape)

print("Columns:")
print(df.columns.tolist())

print("Dataset information:")
df.info()

print("Numeric summary:")
display(df.describe())

print("Missing values:")
display(df.isna().sum())

print("Duplicate rows:")
print(df.duplicated().sum())

After running this starter code, inspect important categories, numeric ranges, dates and identifiers based on the dataset and business question.