Initial Data Inspection

Preview the boundaries first

The point of a first look is to learn what you are holding without paying to render all of it. A ten-million-row table printed in full tells you nothing you could not learn from twenty rows, and it costs you the notebook.

Begin with a bounded preview rather than rendering an entire object. head() returns the first observations and tail() returns the last observations. Both methods display five elements by default, and each accepts a custom count.

The two boundaries hide different problems, which is why they are worth checking separately.

The start of a file is where a stray header row, a units line, or a leftover comment tends to survive the import as if it were data. The end is where totals rows, footers and truncated final records show up. Neither shows in the other view.

Use both views when the first and final records require inspection. On a very large import, head(n) and tail(n) with small explicit counts give you both boundaries for the price of a few rows — the output stays bounded no matter how big the table is.

Summarize the numeric variables

A preview shows you rows; a summary shows you distributions. Use describe() to compute a compact statistical summary for a Series or DataFrame columns, excluding missing values.

For numeric data, the default summary includes count, mean, sample standard deviation, minimum, the twenty-fifth, fiftieth, and seventy-fifth percentiles, and maximum.

The fiftieth percentile is the median, and the median is included even when a custom percentile list is supplied — so asking for your own percentiles adds to that output rather than replacing the center of it.

Summarize the nonnumeric variables

A mean has no meaning for a status code or a country name, so the summary changes shape for these columns. For a nonnumeric Series, describe() reports the nonmissing count, number of unique values, most frequent value, and its frequency.

This is a cardinality report, and it answers different questions: is this column a small set of categories or effectively an identifier? Is one value swallowing the whole column? Those are the failures worth catching early in text data, and no numeric moment would reveal them.

Decide which columns the summary covers

A mixed-type DataFrame needs an explicit scope decision, and the default is narrower than most people expect. By default, describe() on a mixed-type DataFrame restricts the result to numeric columns. Text columns are simply absent from the output — not empty, absent.

Passing include=["object"] requests object columns, include=["number"] requests numeric columns, and include="all" includes both available families. So a report that must cover numeric amounts and text statuses needs include="all"; the plain call silently leaves the statuses out.

One more habit belongs to reading the output rather than requesting it. Descriptive aggregation methods generally skip missing data by default; setting skipna=False for methods that accept it changes that choice. Therefore, the count in a summary is evidence about nonmissing observations, not automatically the total row count.

That distinction is a cheap and effective missing-data check:

• If count matches the number of rows, the column has no missing values.
• If it falls short, the gap is the count of missing values in that column.
• Two columns with different counts have different amounts of missing data, whatever the row total says.

Read that way, describe() does double duty: it characterizes the distribution and it quantifies what is absent.

Order by labels or by values

Sorting makes the inspection order explicit, so that what you look at is a decision rather than an accident of how the file arrived. Two methods do it, and they sort different things.

sort_index() sorts by index labels, and on a DataFrame axis=1 sorts column labels. sort_values() sorts a Series by its values or a DataFrame by values named in by.

Both label and value sorting can be descending with ascending=False.

A list passed to by defines multiple sort keys in sequence. This is precedence, not a set: by=["team", "score"] orders by team first, and only settles ties inside a team by score. Reversing the list gives a genuinely different table.

Choosing the wrong method is a common early mistake, because both produce an order and neither raises an error:

• Sorting labels tells you nothing about which observations are largest.
• Sorting values scrambles the label order you may have relied on for a previous comparison.

Sorting also makes a run reproducible. Two people who sort the same table the same way see the same rows in the same places, which matters as soon as anomalies get reported by position.

Decide where missing values go

Missing-value placement is a separate decision, and leaving it implicit is how missing rows go unnoticed at the bottom of an output. sort_values() places missing values last by default and accepts na_position="first" to place them first.

Both choices are defensible; what matters is that you make one. If the purpose of the sort is to find incomplete records, put them first, where a bounded head() will show them. If the purpose is to read the extremes of a distribution, leaving them last keeps them out of the way. Note that the rows are always in the result either way — sorting places missing values, it never removes them.

Compare on transformed values with a key

Sometimes the order you want is not the order the raw values give. Team, team and TEAM sort into three separate groups if the comparison is literal, which is rarely what a reviewer means.

Sorting also accepts a key callable that transforms the values used for comparison while leaving the returned values in their original representation. A string-lowering key can provide case-insensitive ordering: the comparison sees lowercase text, and the sorted output still shows the names as they were stored.

The contract is what keeps this safe. On a DataFrame, that key is applied to each sort column separately and must return an equally shaped Series or array.

• It receives one column at a time, so it does not need to know about the other sort keys.
• It must return one comparison value per input value — same shape in, same shape out.
• Returning a scalar, a differently sized list, or an extra column breaks the one-to-one pairing the sort depends on.

Put the three tools together and an inspection pass is complete: preview both boundaries, summarize the dtype families you actually care about, and order by the labels or values that make anomalies and later operations reproducible. A review that must order by team then score, compare names without case sensitivity, and surface incomplete records first is a single call with by, key and na_position="first" set together.

Key takeaways

• Use head() and tail() for bounded first-and-last previews — five elements by default, any count you ask for, and both boundaries when a file may be damaged at either end.
• Read describe() for what it is: count, mean, sample standard deviation, minimum, quartiles and maximum for numeric data, and count, unique, top and frequency for nonnumeric data, with missing values excluded.
• Choose the dtype families included in describe() when a DataFrame is mixed — the default is numeric only, and include="all" covers both available families.
• Distinguish sorting labels with sort_index() from sorting observations by values with sort_values(), where a list in by sets the key order and ascending=False reverses either kind of sort.
• Set missing-value placement explicitly when it matters to inspection: last by default, first with na_position="first", and use a key callable when the comparison should run on transformed values.