Prepstellar

Data Engineering Fundamentals · Data Engineering Foundations

22 cards

Analytical Data Modeling

Swipe, scroll or use ← →
  1. Declare the grain first

    Before choosing a single column, decide what one row of the table is going to mean. An analytical table is trustworthy only when the meaning of one row is stable.

    The grain of a fact table states the business event or measurement represented by one row, giving downstream aggregations a stable interpretation. Everything else follows from it: which keys the row needs, which measures may live in it, and which totals are safe to compute.

    Read the grain as a sentence in the language of the business:

    Table One row is… Grain
    Sales one product sold on one date product and date
    Sales target one target for one subcategory in one quarter subcategory and quarter
    Web session one visit by one visitor visitor and session

    A sales fact can be stored at product and date grain while a target fact can use subcategory and quarter grain. Two fact tables in the same model may legitimately disagree about how fine their rows are, as long as each one declares its own grain.

    1 / 22
  2. Declare the grain first

    The grain is a contract with everyone who queries the table. Measures in one fact table must be interpreted at that table's declared grain. Adding up a column of daily product sales gives a product total; adding up a column of quarterly subcategory targets gives a subcategory total. The two numbers are comparable only after they have been brought to a common grain.

    The contract also constrains how the table is joined. Joining a fact to a dimension must preserve the intended one-row meaning rather than multiply facts through an unintended many-to-many match. If a join turns one order line into three rows, the measures triple and every total built on them is wrong — even though no value in the source data changed.

    A useful habit: write the grain down in one sentence before writing the CREATE TABLE. If the sentence needs the word and three times, the grain is probably finer than the business actually needs.

    2 / 22
  3. Quick check

    A sales target table stores exactly one row for each subcategory in each quarter. What is its grain?

    1. AProduct and day, because sales themselves are recorded daily

      Product and day is a finer row meaning that belongs to the sales table, not to this target table.

    2. BSubcategory and quarter

      Right. The grain is the exact combination that one row represents, which here is one subcategory in one quarter.

    3. CCategory and year, once the quarterly rows are rolled up

      Category and year is coarser than what one row actually stores, so it describes a later roll-up rather than the grain.

    3 / 22

  4. Facts, dimensions, and measures

    Dimensional models separate measurable activity from descriptive context, so that a question like revenue by store city has an obvious home for each half.

    A fact table stores business events or observations and commonly contains numeric measures. Rows arrive as things happen: a sale, a shipment, a meter reading. Fact tables are the tall part of the model — millions or billions of rows, few columns.

    A dimension table stores descriptive attributes used to filter, group, and label facts. Rows describe the things the business talks about: products, customers, stores, dates. Dimensions are the wide part of the model — comparatively few rows, potentially hundreds of attributes, because every attribute is a way somebody might want to slice the facts.

    Fact table Dimension table
    A row is an event or measurement an entity being described
    Typical columns dimension keys and numeric measures text attributes and keys
    Typical size very many rows comparatively few rows
    Used in a query for the numbers the filters, the groups, the labels
    4 / 22
  5. Facts, dimensions, and measures

    A shortcut for spotting the dimensions a model needs: listen for the word by. "Revenue by salesperson, by month, by product category" names three dimensions and the attributes they must carry.

    Measures are values such as quantity or amount that analysts aggregate at a chosen dimensional context. They are the things that get summed, averaged, and counted.

    Descriptive attributes such as product category or customer location provide context but are not the measured event itself. They are the things that get grouped and filtered on. A category is never summed; an amount is rarely grouped by.

    5 / 22
  6. Quick check

    A report shows revenue grouped by store city. Which part of the model supplies the city label?

    1. AThe dimension table, which holds descriptive attributes

      Right. Descriptive attributes such as location live in a dimension and are what the report groups and labels by.

    2. BThe fact table, where the revenue rows themselves are stored

      The fact table supplies the revenue values and the keys that reach the store; the descriptive label itself belongs to the dimension.

    3. CThe measure, once it has been aggregated

      Aggregating a measure produces a number, not the descriptive text used to group it.

    6 / 22

  7. Tell a measure from an attribute

    The same table can hold both kinds of column, so the test is behavioral rather than visual: ask whether the value is something you would add up, or something you would group by.

    Field Add it up? Group by it? It is a…
    Sales amount yes rarely measure
    Quantity sold yes rarely measure
    Product category no yes descriptive attribute
    Customer region no yes descriptive attribute
    Calendar month name no yes descriptive attribute

    Numeric does not mean measure. A postal code and a store number are numbers that nobody wants a total of; they label rows, so they behave as attributes. A price, on the other hand, changes often and is genuinely aggregated, which is why a frequently changing numeric attribute is sometimes better stored in the fact table as a measure than in the dimension as text.

    7 / 22
  8. Quick check

    Which of these three fields behaves as a measure rather than as descriptive context?

    1. ACustomer region, which says where the sale happened

      A region locates the event and is used to filter and group; it is never summed.

    2. BProduct category, which groups similar items together

      A category groups similar items and provides context around the numbers.

    3. CSales amount, which analysts add up by region and by category

      Right. An amount is a value that analysts aggregate at a chosen dimensional context, which is exactly what a measure is.

    8 / 22

  9. Keep your progress in the app

    That’s 3 of 8 quick checks. In the app they stay answered, and every lesson remembers where you left off.

  10. Operational shape and analytical shape

    Model shape follows workload shape. The same business data is organized differently depending on whether the job is recording activity or explaining it.

    Normalized operational models reduce redundancy and support consistent transactional updates across related entities. A customer's city is stored once, in one place, so a change of address is a single update that cannot leave two copies disagreeing. That is exactly what an order-entry system needs.

    Dimensional analytical models organize facts and dimensions to make aggregation and business-oriented queries easier. A reporting query does not update anything; it reads enormous numbers of rows and groups them. Spreading the descriptive context across a dozen normalized tables makes every such query pay for another set of joins.

    Normalized operational model Dimensional analytical model
    Optimized for consistent updates aggregation and grouping
    Redundancy minimized accepted where it helps queries
    Typical query touches few rows touches very many rows
    Cost it avoids update anomalies join work
    9 / 22
  11. Operational shape and analytical shape

    Neither shape is wrong; they answer different questions. The reporting layer is normally built from the operational model rather than instead of it, and the source system keeps its normalized design.

    When the reporting priority is frequent aggregation rather than update efficiency, the dimensional star is the better serving design, because it removes the join work that the normalized source imposes on every query. Query authors ask for revenue by customer, product, and date and get it from a fact table and three directly related dimensions.

    The choice is driven by the access pattern, not by taste. If a team reads two columns across a billion rows every morning, the model should make that cheap.

    10 / 22
  12. Quick check

    An operational system already holds every customer detail in normalized tables. Why does the reporting team still build a dimensional model on top of it?

    1. ABecause normalized data becomes invalid once it leaves the source system

      The normalized model remains correct and useful; it is simply optimized for updates rather than for reading many rows at once.

    2. BBecause facts and their dimensions make aggregation direct

      Right. Dimensional organization presents measures and their context so aggregation-oriented queries avoid the join work of the normalized shape.

    3. CBecause a dimensional model removes any need for business keys at all

      Business keys stay important: they tie warehouse rows back to the identity of the source entity.

    11 / 22

  13. Star and snowflake shapes

    A star schema places a central fact table around directly related denormalized dimensions. Draw it and you get the shape it is named after: measures in the middle, context radiating out one join away. Because a dimension is small next to a fact table, storing a product's subcategory and category redundantly on every product row is usually a bargain — the storage cost is small and every query saves a join.

    The one common exception normalizes a hierarchy back out. A snowflake schema normalizes dimension hierarchies into multiple related tables, trading simpler storage maintenance for more joins. A Product / Subcategory / Category chain is the classic example.

    12 / 22
  14. Star and snowflake shapes

    Snowflaking is worth considering in a few specific situations:

    • The dimension is extremely large and storage cost outweighs query speed.
    • Facts exist at different levels: the sales fact sits at product level while the sales target fact sits at subcategory level, and each needs a key to attach to.
    • Historical change has to be tracked at a higher level of the hierarchy.
    Shape Dimension storage Joins per query Fits when
    Star denormalized, one table per dimension fewest most reporting work
    Snowflake normalized across related tables more huge dimensions, mixed-grain facts

    Going further in the other direction — a separate table for every level of every hierarchy — does not simplify anything. It adds joins to precisely the queries the serving layer was built to make easy.

    13 / 22
  15. Quick check

    A product hierarchy is stored as three related tables: Product, Subcategory, and Category. Which shape is that, and what is the trade?

    1. AA star dimension; it removes every join from the query path

      A star keeps the whole hierarchy denormalized inside one dimension table, which is the opposite of this arrangement.

    2. BA fact table; it stores the measures for each category

      These tables describe entities rather than record events, so none of them is a fact table.

    3. CA snowflake dimension; it eases storage maintenance but adds joins

      Right. Normalizing a hierarchy across related tables is the snowflake shape, and it trades simpler storage maintenance for extra joins.

    14 / 22

  16. Business keys and surrogate keys

    Keys connect business identity to analytical history, and the two jobs need two different keys.

    A business key identifies the source-system entity, while a surrogate key identifies a warehouse dimension row. The business key — a customer ID, an employee number — comes from the source and often means something to people. The surrogate key is generated by the warehouse, means nothing on purpose, and is unique to one row of one dimension table.

    Fact rows store dimension keys so each event resolves to the intended descriptive context. The fact carries the surrogate key, not the descriptive text, which is what keeps the fact table narrow and the descriptions in one place.

    Key Assigned by Identifies Stable when the description changes?
    Business key the source system the real-world entity yes, it stays the same
    Surrogate key the warehouse one dimension row no, a new version gets a new one
    15 / 22
  17. Business keys and surrogate keys

    That second column is the whole point. Surrogate keys also let a warehouse consolidate several sources without duplicate identifiers clashing, collapse a multi-column natural key into one narrow integer, and keep the fact table small.

    When descriptive values must retain history, separate dimension versions can share a business key and use different surrogate keys. One customer, one business key, several rows — one per period in which the description was valid. The current row is flagged as current; the older rows keep the values that were true when the older facts happened.

    16 / 22
  18. Quick check

    Why does a warehouse dimension carry a surrogate key in addition to the business key that arrives from the source?

    1. ASo several versions of one entity can be told apart

      Right. Versions of the same entity share the business key, so a warehouse-assigned key is what makes each version individually addressable.

    2. BSo the source system can decide which warehouse version is current

      The warehouse manages its own versions; the source contributes the entity identifier, not the version history.

    3. CSo descriptive attributes can be replaced by numeric measures

      Keys identify rows and entities; they do not turn descriptive context into measures.

    17 / 22

  19. Keep history without multiplying facts

    A star schema preserves historical meaning when each fact points to the dimension version valid for that event. The fact is loaded with the surrogate key of the version that was current at the time, so the description it resolves to is the description that was true then.

    Work through a move. A customer moves from Madrid to Lisbon. Finance needs last year's sales still attributed to Madrid, this year's to Lisbon, and the source customer ID cannot change.

    1. Keep the business key: the source identity is unchanged.
    2. Close the Madrid row by setting its end-of-validity date and clearing its current flag.
    3. Insert a Lisbon row with a new surrogate key, valid from that date, flagged current.
    4. New facts store the Lisbon surrogate key; the old facts keep pointing at the Madrid one.

    Overwriting the single row instead would report every historical sale as Lisbon — history quietly rewritten. Copying both cities onto every fact row abandons the versioned relationship and makes consistent context somebody's manual problem.

    18 / 22
  20. Keep history without multiplying facts

    The failure to watch for is the join that multiplies. Match a fact on the business key alone and it will find every version that ever shared that key: one order line becomes three rows, and the measures triple.

    Response What happens to totals What happens to history
    Join on business key to all versions inflated ambiguous
    Divide the inflated totals by the version count still wrong, the count varies still ambiguous
    Change the fact grain to one row per version the measured event changes meaning not repaired
    Join on the surrogate key of the valid version correct correct

    Only the last row satisfies both requirements at once, because it restores the one-row meaning the grain declared and attaches the description that was valid for that event.

    19 / 22
  21. Quick check

    A join on the business key alone returns three rows for one order line, because the product has three historical versions. What is happening, and what repairs it?

    1. ANothing is wrong; divide the totals afterwards by the number of versions

      The totals are already inflated, and a version count that changes over time cannot be relied on to undo the damage.

    2. BThe measures are multiplied; join instead on the surrogate key of the version valid for that event

      Right. Matching a single version-specific key restores the declared grain and attaches the description that was valid when the event happened.

    3. CThe dimension is broken; redefine the fact grain as one row per dimension version

      Redefining the grain changes what the fact table measures instead of fixing the relationship that multiplied its rows.

    20 / 22

  22. Key takeaways

    • Grain first. The grain of a fact table states the business event or measurement represented by one row, and every measure in that table is interpreted at that grain.
    • Two kinds of table. Facts hold events and numeric measures; dimensions hold the descriptive attributes used to filter, group, and label those facts.
    • Two kinds of column. Amounts and quantities are measures to aggregate; categories, regions, and month names are context to group by.
    • Shape follows workload. Normalized models protect consistent updates; dimensional models make aggregation direct, with the star keeping dimensions denormalized and the snowflake normalizing hierarchies at the cost of extra joins.
    • Two kinds of key. The business key carries source identity, the surrogate key identifies one dimension row, and versions that share a business key with different surrogate keys are what let a star preserve history.
    21 / 22
  23. Quick check

    Which summary separates grain, context, and history correctly?

    1. AGrain fixes what one row means, dimensions supply context, and surrogate keys carry versions

      Right. Those are the three distinct jobs: the grain defines row meaning, dimensions describe, and surrogate-keyed versions preserve historical description.

    2. BGrain counts the dimensions, facts supply the labels, and business keys are dropped on load

      Grain has nothing to do with how many dimensions exist, labels come from dimensions, and business keys are what tie warehouse rows to source identity.

    3. CGrain sets the refresh schedule, measures describe entities, and history stays in the source system

      Refresh cadence is an ingestion concern, measures are aggregated rather than descriptive, and the warehouse is what keeps analytical history.

    22 / 22

  24. 8 quick checks · then the test

    In the app, finishing the quick checks opens this lesson’s 10-question test, and the ones you miss come back exactly when you’re about to forget them.

The whole course, on your phone

Lessons you can read, audio you can listen to on the way to work, and practice that remembers what you got wrong.