New

Our landmark annual report is here.Read the State of Payment Operations 2022

Journal|||November 17, 2022

How to Scale a Ledger, Part II

Image of Matt McNierney
Matt McNierneyEngineering

Note: This post is the second chapter of a broader technical paper on How to Scale a Ledger.

Adding double-entry to financial events

What does it take for a ledger to be immutable and double-entry at the source of financial events? At the most basic level, it means:

  1. Logging money movement in a consistent double-entry data model every time it happens.
  2. Reading from this data model every time money amounts are shown to internal and external customers.

All financial events—from credit card authorizations to crypto on-ramps—can be described with three core objects:

Core objects for financial events
Core objects for financial events

We’ll describe each of these models, how some common financial events map into them, and how they are implemented in the Ledgers API. This section focuses on the core fields and won’t describe every feature available in the Ledgers API. See the documentation for a complete reference.

Account

An Account tracks a sum of money, denominated in a currency. Examples of Accounts include a digital wallet, a company’s operating cash, and loan principal.

Account fields and descriptions
Account fields and descriptions

Money movement between Accounts in a ledger happens instantly. In the real world, however, moving money between bank accounts is always asynchronous. For example, money sent via ACH takes at least a day to show up in a recipient’s bank account. Because real money movement can’t happen instantly, Accounts should be able to report a few different balances:

  • Posted balance: the money that has fully settled in an Account.
  • Pending balance: the money that’s expected to settle in or out of an Account plus the money that’s already settled.
  • Available balance: the money that’s available to send out of an Account. This balance subtracts money that’s expected to leave an Account, but does not include money that’s expected to settle in an Account.
Balance fields and descriptions
Balance fields and descriptions

Not all ledgers model posted, pending, and available balances—those separate balances can be modeled as separate Accounts. We believe that application ledgers should have these concepts built into the core data model, however. You can’t model money movement without them, so they should be simple to use.

Let’s follow an example of a credit card to see how these balances are affected by different actions in the Ledgers API.

Credit card example
Credit card example

We report these three balances separately so that the customer knows how much can be spent from an Account at any given time. Depending on the use case and risk tolerance, an app may choose to use each balance for a different purpose. For example, a customer may be limited by their available balance for transfers out of their Account, but their past-due status for a loan may be optimistically determined by their pending balance.

Entry

Account balances are never directly modified. Instead, balances change as Entries are written to the Account. Entries are an immutable log of balance changes on an Account. All fields on the Entry are immutable, except for discarded_at, which is set when an Entry is replaced by a later Entry.

Let’s follow the same credit card example to see how each action can be represented as an Entry.

1. A credit card starts with a $100 credit limit on the account.

Create new entry

2. A card is swiped to purchase a $10 pizza. This purchase starts out pending on the card account.

$10 pizza entry

3. That night, the purchase settles.

Discard entry 1

4. The card holder initiates a $10 card payment from their bank account.

New entry 3

5. The card payment from the bank account completes.

Replace with new entry_4

6. A hotel places a $50 hold on the card at the beginning of a stay.

Create new entry_5

7. The $50 hold is removed at the end of the stay.

Discard entry_5

Discarding Entries

As we showed through the above example, Entries have a mutable field discarded_at that gets set when they get replaced by a newer Entry. Only pending Entries can be replaced; posted and archived Entries are never modified. Pending Entries get replaced in the following two circumstances:

  1. The pending amount of an Entry needs to change.
  2. The state of the Entry needs to change.

Why introduce this mutability in the API? To see why, it helps to consider the alternative. Instead of discarding Entries, we could create a reversal Entry that undoes the original Entry. In this model, moving an Entry from pending to posted would create two Entries instead of just one. Step 5 above would instead be:

Reverse entry_3

While valid, this approach leads to a messy Account history. We can’t distinguish between debits that were client-initiated and debits that are generated by the ledger as reversals. Listing Entries on an Account doesn’t match our intuition for what Entries actually comprise the current balance of the Account.

Discarding solves this problem by making it easy to see the current Entries (simply exclude any Entries that have discarded_at set), while also not losing any history.

Computing Account balances

Now that all balance changes are logged as Entries, how do we compute Account balances? Here’s where the normal_balance field on Account comes into play. Every Account in a ledger is categorized as debit normal or credit normal. Definitionally, Accounts that represent uses of funds (assets, expenses) are debit normal, and sources of funds (liabilities, equity, revenue) are credit normal.

The balances of credit normal Accounts are increased by credit Entries and decreased by debit Entries; debit normal Account balances are increased by debit Entries and decreased by credit Entries.

Why bother with debits, credits, and Account normality at all? Many ledgers try to avoid complications by using negative numbers to represent debits and positive numbers to represent credits. At first glance, this appears to align better with engineers’ intuitions.

For Modern Treasury’s Ledgers API, we chose to include the credits and debits concepts because, without them, double-entry accounting is messy. Consider a simple flow where a user deposits money into a digital wallet. This flow will affect the company’s cash Account and the user’s wallet Account. Our intuition is that the cash Account will increase (the company got cash from the user), and also the wallet Account will increase (the user now has a positive balance in the digital wallet).

With a positive/negative number approach, both Accounts increasing is not possible. We have to pick one Account to be negative (so that no money is created or destroyed), and it’s not clear which one.

Credits and debits solve this problem. We should debit the cash Account, whose balance increases because it is debit normal. And we should credit the user's wallet Account, whose balance also increases because it is credit normal.

This digital wallet scenario is just one example. For a full primer on double-entry accounting, check out our series on Accounting for Developers.

Here, we’ll focus on how to actually implement a double-entry ledger.

With some simple math, each type of balance (pending, posted, and available) can be calculated from the following 5 fields:

  • posted_debits: Sum of posted debit Entries
  • posted_credits: Sum of posted credit Entries
  • pending_debits: posted_debits plus the sum of non-discarded pending debit Entries
  • pending_credits: posted_credits plus the sum of non-discarded pending credit Entries
  • normal_balance: One of credit or debit, stored on the Account

A ledger database should be optimized to retrieve these 5 fields quickly, and only compute posted balance, pending balance, and available balance upon request.

Each balance type is then computed as follows:

Posted Balance

1case normal_balance
2when "credit"
3  posted_credits - posted_debits
4when "debit"
5  posted_debits - posted_credits
6end

Pending Balance

1case normal_balance
2when "credit"
3  pending_credits - pending_debits
4when "debit"
5  pending_debits - pending_credits
6end

Available Balance

1case normal_balance
2when "credit"
3  posted_credits - pending_debits
4when "debit"
5  posted_debits - pending_credits
6end

Next Steps

This is the second chapter of a broader technical paper with a more comprehensive overview of the importance of solid ledgering and the amount of investment it takes to get right. If you want to learn more, download the paper, or get in touch.

Try Modern Treasury

See how smooth payment operations can be.

Talk to Us
Share

Copied!