Python OCR Pipeline for Legacy Lease PDFs (ASC 842 / IFRS 16)

Problem Statement #Link to this section

Legacy lease agreements almost always arrive as scanned, non-searchable PDFs — skewed, low-contrast, multi-column pages where a payment schedule is just a grid of pixels. This page answers one precise engineering-plus-compliance question: how do you preprocess a scanned lease image so that Tesseract recognises the rent-schedule table as structured rows and columns rather than a scrambled block of numbers, and every recognised token carries a confidence score and a bounding box the auditor can trace back to the source scan? The stakes are measurement-level: the commencement date, discount rate, payment frequency, and escalation terms your OCR pass emits become the direct inputs to the right-of-use asset and lease liability. A digit misread inside an amount cell silently corrupts the present value; a paragraph of legal boilerplate that bleeds into a payment column corrupts the whole schedule. OCR is the first-mile stage inside the broader lease document extraction and clause parsing pipelines, and this page owns the narrow slice before any semantic parsing happens: geometric stabilization, layout segmentation, and confidence-gated token capture.

The stages are order-dependent: geometry is stabilized (deskew → threshold → close) before layout segmentation isolates the schedule grid, so only the financial region reaches Tesseract. The confidence gate is the control point — tokens at or above 0.85 flow forward to feed the liability, while anything below is quarantined by bounding box for human review rather than forwarded into the measurement.

Standard Anchor #Link to this section

OCR is not itself a measurement rule, but its output is the evidence the measurement rules test against, so two provisions govern what the pipeline must preserve. ASC 842-10-30-5 requires the lease liability to be measured at the present value of the lease payments not yet paid — which means the pipeline must recover every fixed and in-substance fixed amount, in the correct period, or the summation is understated. IFRS 16.27 carries the same requirement for the initial measurement of the liability. Both standards distinguish fixed payments (included) from usage-based variable payments (expensed as incurred), so the OCR layer must keep amount cells geometrically anchored to their period labels — the row/column relationship is the fixed-payment-to-period mapping the standard depends on. Preserving that structure is why table segmentation runs before token recognition rather than after.

Algorithm Specification #Link to this section

The preprocessing sequence is deterministic and order-dependent. Let a page image be $I$. Deskew rotates by the dominant text-baseline angle $\theta$ estimated from a Hough line transform:

$$\theta =\underset{k}{\mathrm{median}}\left(\arctan \frac{y_{2,k}-y_{1,k}}{x_{2,k}-x_{1,k}}\right),I^{\prime }=R_{-\theta }I$$

where $R_{-\theta }$ is the affine rotation that drives $|\theta |$ below the $0.5^{\circ }$ target. Binarization then uses an adaptive Gaussian threshold rather than a single global cut, computing a local threshold $T(x,y)$ per pixel from its neighbourhood mean ${\mu }_w(x,y)$ over a window of width $w$:

$$T(x,y)={\mu }_w(x,y)-C,B(x,y)=\{\begin{array}{ll}1& I^{\prime }(x,y)>T(x,y)\\ 0& \text{otherwise}\end{array}$$

Here $C$ is a small bias constant that keeps faint schedule rules from dissolving into a darkened paper background. A morphological closing then bridges broken table borders before layout segmentation isolates the schedule region. Each OCR token $t$ is emitted as a triple $({\text{text}}_t,c_t,{\text{bbox}}_t)$ with confidence $c_t\in [0,1]$, and any token with $c_t<0.85$ is quarantined for human review rather than forwarded.

Annotated Python Snippet #Link to this section

The example below runs the full deterministic sequence on one page and returns only high-confidence tokens, routing the rest to review. It uses opencv-python, pytesseract, and Decimal where amounts are later parsed.

from dataclasses import dataclass
from decimal import Decimal
import cv2
import numpy as np
import pytesseract

CONFIDENCE_FLOOR = Decimal("0.85")  # ASC 842-10-30-5 / IFRS 16.27: no ambiguous amount forwarded


@dataclass(frozen=True)
class Token:
    text: str
    confidence: Decimal
    bbox: tuple[int, int, int, int]  # (x, y, w, h) in page pixels for audit traceback


def deskew(gray: np.ndarray) -> np.ndarray:
    """Align text baselines to within 0.5 degrees using the dominant Hough angle."""
    edges = cv2.Canny(gray, 50, 150)
    lines = cv2.HoughLinesP(edges, 1, np.pi / 180, threshold=200, minLineLength=200, maxLineGap=20)
    if lines is None:
        return gray
    angles = [np.degrees(np.arctan2(y2 - y1, x2 - x1)) for x1, y1, x2, y2 in lines[:, 0]]
    theta = float(np.median([a for a in angles if abs(a) < 45]))  # ignore vertical rules
    h, w = gray.shape
    rot = cv2.getRotationMatrix2D((w / 2, h / 2), theta, 1.0)
    return cv2.warpAffine(gray, rot, (w, h), flags=cv2.INTER_CUBIC, borderValue=255)


def preprocess(page_bgr: np.ndarray) -> np.ndarray:
    """Deterministic geometric stabilization: deskew -> adaptive threshold -> close broken rules."""
    gray = cv2.cvtColor(page_bgr, cv2.COLOR_BGR2GRAY)
    gray = deskew(gray)
    binary = cv2.adaptiveThreshold(
        gray, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY, blockSize=31, C=10
    )
    kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (2, 2))
    return cv2.morphologyEx(binary, cv2.MORPH_CLOSE, kernel)  # bridge broken table borders


def ocr_page(page_bgr: np.ndarray) -> tuple[list[Token], list[Token]]:
    """Return (clean_tokens, review_tokens) split on the confidence floor."""
    clean = cv2.bitwise_not(preprocess(page_bgr))  # Tesseract expects dark text on light
    data = pytesseract.image_to_data(clean, output_type=pytesseract.Output.DICT)
    clean_tokens, review_tokens = [], []
    for i, word in enumerate(data["text"]):
        if not word.strip() or int(data["conf"][i]) < 0:
            continue
        tok = Token(
            text=word,
            confidence=(Decimal(int(data["conf"][i])) / Decimal(100)),
            bbox=(data["left"][i], data["top"][i], data["width"][i], data["height"][i]),
        )
        (clean_tokens if tok.confidence >= CONFIDENCE_FLOOR else review_tokens).append(tok)
    return clean_tokens, review_tokens


# --- terminal test: a synthetic page with a printed "1,250.00" must survive as one clean token ---
if __name__ == "__main__":
    canvas = np.full((120, 400, 3), 255, dtype=np.uint8)
    cv2.putText(canvas, "1,250.00", (30, 80), cv2.FONT_HERSHEY_SIMPLEX, 1.4, (0, 0, 0), 3)
    clean, review = ocr_page(canvas)
    recovered = "".join(t.text for t in clean)
    assert Decimal(recovered.replace(",", "")) == Decimal("1250.00"), recovered
    assert all(t.confidence >= CONFIDENCE_FLOOR for t in clean)
    print("OK: amount recovered as", recovered)

Correct vs Naive Preprocessing #Link to this section

The single choice that decides whether a scanned schedule survives is global versus adaptive thresholding — the naive path is the most common reason payment cells vanish before Tesseract ever sees them.

Gotcha: Deskew Rotates the Vertical Table Rules Into the Angle Estimate #Link to this section

The failure mode that ruins otherwise-clean pages: a lease schedule has strong vertical column rules, and if you feed every Hough line into the angle median, those near-90° lines dominate and the page gets rotated by tens of degrees. The recognizer then reads garbage and every amount lands in the review queue.

Before — every detected line contributes:

angles = [np.degrees(np.arctan2(y2 - y1, x2 - x1)) for x1, y1, x2, y2 in lines[:, 0]]
theta = float(np.median(angles))  # vertical column rules pull theta toward 90 degrees

After — restrict the estimate to near-horizontal text baselines:

angles = [np.degrees(np.arctan2(y2 - y1, x2 - x1)) for x1, y1, x2, y2 in lines[:, 0]]
theta = float(np.median([a for a in angles if abs(a) < 45]))  # keep only baseline candidates

A companion debug checklist for a page that OCRs poorly:

1. Save the post-deskew image and confirm theta is within a few degrees of zero — a double-digit angle means vertical rules leaked into the estimate.
2. Save the post-adaptiveThreshold image and confirm the schedule grid lines are still visible; if cells have merged, lower C or shrink blockSize.
3. Confirm you inverted (bitwise_not) before Tesseract — dark-on-light is required, and a white-on-black image returns zero tokens.
4. Inspect the review queue: a group of low-confidence tokens sharing a bounding-box band usually means one skew or threshold defect, not many independent misreads.

Frequently Asked Questions #Link to this section

Why segment the table layout before OCR instead of reflowing the text afterward?

Once Tesseract flattens a page into a single reading order, the geometric link between an amount and its period label is gone, and no amount of post-processing reliably rebuilds which figure belonged to which row. Isolating the schedule region first — then OCRing it as a grid — preserves the row/column relationship that is the fixed-payment-to-period mapping ASC 842-10-30-5 and IFRS 16.27 measure against.

Why parse recovered amounts as Decimal rather than float?

A scanned schedule can run for hundreds of monthly periods, and binary floating point accumulates representation error that shows up as sub-cent drift in the present-value summation and the amortization roll-forward. Parsing every recovered amount into Decimal with explicit rounding keeps figures cent-exact and reconcilable back to the source scan, which is what an auditor tests.

What confidence threshold should gate the human-review queue?

A 0.85 floor is a defensible default for lease amounts, but it is a policy control, not a law — tighten it for high-value or near-expiry contracts. The important discipline is that any token below the floor is quarantined by its bounding box and blocked from the calculation until a reviewer resolves it, rather than propagating an ambiguous amount into the liability.

Does adaptive thresholding replace image super-resolution for very low-DPI scans?

No. Adaptive thresholding fixes uneven contrast, but a genuinely low-resolution scan (below roughly 200 DPI for body text) lacks the pixels Tesseract needs regardless of thresholding. For those pages, upscale before binarization or route to a higher-capacity vision model; thresholding cannot recover information the scan never captured.

Related #Link to this section

• Sibling: Extracting renewal options with spaCy and regex — the semantic stage that consumes these clean tokens to build the lease term.
• Parent: PDF/DOCX Lease Ingestion Workflows — the routing layer that hands scanned pages to this OCR pipeline.
• Section: Lease Document Extraction & Clause Parsing Pipelines — ingestion through normalization and sync into the measurement chain.