Why LLM chat summaries fail credit committee and audit review

A general-purpose chat model can read a borrower's statements and write a fluent memo in seconds. The memo reads well. The trouble starts when someone asks where a number came from.

A credit committee, an LP and an external auditor all ask the same 3 questions about a figure: which document it came from, how it was calculated, and whether the answer would be the same if they ran the analysis again. A chat summary struggles with all 3.

  • Numbers that move between runs. The same prompt over the same files can return a different EBITDA or a different DSCR on the next run, or after the provider updates the model.
  • Estimated arithmetic. A language model generates ratios as text. It predicts what a division probably returns, and a rounding slip in one step carries into every ratio built on it.
  • No page-level source. A summary may name a document, and checking that the figure actually appears there is left to the analyst.
  • No version history. When the borrower sends restated accounts, nothing records which version of the file the earlier memo relied on.
  • No record of what the model saw. Chat history sits inside the vendor's application, and it can't show which files, in which versions, were in the context when the answer was written.

Each gap sends an analyst back to rebuild the spread by hand before the deal goes to committee. The model saves time on the draft and the team spends it again on checking.

An auditable pipeline splits the work along a clear line. A language model reads and extracts. Code computes. Hashes record the evidence. The rest of this guide walks through each stage of that split.

Stage 1: document intake and cited extraction

Every figure in a spread is only as good as the document behind it. Stage 1 builds a complete, hashed room before any number is extracted.

A borrower portal under your brand

You white-label the borrower ingestion portal with your own brand and choose which checklist templates each applicant sees. The applicant signs in, opens a request, uploads documents and gives consent.

The company checklist asks for 8 required items, from the P&L / Income Statement and Balance Sheet to the Debt Schedule / Credit Report. Each of 11 deal types adds its own product checklist, from trade receivables and CRE income-producing loans to SMB acquisition and crypto-backed lending.

Redact, then classify

PII is redacted before classification, so only redacted text reaches the classifier. A keyword match and an LLM then place each file in a checklist category.

A keyword hit at confidence 0.8 or above returns at once, and a keyword result overrides a low-confidence LLM answer. Files that match no category land in a catch-all folder named "Other".

Unreadable files and the data-room inventory

Each upload is hashed with SHA-256 to check that the file arrived intact. Files the pipeline can't read go on an "unreadable" list for your team to chase, and the borrower can't submit until every document's classification has finished.

The data-room inventory then lists every file in the room, sorted by relative path, with its size, sha256, extension, MIME type and file class. Re-running it over an unchanged room produces byte-identical output, apart from the timestamp. The document receipt counts required items, completed required items and gaps, so missing documents are as visible as the ones that arrived.

Citations resolved against the inventory

Every documentary figure is stored with 3 parts: the value, a rationale and the source document. The source is resolved against the data-room inventory, and the cited file's SHA-256 is recorded next to the figure.

The spread links each figure back to its source: an analyst clicks a figure in the spread and sees the line in the original statement. The interface shows the source document's name, plus the page wherever the extractor resolved one. A citation can also carry the anchored proof ID and the catalog version ID of its source.

An unresolvable citation is a kill-gate. The run stops, a person fixes the citation, and nothing rewrites it automatically.

Stage 2: deterministic metrics

Stage 2 turns cited figures into ratios. Rekord AI spreads the borrower's financials through a fixed set of named metric functions. Each formula is transcribed from its definition, and a language model plays no part in the arithmetic.

The core formulas

  • EBITDA = net income + interest expense + income tax expense + depreciation and amortization.
  • DSCR = EBITDA ÷ total debt service. Adjusted DSCR = adjusted EBITDA ÷ total debt service.
  • Debt Yield = NOI ÷ loan amount.
  • LTV = loan amount ÷ collateral value.
  • ABL borrowing base = eligible pool × advance rate − specific reserves, where advance rate = 1 − dynamic credit enhancement, optionally capped by a configured advance-rate cap.

Other named metrics include gross profit, EBITDA margin, net profit margin, YoY revenue growth, current ratio, debt-to-equity, working capital, debt-to-EBITDA, free cash flow, SDE, enterprise value, collateral-to-debt and portfolio loss rate.

Golden test vectors

A golden test vector is a fixed set of inputs paired with the output the formula must return. Each metric function is checked against its vectors.

If a change to a formula alters a result, the matching vector fails and the change is caught. That's what makes a ratio reproducible: the same inputs return the same DSCR, Debt Yield or borrowing base, this quarter and next, for the analyst and for the auditor.

The calculation trace

Each step of the borrowing base lands in a calculation trace with its formula, inputs, unit and source. A reviewer reads the trace top to bottom and can redo every step with a calculator, without access to the model or to the spreadsheet that produced it.

Acceptance thresholds belong to each lender's credit policy. The formulas produce the figures; the policy decides what a given DSCR or LTV means for the deal.

Stage 3: factor-attributed scorecards and the evidence revision manifest

Stage 3 packages the outcome and the evidence behind it into records that a third party can check later.

Factor-attributed scorecards

Every scorecard breaks down to the factors and weights that drove it, ready for a credit committee or a regulator. The committee sees which inputs moved the result and by how much.

For illustration only, a factor could be a coverage ratio taken from the spread, or the share of required checklist items the borrower delivered. The factors, weights and acceptance thresholds in a live mandate follow each lender's credit policy.

Scorecards are serialized as canonical JSON before hashing, and every scorecard emits a cryptographic proof anchored by Rekord Kloud, so any decision can be reconstructed and verified later.

The evidence revision manifest

When you publish an evidence revision, Rekord AI writes a manifest of its member files, serializes it as canonical JSON, hashes it with SHA-256 and anchors it as a proof through Rekord Kloud.

Each revision points to the previous one, so the history of the room is a chain you can walk. When a borrower sends restated accounts, the new revision records the new file version, and the earlier revision still shows what stood behind the earlier decision.

For each member file, the manifest records its logical document ID, proof ID, version ID, filename, MIME type, fileHash and size. The worked example below shows the shape.

Black-box LLM summarization, manual spreading and Rekord AI compared

The table compares the 3 approaches on the points an investment committee, an LP or an external auditor asks about.

On turnaround, Rekord AI has delivered a 95% reduction in time for SMB acquisition finance underwriting analysis.

Worked example: DSCR, Debt Yield, LTV and the ABL borrowing base

The figures below are round illustrative inputs for 3 separate facilities: a corporate term loan, an income-producing CRE loan and an ABL revolver on trade receivables. Source references are placeholders that show the shape of a citation.

In Rekord AI the calculation trace covers each step of the borrowing base. The DSCR, Debt Yield and LTV rows use the same shape to show how each ratio traces back to its inputs.

Read the trace step by step. EBITDA adds back interest, tax and D&A to net income. Total debt service adds scheduled principal to interest, and DSCR divides the first by the second.

In the borrowing base, the dynamic credit enhancement of 12.00% would allow an advance rate of 88.00%, and the configured cap of 85.00% binds first. The cap is optional; without it, step 7 would carry the uncapped rate.

Every input in the trace points to a document, and every result follows from the row above it. An auditor who disagrees with a figure can name the exact step and the exact source instead of re-running the whole analysis. The thresholds that decide whether these figures pass belong to the lender's credit policy, and none are implied here.

Worked example: verifying an evidence revision manifest

The excerpt below uses the real field names of the evidence revision manifest with placeholder values. Identifiers prefixed example- are placeholders.

To keep the example checkable, each fileHash is the real SHA-256 of a short stand-in string in place of the PDF bytes. For the first member, that string is example bytes of balance-sheet-fy2025.pdf, UTF-8 encoded, and sizeBytes is its length in bytes.

{
  "kind": "underwriting_evidence_revision",
  "schemaVersion": 1,
  "canonicalizationVersion": 1,
  "dataroomId": "example-dataroom-id",
  "revisionId": "example-revision-3",
  "revisionNumber": "3",
  "previousRevisionId": "example-revision-2",
  "reason": "example: FY2025 audited statements received",
  "publishedBy": "example-analyst-id",
  "publishedAt": "…",
  "members": [
    {
      "logicalDocumentId": "example-balance-sheet",
      "proofId": "example-proof-balance-sheet",
      "versionId": "example-version-balance-sheet",
      "filename": "balance-sheet-fy2025.pdf",
      "mimeType": "application/pdf",
      "fileHash": "e8cc2f7cf833286d5bd67f200bcca2b4bf5732fe142292563545e24271b8b501",
      "sizeBytes": "41"
    },
    {
      "logicalDocumentId": "example-income-statement",
      "proofId": "example-proof-income-statement",
      "versionId": "example-version-income-statement",
      "filename": "income-statement-fy2025.pdf",
      "mimeType": "application/pdf",
      "fileHash": "2dde2c4638ebf9b14652ae610d39a042fcd8924c85d92196aa46fc49b89698d6",
      "sizeBytes": "44"
    }
  ]
}

How an auditor verifies it

  1. Take the anchored proof of the revision from Rekord Kloud and the manifest the lender hands over.
  2. Serialize the manifest as canonical JSON, hash it with SHA-256 and compare the result with the anchored hash. A match shows the manifest is the one that was published.
  3. For each member, recompute the SHA-256 of the file and compare it with its fileHash. Any mismatch shows the file changed after publication.
  4. Follow previousRevisionId to the earlier revision and repeat, so the full history of the room is checked in order.
  5. Recompute the metrics from the cited figures. The formulas are deterministic, so the same inputs return the same DSCR, Debt Yield or borrowing base.

You can reproduce the first fileHash on any machine with a standard SHA-256 tool over the stand-in string. With real files, the auditor runs the same check over the PDF bytes the lender provides.

The full architecture walkthrough shows where this step sits between borrower ingestion and on-chain anchoring.

The data stays inside your perimeter

Rekord AI runs inside your own infrastructure. It connects directly to your data sources, bureau feeds and bank integrations, so borrower files stay where your controls already apply.

  • Redaction first. PII is redacted before classification, so only redacted text reaches the classifier.
  • Hashes only on-chain. Rekord Kloud anchors SHA-256 hashes. A hash holds none of the document content.
  • Documents stay with the lender. Financial statements, bank data and legal agreements stay with you.

The anchored record is tamper-detectable. It shows that a given manifest existed at a given time and lets anyone with the files check that nothing has changed since. Reviewers read anchored proofs through the Proof Console in Rekord Kloud.

Rekord AI and Rekord Kloud are built by Rekord AG, Zug, Switzerland. The Rekord AI product page covers the full feature set, including the "Ask the room" copilot that answers questions about the dataroom with numbered citations.

Frequently asked questions

Why is an LLM chat summary not enough for a credit committee?

A chat summary gives a conclusion without a reproducible path to it. The same prompt can return different numbers on different runs, the arithmetic is estimated by the model, and the summary rarely points to the page a figure came from. A credit committee and an auditor need every figure traced to a source and every ratio recomputable.

Does the language model calculate DSCR in Rekord AI?

No. A language model extracts figures from the documents, and code computes every ratio from them through named metric functions. DSCR, Debt Yield, LTV and the ABL borrowing base are fixed formulas locked behind golden test vectors, so the same inputs always return the same outputs.

How does Rekord AI cite the source of a figure?

Every documentary figure is stored with its value, a rationale and the source document, resolved against the dataroom manifest so the file SHA-256 is recorded next to the figure. If a citation cannot be resolved, the run stops and a person fixes it. The interface shows the document name, plus the page wherever the extractor resolved one.

What is the evidence revision manifest?

When you publish an evidence revision, Rekord AI writes a manifest of its member files, serializes it as canonical JSON, hashes it with SHA-256 and anchors it as a proof through Rekord Kloud. Each revision points to the previous one, so an auditor can walk the history of the dataroom and recompute every file hash.

Does borrower data leave our environment?

Rekord AI runs inside your perimeter. PII is redacted before classification, so only redacted text reaches the classifier, and only hashes are anchored on-chain. Financial statements, bank data and legal agreements stay with you.

Underwriting your auditor can recompute.