Product Updates

Regloom Feed v2: Structured Change Summaries and Control Confidence Scores

Laura Bennett Back to blog
Regloom Feed v2: Structured Change Summaries and Control Confidence Scores article cover

We shipped Feed v2 today. This post explains what we built, what we changed from v1, and why we made those specific choices. If you are a current user, here is what is different in your dashboard. If you are evaluating Regloom, this gives you a clearer picture of how the product actually works.

What v1 Produced and Where It Fell Short

Feed v1 did the job of monitoring regulatory sources and surfacing changes. When an agency published a new rule, proposed amendment, or guidance document, the feed picked it up, extracted the relevant obligation text, and ran a match against controls in a user's library. The output was a feed item with the regulatory source, the relevant text excerpt, and the list of controls the system identified as potentially affected.

What v1 did not do: it treated every feed item as if it had equal weight and equal confidence. A minor technical correction to a footnote in an agency FAQ and a substantive final rule amending a core regulatory obligation both appeared in the same feed format, with the same visual presentation. Users told us that triage was taking too long because distinguishing "this needs immediate review" from "this is probably fine" required reading the full regulatory text for each item.

The second gap: v1 produced control matches but no signal about how confident the system was in each match. A user would see that control CTRL-0147 was flagged for a given regulatory change, but they had no basis for knowing whether that was a tight match on specific obligation language or a loose match on general domain terminology. Both looked the same in the interface.

What v2 Adds: Structured Change Summaries

Every feed item in v2 now includes a structured change summary at the top. The summary has four components:

Change type. One of five classifications: Final Rule, Proposed Rule, Guidance, Examination Priority Update, or Administrative Correction. This is machine-classified with human spot-check on a rolling basis. Classification lets users filter the feed by change type, and it also drives the urgency indicator below.

Obligation-level breakdown. For regulatory changes that create or modify specific obligations, the summary extracts each distinct obligation statement from the source text and presents it as a numbered list. A final rule that amends three separate regulatory requirements shows up as three discrete obligation items, each tagged to the relevant CFR section or subsection. This is the part of v2 that took the longest to get right. Regulatory text is not cleanly structured for machine extraction. We went through multiple parsing iterations and there are still edge cases where the breakdown is imperfect, but for the majority of rule changes from the agencies we cover, the obligation extraction is accurate enough to be useful as a first-pass read.

Urgency signal. A three-level indicator derived from change type and compliance timeline. Final rules with an implementation date within 90 days are High. Proposed rules and guidance are Low by default unless they fall within a product area that the user has flagged as elevated priority. This is not meant to replace human judgment about what warrants immediate attention. It is meant to answer the question "which items in this week's feed should I look at today versus next week?"

Affected agency and rulemaking citation. Unchanged from v1, but now linked directly to the source document so users can get to the full text in one click rather than navigating through an agency's document portal.

What v2 Adds: Control Confidence Scores

Each control mapping in v2 now carries a confidence score expressed as a percentage. The score reflects the semantic similarity between the obligation text extracted from the regulatory change and the scope statement and regulatory citation fields of the matched control. High confidence (above roughly 75%) means the obligation language closely matches what the control's documentation says the control is designed to address. Low confidence (below roughly 45%) means the system found a domain-level match, but the specific obligation language does not map tightly to the control's documented scope.

We are not saying low-confidence matches should be ignored. There are legitimate cases where a broad control has been correctly matched even though the confidence score is low, because the control was written at a level of abstraction that does not use the same specific terminology as the regulatory source. What the score tells you is when to verify the match and when you can move faster.

In practice, the workflow shift looks like this: high-confidence mappings can be reviewed and acknowledged by a compliance analyst in about two to three minutes each, because the connection between the regulatory change and the control is already clear. Low-confidence mappings need a substantive review where someone reads both the obligation text and the control documentation and makes a judgment call. Those are the ones that should get calendar time, not the ones where the system is already pointing at the right control with high certainty.

What Did Not Change

The agency coverage, ingestion frequency, and the control library import format are all unchanged from v1. Existing integrations and API connections continue to work without modification. If you have a control library already imported in v1, it will populate automatically into the v2 feed view. No migration step required.

We also did not change the core mapping algorithm in this release. The confidence score is a new output derived from the existing matching engine, not a new matching engine. We have improvements to the matching algorithm in the roadmap, but those will come in a separate release. For now, the confidence score makes the existing match quality more visible rather than changing the underlying match quality.

A Note on the Obligation Extraction Edge Cases

We want to be direct about where the obligation extraction is imperfect. Preamble language in final rules sometimes contains obligation-like statements that are actually explanatory context, not binding requirements. The parser occasionally extracts these as obligations when they are not. It happens most often with FinCEN guidance documents and with CFPB supervisory highlights, which use a narrative structure that is harder to parse than the formal regulatory text in the CFR.

When the parser extracts something that is clearly not an obligation, the correct action is to mark it "not applicable" in the feed item review. That feedback is logged and factored into future parsing improvements. We are not asking users to tolerate a bad parse indefinitely. We are asking for the feedback loop so the parser gets better over time on the specific document types where it is currently imperfect.

What Is Next

The two features most requested by users on the v1 waitlist that are not yet in v2: batch acknowledgment for low-priority feed items, and a policy document linking flow that attaches internal policy documents to specific controls. Both are in development. Batch acknowledgment is closer and we expect to ship it in the next release cycle. Policy linking requires more design work around the document storage and access model, and we are being deliberate about not shipping something that creates compliance documentation risks.

We also have state-level regulatory monitoring improvements in progress. The current v2 release covers federal agency sources comprehensively. State-level coverage is partial, with the highest-volume states prioritized. Extending state coverage while maintaining the accuracy of the obligation extraction is the technical constraint we are working through. More on that as it progresses.

If you are on the current feed and have feedback on the v2 changes, the feedback link is in the dashboard. We read all of it.