B5: Continuous learning across patients

A federated system has a central coordinator and many local nodes (typically hospital systems or research institutions). The coordinator distributes a model to each node, each node trains on its local data, and only model updates — gradients or weight deltas, never raw patient data — flow back to the coordinator, which aggregates the updates into an improved global model and redistributes it. Patient data never leaves the institution that holds it.

The canonical algorithm is FedAvg (federated averaging): each node trains for a few local epochs, sends its updated weights, and the coordinator averages them weighted by each node's dataset size. In practice, several refinements are layered on:

Handling non-IID data. Medical data is emphatically not independent and identically distributed across institutions — a paediatric hospital, an oncology centre, and a rural general hospital have radically different patient populations. Naive FedAvg can fail to converge or can produce a global model that serves no institution well. Algorithms like FedProx, SCAFFOLD, and personalised federated learning address this by allowing controlled local divergence from the global model.

Secure aggregation. Cryptographic protocols ensure the coordinator can compute the aggregate of node updates without seeing any individual node's update — protecting against a curious or compromised coordinator.

Differential privacy. Calibrated noise is added to updates so that the contribution of any single patient cannot be inferred from the model, providing a formal mathematical privacy guarantee at a measurable cost to model accuracy.

The full stack — FedAvg with non-IID handling, plus secure aggregation, plus differential privacy — is mature enough for deployment but operationally heavy. The MELLODDY consortium demonstrated this approach at pharmaceutical-industry scale, training shared models across competing companies' proprietary chemical libraries without any company seeing another's data.

Three classes of residual risk persist even with a well-implemented federated stack, and each requires ongoing operational attention rather than a one-time fix.

Gradient leakage. In some configurations — particularly with large model updates and small local batches — the model updates a node sends back can be partially inverted to reconstruct features of the training data that produced them. Research has demonstrated reconstruction of recognisable training images from gradients in certain settings. Differential privacy mitigates this by adding noise, but stronger privacy guarantees mean more noise and lower model accuracy, so deployments must navigate a privacy-utility tradeoff rather than eliminating the risk outright.

Adversarial nodes. A participating institution with corrupted data, faulty instrumentation, or malicious intent can poison the global model — either degrading it generally or implanting targeted backdoors that cause specific misbehaviour on specific inputs. Byzantine-robust aggregation methods (Krum, trimmed mean, median-based aggregation) detect and down-weight anomalous updates, but a sufficiently sophisticated adversary staying within statistical norms can still exert influence. This connects directly to the adversarial-risk concerns in the study's risk assessment.

Distribution shift and fairness. A federated model trained across heterogeneous institutions may perform well on average but poorly for patients dissimilar from any institution's typical population — and if certain demographics are underrepresented across all participating nodes, the federated model inherits and can amplify that gap. This intersects with the equity concerns the study raises repeatedly: federation does not automatically produce equitable models, and can entrench existing representation gaps if not deliberately countered.

None of these is a showstopper, but each is a permanent feature of the operational landscape that monitoring and governance must address continuously, not a bug to be fixed once.

The core tension: a model that updates after approval is not, in a strict regulatory sense, the same model that was approved. Continuous learning across patients (which is what B5 enables) collides directly with regulatory frameworks built around fixed, validated artefacts.

The FDA's predetermined change control plan (PCCP) framework is the key enabling mechanism. It allows an AI system to update after approval if the update mechanism itself is approved at certification time — the manufacturer specifies in advance what kinds of changes the model may undergo, how those changes will be validated, and what performance bounds must be maintained. For federated learning, the update mechanism is the federated training pipeline, so the PCCP must characterise and bound what that pipeline can do to the model.

Several practical complications follow:

Jurisdictional fragmentation. FDA, EMA, MHRA, and Health Canada are aligning on continuous-learning frameworks but not identically. A federated system training across institutions in multiple jurisdictions must satisfy all applicable frameworks simultaneously, and the data-residency rules in each jurisdiction constrain which nodes can participate in which training rounds.

Layered consent. A patient consenting to clinical care is not thereby consenting to have their data inform a federated model. Separate, explicit, revocable consent for federated training participation is required — and handling consent revocation in a model that has already learned from the data is technically and legally unsettled (machine unlearning is an active research area).

Auditability. The audit trail for federated training must be reconstructable: which institution contributed what to the model at each update round, so that if a problem emerges, its origin can be traced. This implies infrastructure — model lineage tracking, versioned checkpoints, contribution logging — not yet standard in medical AI deployment.

This is why B5 is classified as policy-and-infrastructure rather than science. The technical pieces exist; the regulatory frameworks, governance structures, and operational infrastructure are the work. It connects to the broader continuous-learning challenge discussed under A4 (how the integrated state representation updates without breaking safety validation) — B5 provides the distributed-training substrate that A4's update problem is one instance of.