I want to talk about a rabbit hole I have fallen down since reading a paper on The Promises and Predicaments of Federated Learning in Healthcare.
Last year, I had the privilege of working with an awesome team, we explored ideas applying machine learning to tackle data interoperability challenges within our healthcare system. The problem: disparate data formats, strict privacy regulations, and the sheer volume of sensitive patient information scattered across multiple institutions. Our thesis was that the near-term impact of data interoperability is obvious, time saved and admin (huge burden to the NHS) reduced. We validated this with our study across thousands of hospital reported EHR systems, and with firsthand NHS clinical experience in the team. But the bigger picture? The data unlock.
I originally thought the bottleneck in healthcare AI would be the models themselves, then perhaps compute, and the quality of data. The deeper problem is coordination.
The problem was that the healthcare system is fundamentally fragmented. Different hospitals store data differently. Incentives are misaligned. Privacy constraints are real. Trust boundaries matter. Even defining a shared dataset becomes difficult once liability, governance, and ownership enter the picture.
Federated learning became interesting because it assumes the opposite: that data may never fully converge into a single place, and that intelligence must emerge across organisational boundaries instead. Centralised learning assumes everyone contributes to one canonical truth.
The concept of federated learning involves partitioning the learning process into smaller units that can occur locally, while model parameters or gradients are shared with a central server for aggregation.
This decentralisation ensures that the raw data never leaves the originating device or organisation, thereby reducing the risk of exposure.
Multiple entities(e.g., mobile devices, hospitals, or organisations) collaboratively train a shared global model without exchanging their local datasets.
So it’s an ML paradigm that allows a model to be trained across decentralised devices or servers holding local data points without exchanging them.
Local models are trained on local data while gradients (model updates) are shared with central server. And, the central server aggregates these updates to improve the global model.
This is relevant where the data used to train models should be kept private, and it’s not desirable to centralise ALL data (Healthcare, edge computing etc. etc.)
Privacy-Enhancing Technologies
This brings us to some problems with FL and the HC application… The potential for leakage from updates. While federated learning reduces the need to share raw data, it isn’t impervious to all threats. Model updates can be reverse-engineered to reveal sensitive information a process known as model inversion attacks.
Here differential privacy and homomorphic encryption become interesting, enable AI systems to learn from data without exposing it in raw form.
Differential Privacy introduces carefully calibrated noise to the data or model updates, making it mathematically improbable to identify any individual’s information from the aggregated data. The noise masks the contribution of a single data point. A general concept for securing data sets.
By adding calibrated noise to the shared parameters or gradients during federated learning, differential privacy ensures that individual records contribute to the model without exposing sensitive information. In FL, DP is applied to the model updates before they are sent to the server.
Homomorphic Encryption extends privacy protection to the computational level by allowing computations on encrypted data. For example, sensitive patient data can remain encrypted while computations (such as training updates) are performed on it. The result, when decrypted, matches the outcome of operations performed on the plaintext data. Data can remain confidential while contribute to a collective learning process.
Partially Homomorphic Encryption (PHE): Supports either addition or multiplication.
Fully Homomorphic Encryption (FHE): Supports arbitrary computations, both additions and multiplications, on encrypted data.
This brings us to the second issue - heterogeneity… hospitals do not just hold different pieces of one same puzzle (dataset). The data is formed from different worlds, diff scanners, label distributions, EHR systems, writing standards, resources. Averaging model updates would degrade performance through client drift. Adding differential privacy noise could sometimes worsen convergence further. Because now the signal is already noisy. So the problem moves from how to we securely federate training to: How do we learn coherently from heterogeneous worlds?
Combining in FL
Process Flow:
Each entity trains the model on its local data.
Entities add random noise to their model updates according to differential privacy mechanisms.
The differentially private updates are then encrypted using a homomorphic encryption scheme.
The server aggregates the encrypted updates without decrypting them, thanks to HE.
The aggregated result is decrypted (if necessary) and used to update the global model.
The updated global model is sent back to for the next training round.
Conclusion
Honestly no strong argument or conclusion to this post, I found the topic interesting and want to get better at publicly documenting my learning rabbit holes. As interesting the topic of target validation and selection in drug discovery or technologies evolving LVV engineering, I think there are incredible and real opportunities in data governance and privacy in healthcare research. Albiet not as sexy a research area, the idea to learn from real live data - the “unlock” on masses of patient data if secure and interoperable would have insane leverage on predicting patient outcomes and potentially trial recruitment.
Appendix
A.1 Federated Learning Optimisation
The objective in FL is to minimize the global loss function:
A.2 Differential Privacy Mechanisms
- Gaussian Mechanism: For function f with sensitivity Δf, the mechanism is:
A.3 Homomorphic Encryption Schemes
- Paillier Cryptosystem: An additive homomorphic encryption scheme where:
