Introduction to Federated Learning
In Federated Learning, a model is trained from user interaction with mobile devices. Federated Learning allows mobile phones to collaborate on a shared prediction model while retaining all of the training data on the device, removing the requirement to save data in the cloud from machine learning techniques. This strategy goes beyond using local models to create predictions based on mobile device APIs like the Mobile Vision API or the On-Device Smart Reply, by also bringing model training to the device.
Here are some some popular Federated Learning Frameworks
Flower: Used by Nokia, Samsung, University of Cambridge
Tensorflow: Most popular, used extensively in research
Nvidia Clara: GPU based Federated Learning
There is a difference between traditional learning and federated learning.
Traditional Machine Learning involves a single system that contains both the data and the model. The model is trained locally and the entire dataset is utilized.
Federated Learning, on the other hand, decentralizes the data. That is, the data is distributed between multiple clients and multiple local models are averaged to form one central model.
As is the case with Traditional Machine learning, the data in a Federated Learning model can be both homogeneous and heterogeneous. However, heterogeneous data in Federated Learning is still an area of research and question.
As stated, the data in Federated Learning is decentralized. That is, data from multiple sources can be trained upon individually and then a resultant model can be formed in a server. This doesn’t only increase the variety of data we can train on but is also less computationally expensive.
There are various applications of federated learning, such as:
Mobile applications:
Federated learning can be used to build models on user behavior from data pool of smart phones without leaking personal data, such as for next-word prediction, face detection, voice recognition, etc. For example, Google uses federated learning to improve on-device machine learning models like “Hey Google” in Google Assistant which allows users to issue voice commands.
Autonomous Vehicles:
Autonomous vehicles need these to respond to new situations: real-time information about the traffic and roads, real-time decision making and continual learning. Federated learning can achieve all of these objectives and allow the models to improve over time with input from different vehicles.
Healthcare:
Healthcare and health insurance industry can take advantage of federated learning because it allows the protection of sensitive data in the original source. Federated learning models can provide better data diversity by gathering data from various locations.
Moving forwards to the current reasearch on federated learning, a few significant ones are listed below:
Federated Learning for Mobile Keyboard Prediction :
The feasibility and benefits of training language models on client devices without exporting sensitive user data to servers are demonstrated in this article by Google researchers.
Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge (FedGKT)
It proposes an alternating minimisation strategy for training small CNNs on edge nodes and regularly transferring their knowledge to a large server-side CNN via knowledge distillation.
Central Server Free Federated Learning over Single-sided Trust Social Networks
Researchers from WeBank, Kwai, the University of Southern California, the University of Michigan, and the University of Rochester introduced the Online Push-Sum (OPS) approach as a central server free federated learning algorithm to handle various issues in a general context in this work.
Now that we’ve grasped the fundamental definition and present applications of federated learning, it’s critical to understand its future implications.
Extreme communication schemes:
In federated learning, how much communication is required remains to be seen. Can we, for example, learn more about one-shot/few-shot communication techniques in huge and statistically heterogeneous networks from a theoretical and empirical standpoint?
Heterogeneity diagnostics:
This prompts the following inquiries: Are there any easy diagnostics that can be utilized before training to quantify systems and statistical heterogeneity? Is it possible to use these diagnostics to improve the convergence of federated optimization methods?
Productionizing federated learning:
When operating federated learning in production, a number of practical difficulties occur. How do we deal with difficulties like concept drift (when the underlying data-generation model evolves over time), diurnal variations (when devices behave differently at different times of the day or week), and cold start problems (when new devices enter the network), for example?
In conclusion, Federated Learning appears to have ushered in a new era of safe and secure AI, with much of promise for future development. It not only secures sensitive user information, but it also combines results and identifies similar trends across a large number of them, making the model more resilient.
