Tomasz Kołodziej, Paweł Rościszewski
https://link.springer.com/chapter/10.1007/978-3-030-92307-5_29
Federated learning (FL) allows to train models on decentralized data while maintaining data privacy, which unlocks the availability of large and diverse datasets for many practical applications. The ongoing development of aggregation algorithms, distribution architectures and software implementations aims for enabling federated setups employing thousands of distributed devices, selected from millions. Since the availability of such computing infrastructure is a big barrier to experimenting with new approaches, we claim that efficient simulation of FL is necessary and propose the PaSSiFLora library for simulating FL clients in a cluster environment. In PaSSiFLora, the training algorithm is actually performed on real data, but each cluster node can simulate multiple FL clients. Because uniform random selection of clients results in poor simulation performance due to load imbalance, we propose to use uniform random selection of MultiClients. Each MultiClient runs on a single cluster node and in each training iteration is responsible for simulating several clients, selected from a set of local clients. Our experimental results based on the FEMNIST dataset show that PaSSiFLora is capable of simulating 1536 clients and has a good scalability on 48 cluster nodes, which reduces the average iteration time to 13.57 s, from 330.61 s in the case of one cluster node. The MultiClient architecture allows to improve the average performance by up to 75% while it does not cause significant differences in model accuracy during the training. Additionally, correctness of the training is verified against existing FL frameworks: LEAF and TFF.