https://doi.org/10.1140/epje/s10189-023-00271-0
Regular Article - Flowing Matter
Taming Lagrangian chaos with multi-objective reinforcement learning
1
Department of Physics & INFN, University of Rome ‘Tor Vergata’, Via della Ricerca Scientifica 1, 00133, Rome, Italy
2
Laboratory of Physics of the École Normale Supérieure, 24 RueLhomond, 75005, Paris, France
3
Quantitative Life Sciences, The Abdus Salam International Centre for Theoretical Physics, ICTP, 34151, Trieste, Italy
4
Istituto dei Sistemi Complessi, CNR, Via dei Taurini 19, 00185, Rome, Italy
5
INFN ‘Tor Vergata’, Rome, Italy
Received:
19
December
2022
Accepted:
15
February
2023
Published online:
3
March
2023
We consider the problem of two active particles in 2D complex flows with the multi-objective goals of minimizing both the dispersion rate and the control activation cost of the pair. We approach the problem by means of multi-objective reinforcement learning (MORL), combining scalarization techniques together with a Q-learning algorithm, for Lagrangian drifters that have variable swimming velocity. We show that MORL is able to find a set of trade-off solutions forming an optimal Pareto frontier. As a benchmark, we show that a set of heuristic strategies are dominated by the MORL solutions. We consider the situation in which the agents cannot update their control variables continuously, but only after a discrete (decision) time, 
. We show that there is a range of decision times, in between the Lyapunov time and the continuous updating limit, where reinforcement learning finds strategies that significantly improve over heuristics. In particular, we discuss how large decision times require enhanced knowledge of the flow, whereas for smaller all a priori heuristic strategies become Pareto optimal.
© The Author(s), under exclusive licence to EDP Sciences, SIF and Springer-Verlag GmbH Germany, part of Springer Nature 2023. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
