IEEE.orgIEEE Xplore Digital Library IEEE Standards IEEE SpectrumMore Sites
T14: When Non-Terrestrial Networks Meet Deep Reinforcement Learning: Technologies, Challenges, and Opportunities - VTC2022-Fall - London/Beijing

T14: When Non-Terrestrial Networks Meet Deep Reinforcement Learning: Technologies, Challenges, and Opportunities


Co-organizer: Yu-Jia Chen, National Central University, Taiwan
Co-organizer: Shao-Yu Lien, National Chung Cheng University, Taiwan


Abstract: The sixth-generation (6G) network aims to provide seamless global connectivity and high-speed broadband access through developing non-terrestrial networks (NTNs) which integrate unmanned aerial vehicles (UAVs) networks, high altitude platform systems, and satellite communication networks. However, NTNs are complex systems due to its decentralized and ad-hoc nature. In particular, a large number of network entities in NTNs such as UAVs need to make local and autonomous decisions to optimize different design objectives including completion time minimization and throughput maximization. Conventional optimization techniques like convex optimization are difficult to handle such large-scale problems, especially in a much more uncertain and stochastic environment. In recent years, deep reinforcement learning (DRL) has been developing as a promising solution to overcome these challenges. In DRL, an agent can learn the optimal policy by interacting with the unknown environment and discovering which actions yield the highest reward. In this tutorial, we will first overview the latest 3GPP standardization status of NTNs, including the key performance issues of UAVs and satellite networks. As a step further, we will present how the state-of-art DRL methods enable autonomous aerial platforms, including satellites and UAVs. For UAVs, the following subjects will be addressed: (i) multi-agent distributed/federated reinforcement learning (RL) technique for UAV assisted mobile edge computing, (ii) self-imitation learning for UAV trajectory optimization, (iii) DRL with graph neural network (GNN) for NTN base station deployment, and (iv) multi-step/multi-agent RL for non-terrestrial base station deployment. For satellites, the following subjects will be elaborated: (i) DRL for multi-user access control in low-earth orbit (LEO) satellite networks, (ii) multi-tier DRL for LEO satellite networks, and (iii) multi-tier DRL for satellite-UAV multi-tier networks. Finally, research challenges and open issues will be discussed both in terms of practical applicability to various scenarios and algorithm perspectives.


Co-organizer’s bios:

Yu-Jia Chen

Bio: Yu-Jia Chen received the B.S. degree and Ph.D. degree in electrical engineering from National Chiao Tung University, Taiwan, in 2010 and 2015, respectively. From 2015 to 2018, he was a postdoctoral research fellow with National Chiao Tung University, Taiwan, and he was a postdoctoral research fellow with Harvard University from 2018 to 2019. In 2019, he joined National Central University, Taiwan, where he is currently an assistant professor at the department of communication engineering. His research interests include non-terrestrial networks, wireless sensing and localization, and IoT security. Dr. Chen has published more than 40 articles in peer-reviewed international journal and conference papers. He is holding four US patents and four ROC patents.

Dr. Chen has been serving as Technical Organizing Committee and Symposium Co-chair for many international conferences and symposia, including Globecom, ICC, and PIMRC. He is also co-founder of the IEEE workshop SPSCS, focusing on security and privacy in smart and connected systems. Prof. Chen has experience with tutorials at academic conferences such as Globecom and VTC. He also serves as a guest editor for IEEE Vehicular Technology Magazine special issue on Artificial Intelligence for Autonomous Vehicular Communication Networks. He is a Senior Member of IEEE.


Shao-Yu Lien

Bio: Shao-Yu Lien received his B.S. degree from National Taiwan Ocean University in 2004, M.S. degree from National Cheng Kung University in 2006, and Ph.D. degree from National Taiwan University in 2011. He was with the National Formosa University, as an assistant professor and associate professor from 2013 to 2017, and he is now with National Chung Cheng University as an associate professor. Dr. Lien is also a technical director of Institute for Information Industry, since 2020. Dr. Lien received a number of prestigious research recognitions, including IEEE Tainan Section Best Young Professional Member Award 2019, IEEE Communications Society Asia-Pacific Outstanding Paper Award 2014, Scopus Young Researcher Award (issued by Elsevier) 2014, and IEEE ICC 2010 Best Paper Award. Dr. Lien is a guest editor of IEEE Transactions on Cognitive Communications and Networking in 2019, and a guest editor of Wireless Communications and Mobile Computing (WCMC) in 2017. In the meantime, Dr. Lien also served as the leading organizers of a number of technical workshops in IEEE VTC-Spring 2015, IEEE GLOBECOM 2015, Qshine 2015 and 2016, and IEEE PIMRC 2017, IEEE GLOBECOM 2019 and IEEE ICC 2020. Dr. Lien’s research interests include configurable networks, cyber-physical systems, radio access networks and robotic networks.