T10: Reinforcement Learning for Optimization of Wireless Systems: Methods, Exploration and Sensing

Presented by: Haris Gacanin (Department Head, Nokia Bell Labs)

Time: 14:00–17:30
Room: Kou

Abstract—This tutorial discusses technology and opportunities to embrace artificial intelligence (AI) in the design of autonomous wireless systems. We aim to provide readers with motivation and general AI methodology of autonomous agents in the context of self-organization in real time unifying sensing, perception, reasoning and learning. We discuss differences between training-based and training-free AI methodology for both matching and dynamic problems, respectively. Finally, we introduce the conceptual functions of autonomous agent with knowledge management. Finally, a practical case study is given to illustrate the application and potential gains.

Tutorial Objectives
Design of autonomous wireless systems with simultaneous service delivery cannot be accomplished by incremental changes to the present deterministic control and optimization methodologies. It requires a fundamental leap in the system’s thinking by embedding active learning and sensing strategies into temporal wireless infrastructure itself. This means that the infrastructure will become aware of the way it is being used to anticipate actual requirements at the specific moment and what it is likely to be required at later time. As such it will facilitate wireless as a true application-aware platform for a plethora of novel applications. The above issues capture new coming challenges and unveil necessary future applications of AI in wireless systems with design challenges. We address the shortcomings of contemporary rule-based optimization protocols and re-thinking our wireless operations for boosting the system autonomy. We discuss about paradigm shift from data-driven knowledge-discovery with ML toward fully inspired knowledge-driven wireless operation with AI in real time.

It is highly expected that different participants may find their interests within this tutorial. The tutorial provides a diverse viewpoint of this topic, as well as facilitate discussions that would enable individuals to think beyond the technology itself.

Tutorial Outline
1) Introduction (15 minutes) a. The relevance and rise of autonomous systems b. User experience systems c. Wireless systems complexity 2) The System-of-Systems (SoS) (35 minutes) a. General theory (systems, interactions, environment) b. Machine learning vs. artificial intelligence c. ML-driven knowledge discovery vs. AI-driven knowledge management 3) Traditional optimization theory vs. machine learning (35 minutes) a. Convex optimization and iterative algorithms b. Deep neural networks c. Reinforcement learning d. Use case study: bit loading, channel estimation, nonlinear distortion 4) Intelligent agent design (40 minutes) a. Type of intelligent agents and environments b. Sensing and percept design c. Reasoning, planning and searching with uncertainty d. Supervised and unsupervised learning e. Learning to act (Reinforcement learning) 5) Case studies: Self-optimization (40 minutes) a. Reinforcement learning optimization framework b. Knowledge Base Representation c. Exploitation vs. exploration dilemma 6) Conclusions and Future Directions (15 minutes)

Primary Audience
Entry level graduate students (PhD-level) who are seeking to pursue dissertation research in 5G and B5G systems, as well as industry practitioners who need to upgrade their skill sets and rethink how to view future communications from network operations perspective. Hence, it is ideally suited for attendees of the conference. No knowledge of communication protocols is required to attend the tutorial.

Novelty
The above addressed issues provoke new coming challenges and unveil necessary directions across multi-disciplinary research areas related to applications of AI with ML in future wireless systems. The following aspects are important:
1) We present wireless system’s requirements and relate those to machine learning properties
2) We present comparison of training-based and training-free AI methods for solving fundamental problems in wireless communications. As example, we show comparison between traditional optimization, deep learning and reinforcement learning
3) We discuss about importance of training-free AI methods for time-sensitive comm.

Biography
Haris Gacanin received his Dipl.-Ing. degree in Electrical engineering from University of Sarajevo, Bosnia and Herzegovina, in 2000. In 2005 and 2008, he received M.E.E. and Ph.D. from Tohoku University, Japan. He was with Tohoku University from April 2008 until May 2010 first as Japan Society for Promotion of Science postdoctoral fellow and then, as Assistant Professor. Since 2010, he is with Alcatel-Lucent (now Nokia), where he is currently Department Head at Nokia Bell Labs leading research activities related to application of artificial intelligence in network optimization with focus on mobile/wireless/wireline physical (L1) and media access (L2) layer technologies and network architectures. He has more than 200+ publications (journals, conferences and patens) and invited/tutorial talks. He organized and hosted several tutorials and industry panels at IEEE conferences. He is senior member of the Institute of Electrical and Electronics Engineers (IEEE) and the Institute of Electronics, Information and Communication Engineering (IEICE).