Reinforcement Learning for Reconfigurable Intelligent Surfaces: Assisted Wireless Communication Systems (可重構智慧表面的強化學習：輔助無線通信系統)

本書介紹了兩個動態領域的交集：強化學習（Reinforcement Learning, RL）和RIS輔助無線通信。重點在於離散和連續問題，提供了RL技術及其在不斷發展的RIS輔助無線通信世界中的應用的全面概述。第一章介紹了RL和深度強化學習（Deep RL, DRL）的基本原理，為理解後續章節奠定了堅實的基礎。它還介紹了Q-learning、深度Q-learning和深度確定性策略梯度算法。第二章提供了RIS輔助系統的整體概述，並詳細說明了無線通信中的幾個使用案例。接下來，第三章和第四章展示了離散和連續DRL在RIS輔助無線通信中的各種應用，從最大化總速率到最小化系統資源和最大化能源效率。這些章節展示了DRL算法在應對各種挑戰中的多樣性。本書以第五章作結，介紹了該領域的挑戰和未來方向。它探討了超參數調整、問題設計和複雜性分析的細節，同時強調了混合DRL、多智能體DRL和遷移學習技術在推進無線通信系統中的潛力。

優化RIS輔助無線系統需要強大的算法來應對動態的傳播環境。DRL被視為發揮RIS輔助無線通信系統全部潛力的關鍵技術之一。它使這些系統能夠智能地適應動態無線環境，最大化性能指標，並調整其配置以有效地滿足多樣化的使用案例。

本書作為一個寶貴的資源，闡明了DRL在優化RIS輔助無線通信中的潛力，使研究人員、工程師、計算機科學和電機工程的高級學生以及愛好者能夠掌握這一主題的複雜性。它提供了對原則、應用和挑戰的全面理解，使其成為認識RIS技術在現代無線通信系統中全部潛力的參考資料。

Alice Faisal received her B.Sc. degree in electrical and computer engineering from Effat University, Jeddah, Saudi Arabia, in 2020. She received her M.Sc. degree in electrical engineering from Memorial University, St. John's, NL, Canada, in 2022. Her research interests include wireless communications, 5G enabling technologies, reconfigurable intelligent surfaces, and full-duplex communications.Ibrahim Al-Nahhal (IEEE senior member) is a research associate and per-course instructor at Memorial University, Canada, since 2021. He received the B.Sc. (Honours), M.Sc., and Ph.D. degrees in Electronics and Communications Engineering from Al-Azhar University in Cairo, Egypt-Japan University for Science and Technology, Egypt, Memorial University, Canada, in 2007, 2014, and 2020, respectively. Between 2008 and 2012, he was an engineer in industry, and a Teaching Assistant at the Faculty of Engineering, Al-Azhar University in Cairo, Egypt. From 2014 to 2015, he was a physical layer expert atNokia (formerly Alcatel-Lucent), Belgium. He holds three patents. He co-authored 30+ peer-reviewed journals and conference papers in top-ranked venues. He serves as Editor of IEEE Wireless Communications Letters. He served as a Technical Program Committee and Reviewer for various prestigious journals and conferences. He was awarded the Exemplary Reviewer of IEEE Communications Letters in 2017. His research interests are reconfigurable intelligent surfaces, full-duplex communications, integrated sensing and communication, channel estimation, machine learning, design of low-complexity receivers for emerging technologies, spatial modulation, multiple-input multiple-output communications, sparse code multiple access, and optical communications.Octavia A. Dobre is a Professor and Canada Research Chair Tier 1 with Memorial University, Canada. She was a Visiting Professor with Massachusetts Institute of Technology, USA and Université de Bretagne Occidentale, France. Her research interests encompass wireless communication and networking technologies, as well as optical and underwater communications. She has (co-)authored over 450 refereed papers in these areas.Dr. Dobre serves as the Director of Journals of the Communications Society. She was the inaugural Editor-in-Chief (EiC) of the IEEE Open Journal of the Communications Society and the EiC of the IEEE Communications Letters.Dr. Dobre was a Fulbright Scholar, Royal Society Scholar, and Distinguished Lecturer of the IEEE Communications Society. She obtained Best Paper Awards at various conferences, including IEEE ICC, IEEE Globecom, IEEE WCNC, and IEEE PIMRC. Dr. Dobre is an elected member of the European Academy of Sciences and Arts, a Fellow of the Engineering Insitute of Canada, a Fellow of the Canadian Academy of Engineering, and a Fellow of the IEEE. Telex M. N. Ngatched is an Associate Professor at McMaster University, Canada. His research interests include 5G and 6Genabling technologies, optical wireless communications, hybrid optical wireless and radio frequency communications, and artificial intelligence and machine learning for communications. Dr. Ngatched serves as an Area Editor of the IEEE Open Journal of the Communications Society, an Associate Technical Editor of the IEEE Communications Magazine, an Editor of the IEEE Communications Society On-Line Content, and Co-Chair and Technical Program Committee member for many prominent IEEE conferences in his area of expertise.

Alice Faisal於2020年在沙烏地阿拉伯吉達的Effat University獲得電機與計算機工程學士學位。她於2022年在加拿大聖約翰的Memorial University獲得電機工程碩士學位。她的研究興趣包括無線通信、5G啟用技術、可重構智能表面以及全雙工通信。

Ibrahim Al-Nahhal（IEEE資深會員）自2021年以來擔任加拿大Memorial University的研究助理和課程講師。他於2007年、2014年和2020年分別在埃及的阿茲哈大學、埃及-日本科技大學和加拿大的Memorial University獲得電子與通信工程的學士（榮譽）、碩士和博士學位。在2008年至2012年間，他在業界擔任工程師，並在埃及開羅的阿茲哈大學工程學院擔任教學助理。2014年至2015年，他在比利時的Nokia（前身為Alcatel-Lucent）擔任物理層專家。他擁有三項專利，並在頂尖期刊和會議上共同撰寫了30多篇經過同行評審的論文。他擔任IEEE Wireless Communications Letters的編輯，並曾擔任多個知名期刊和會議的技術程序委員會成員及審稿人。他於2017年獲得IEEE Communications Letters的傑出審稿人獎。他的研究興趣包括可重構智能表面、全雙工通信、集成感測與通信、信道估計、機器學習、新興技術的低複雜度接收器設計、空間調變、多輸入多輸出通信、稀疏碼多重接入和光通信。

Octavia A. Dobre是加拿大Memorial University的教授及加拿大研究主席Tier 1。她曾擔任美國麻省理工學院和法國西布列塔尼大學的訪問教授。她的研究興趣涵蓋無線通信和網絡技術，以及光通信和水下通信。她在這些領域（共同）撰寫了超過450篇經過審核的論文。

Dobre博士擔任通信學會期刊的主任。她是IEEE Open Journal of the Communications Society的首任主編，以及IEEE Communications Letters的主編。Dobre博士曾是富布賴特學者、英國皇家學會學者和IEEE通信學會的傑出講者。她在多個會議上獲得最佳論文獎，包括IEEE ICC、IEEE Globecom、IEEE WCNC和IEEE PIMRC。Dobre博士是歐洲科學與藝術學院的當選成員，加拿大工程學會的院士，加拿大工程學院的院士，以及IEEE的院士。

Telex M. N. Ngatched是加拿大麥克馬斯特大學的副教授。他的研究興趣包括5G和6G啟用技術、光無線通信、混合光無線和射頻通信，以及用於通信的人工智慧和機器學習。Ngatched博士擔任IEEE Open Journal of the Communications Society的區域編輯、IEEE Communications Magazine的副技術編輯、IEEE Communications Society線上內容的編輯，以及多個他專業領域內知名IEEE會議的共同主席和技術程序委員會成員。