Double Deep Q-Network Techniques for Optimizing Performance of 6G Wireless Network
DOI:
https://doi.org/10.31185/wjes.Vol13.Iss4.726Keywords:
Sixth Generation Networks, Massive MIMO-OFDM, Double Deep Q-NetworkAbstract
This study introduces a Double Deep Q-Network (DDQN) optimization framework to improve massive MIMO-OFDM systems via reinforcement learning-driven adaptive parameter selection. It utilizes a dual network architecture to mitigate overestimation bias and incorporates dynamic optimization for power allocation, subcarrier fraction distribution, and modulation scheme selection across QAM-16, QAM-64, and QAM-128 configurations. Extensive simulations performed across Signal-to-Noise Ratio ranges from -5 to 35 dBm reveal substantial performance enhancements, with DDQN-augmented systems attaining 5-6 dB SNR savings for equivalent SE, a 50% increase in EE reaching 15.5-16 Gbps/W compared to conventional 10.5-11 Gbps/W implementations, and a 2.5 dB SNR reduction for a BER performance of 10⁻⁵. The optimization framework ensures uniform parameter selection across diverse SNR conditions, facilitating a 40-50% increase in coverage through enhanced low-SNR performance while delivering a 5 dB SNR improvement in low-power operating scenarios. The study establishes a basis for intelligent communication systems that can autonomously adapt to 6G wireless networks, supporting ultra-reliable low-power communications and mobile edge computing applications.
References
[1] L. Li, “Research on future 6G green wireless networks,” Green Technologies and Sustainability, vol. 3, no. 2, p. 100156, 2025, doi: 10.1016/j.grets.2024.100156. DOI: https://doi.org/10.1016/j.grets.2024.100156
[2] X. You et al., “Towards 6G wireless communication networks: vision, enabling technologies, and new paradigm shifts,” Jan. 01, 2021, Science in China Press. doi: 10.1007/s11432-020-2955-6. DOI: https://doi.org/10.1007/s11432-020-2955-6
[3] Z. Hu et al., “Highly Efficient MIMO-OFDM with Index Modulation,” Wirel Commun Mob Comput, vol. 2022, 2022, doi: 10.1155/2022/7399457. DOI: https://doi.org/10.1155/2022/7399457
[4] J. Hou, J. M. Purushothama, H. Fan, C. Song, Y. Ding, and M. Sellathurai, “Energy efficient time-modulated OFDM directional modulation transmitters,” Microw Opt Technol Lett, vol. 65, no. 1, pp. 5–13, 2023, doi: 10.1002/mop.33438. DOI: https://doi.org/10.1002/mop.33438
[5] H. Kasban, S. Hashima, S. Nassar, E. M. Mohamed, and M. A. M. El-Bendary, “Performance enhancing of MIMO-OFDM system utilizing different interleaving techniques with rate-less fountain raptor code,” IET Communications, vol. 16, no. 20, pp. 2479–2491, Dec. 2022, doi: 10.1049/cmu2.12503. DOI: https://doi.org/10.1049/cmu2.12503
[6] I. H. Ahmed and A. A. Abdulkafi, “Energy-Efficient Massive MIMO Network,” Tikrit Journal of Engineering Sciences, vol. 30, no. 3, pp. 1–8, 2023, doi: 10.25130/tjes.30.3.1. DOI: https://doi.org/10.25130/tjes.30.3.1
[7] A. Zaki, A. Métwalli, M. H. Aly, and W. K. Badawi, “Wireless Communication Channel Scenarios: Machine-Learning-Based Identification and Performance Enhancement,” Electronics (Switzerland), vol. 11, no. 19, 2022, doi: 10.3390/electronics11193253. DOI: https://doi.org/10.3390/electronics11193253
[8] E. J. Han, M. Sengly, and J. R. Lee, “Balancing Fairness and Energy Efficiency in SWIPT-Based D2D Networks: Deep Reinforcement Learning Based Approach,” IEEE Access, vol. 10, pp. 64495–64503, 2022, doi: 10.1109/ACCESS.2022.3182686. DOI: https://doi.org/10.1109/ACCESS.2022.3182686
[9] F. H. Juwono and R. Reine, “Future OFDM-based Communication Systems Towards 6G and Beyond: Machine Learning Approaches,” Green Intelligent Systems and Applications, vol. 1, no. 1, pp. 19–25, 2021, doi: 10.53623/gisa.v1i1.34. DOI: https://doi.org/10.53623/gisa.v1i1.34
[10] Z. Liwen, F. Qamar, M. Liaqat, M. Nour Hindia, and K. Akram Zainol Ariffin, “Toward Efficient 6G IoT Networks: A Perspective on Resource Optimization Strategies, Challenges, and Future Directions,” IEEE Access, vol. 12, no. April, pp. 76606–76633, 2024, doi: 10.1109/ACCESS.2024.3405487. DOI: https://doi.org/10.1109/ACCESS.2024.3405487
[11] J. M. Hamamreh, A. Hajar, and M. Abewa, “Orthogonal frequency division multiplexing with subcarrier power modulation for doubling the spectral efficiency of 6G and beyond networks,” Transactions on Emerging Telecommunications Technologies, vol. 31, no. 4, pp. 1–31, 2020, doi: 10.1002/ett.3921. DOI: https://doi.org/10.1002/ett.3921
[12] N. Sivapriya, M. K. Vanteru, K. K. Vaigandla, and G. Balakrishna, “Evaluation of PAPR, PSD, Spectral Efficiency, BER and SNR Performance of Multi-Carrier Modulation Schemes for 5G and Beyond,” SSRG International Journal of Electrical and Electronics Engineering, vol. 10, no. 11, pp. 100–114, Nov. 2023, doi: 10.14445/23488379/IJEEE-V10I11P110. DOI: https://doi.org/10.14445/23488379/IJEEE-V10I11P110
[13] L. Ge, C. Shi, S. Niu, G. Chen, and Y. Guo, “Mixed RNN-DNN based channel prediction for massive MIMO-OFDM systems,” IET Communications, vol. 17, no. 19, pp. 2152–2161, Dec. 2023, doi: 10.1049/cmu2.12685. DOI: https://doi.org/10.1049/cmu2.12685
[14] H. Kasban, S. Hashima, S. Nassar, E. M. Mohamed, and M. A. M. El-Bendary, “Performance enhancing of MIMO-OFDM system utilizing different interleaving techniques with rate-less fountain raptor code,” IET Communications, vol. 16, no. 20, pp. 2479–2491, 2022, doi: 10.1049/cmu2.12503. DOI: https://doi.org/10.1049/cmu2.12503
[15] H. I. Bitat, F. Maamri, F. Khelfaoui, H. Djellab, Y. Belhocine, and Y. Messai, “Optimizing Spectral and Energy Efficiency of Massive MIMO Networks Using MVO and API Algorithms,” Journal of Telecommunications and Information Technology, vol. 90, no. 1, pp. 81–91, 2025, doi: 10.26636/jtit.2025.1.1993. DOI: https://doi.org/10.26636/jtit.2025.1.1993
[16] I. Jaiswal, R. G. Sangeetha, and M. Suchetha, “Performance of M-ary quadrature amplitude modulation -based orthogonal frequency division multiplexing for free space optical transmission,” IET Optoelectronics, vol. 10, no. 4, pp. 156–162, Aug. 2016, doi: 10.1049/iet-opt.2015.0091. DOI: https://doi.org/10.1049/iet-opt.2015.0091
[17] S. T. Başaran and G. K. Kurt, “Joint subcarrier and power allocation in OFDMA systems for outage minimization,” IEEE Communications Letters, vol. 20, no. 10, pp. 2007–2010, Oct. 2016, doi: 10.1109/LCOMM.2016.2586038. DOI: https://doi.org/10.1109/LCOMM.2016.2586038
[18] S. Muy, D. Ron, and J. R. Lee, “Energy Efficiency Optimization for SWIPT-Based D2D-Underlaid Cellular Networks Using Multiagent Deep Reinforcement Learning,” IEEE Syst J, vol. 16, no. 2, pp. 3130–3138, Jun. 2022, doi: 10.1109/JSYST.2021.3098860. DOI: https://doi.org/10.1109/JSYST.2021.3098860
[19] R. Ali, I. Ashraf, A. K. Bashir, and Y. Bin Zikria, “Reinforcement-Learning-Enabled Massive Internet of Things for 6G Wireless Communications,” IEEE Communications Standards Magazine, vol. 5, no. 2, pp. 126–131, Jun. 2021, doi: 10.1109/MCOMSTD.001.2000055. DOI: https://doi.org/10.1109/MCOMSTD.001.2000055
[20] L. Liang, H. Ye, G. Yu, and G. Y. Li, “Deep-Learning-Based Wireless Resource Allocation with Application to Vehicular Networks,” Proceedings of the IEEE, vol. 108, no. 2, pp. 341–356, Feb. 2020, doi: 10.1109/JPROC.2019.2957798. DOI: https://doi.org/10.1109/JPROC.2019.2957798
[21] J. Huang, Y. Yang, Z. Gao, D. He, and D. W. K. Ng, “Dynamic Spectrum Access for D2D-Enabled Internet of Things: A Deep Reinforcement Learning Approach,” IEEE Internet Things J, vol. 9, no. 18, pp. 17793–17807, Sep. 2022, doi: 10.1109/JIOT.2022.3160197. DOI: https://doi.org/10.1109/JIOT.2022.3160197
[22] X. Li, J. Fang, W. Cheng, H. Duan, Z. Chen, and H. Li, “Intelligent Power Control for Spectrum Sharing in Cognitive Radios: A Deep Reinforcement Learning Approach,” IEEE Access, vol. 6, pp. 25463–25473, Apr. 2018, doi: 10.1109/ACCESS.2018.2831240. DOI: https://doi.org/10.1109/ACCESS.2018.2831240
[23] P. Lv, “Design and application of deep reinforcement learning algorithms based on unbiased exploration strategies for value functions,” Measurement: Sensors, vol. 34, p. 101241, Aug. 2024, doi: 10.1016/j.measen.2024.101241. DOI: https://doi.org/10.1016/j.measen.2024.101241
[24] S. S. Omar, A. M. A. El-Haleem, I. I. Ibrahim, and A. M. Saleh, “Capacity Enhancement of Flying-IRS Assisted 6G THz Network Using Deep Reinforcement Learning,” IEEE Access, vol. 11, pp. 101616–101629, 2023, doi: 10.1109/ACCESS.2023.3315660. DOI: https://doi.org/10.1109/ACCESS.2023.3315660
[25] G. Bacci, E. V. Belmega, P. Mertikopoulos, and L. Sanguinetti, “Energy-Aware Competitive Power Allocation for Heterogeneous Networks under QoS Constraints,” IEEE Trans Wirel Commun, vol. 14, no. 9, pp. 4728–4742, Sep. 2015, doi: 10.1109/TWC.2015.2425397. DOI: https://doi.org/10.1109/TWC.2015.2425397
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Yilmaz B. Kamal, Ayad A. Abdulkafi
This work is licensed under a Creative Commons Attribution 4.0 International License.
