Heurisztikusan gyorsított Fuzzy szabály-interpoláció alapú Q-tanulás
Heuristically Accelerated Fuzzy Rule Interpolation-based Q-learning
Keywords:
reinforcement learning, Q-learning, Fuzzy Rule Interpolation, expert knowledge, knowledge optimization, /, megerősítéses tanulás, Q-tanulás, Fuzzy Szabály-Interpoláció, szakértői tudásbázis, tudásbázisAbstract
The learning phase of the conventional reinforcement learning methods (e.g. Q-learning, SARSA, and Fuzzy Q-learning) starts with an empty knowledge base, which the system gradually builds during the learning process based on the feedback from the environment. However, if partially knowledge base is available and can be integrated into the learning phase, this can have a positive effect on learning performance. The aim of the paper is to introduce a Fuzzy Rule-Interpolation method (‘FIVE’) based Q-learning method which is suitable for incorporating external expert knowledge into the learning process and capable of fine-tuning the initially imprecisely defined expert knowledge during the learning process, thereby correcting (optimizing) it.
Kivonat
A klasszikus megerősítéses tanulási módszerek (Q-tanulás, SARSA, Fuzzy Q-tanulás), tudásbázisa kezdetben ismeretlen, a rendszer ezt a tanulási folyamat során alakítja ki a környezet visszajelzései alapján. Azonban, ha rendelkezésre áll egy részlegesen ismert tudásbázis, amely beilleszthető a tanulási folyamatba, akkor ez által a tanulás hatékonysága növelhető. A cikk bemutat egy olyan fuzzy szabály-interpolációs módszeren (’FIVE’) alapúló Q-tanulási módszert, amely alkalmas külső szakértői tudásbázis injektálására a tanulási folyamatba és alkalmas továbbá a pontatlanul megadott kezdeti szakértői tudásbázis finomhangolására a tanulási folyamat során, így pontosítva (optimalizálva) azt.
References
Berenji, Hamid R. "Fuzzy Q-learning for generalization of reinforcement learning." Proceedings of IEEE 5th International Fuzzy Systems. Vol. 3. IEEE, 1996.
Fan, Jianqing, et al. "A theoretical analysis of deep Q-learning." Learning for Dynamics and Control. PMLR, 2020.
Kovács, S., Vincze, D., Gácsi, M., Miklósi, Á., & Korondi, P. (2011, May). Ethologically inspired robot behavior implementation. In 2011 4th International Conference on Human System Interactions, HSI 2011 (pp. 64-69). IEEE
Kovács, Szilveszter. "Extending the fuzzy rule interpolation" FIVE" by fuzzy observation." Computational Intelligence, Theory and Applications. Springer, Berlin, Heidelberg, 2006. 485-497.
Naeem, M., Rizvi, S. T. H., & Coronato, A. (2020). A gentle introduction to reinforcement learning and its application in different fields. IEEE access, 8, 209320-209344.
Piller, Imre, and Szilveszter Kovács. "FBDL: A Declarative Language for Interpolative Fuzzy Behavior Modeling." 2019 IEEE 23rd International Conference on Intelligent Engineering Systems (INES). IEEE, 2019.
Rummery, Gavin A., and Mahesan Niranjan. On-line Q-learning using connectionist systems. Vol. 37. Cambridge, UK: University of Cambridge, Department of Engineering, 1994.
Sutton, Richard S., and Andrew G. Barto. Reinforcement learning: An introduction. MIT press, 2018.
Tompa, T., Kovács, S., Vincze, D., & Niitsuma, M. (2021, January). Demonstration of expert knowledge injection in Fuzzy Rule Interpolation based Q-learning. In 2021 IEEE/SICE International Symposium on System Integration (SII) (pp. 843-844). IEEE.
Tompa, Tamás, and Szilveszter Kovács. "Applying Expert Heuristic as an a Priori Knowledge for FRIQ-Learning." Acta Polytechnica Hungarica 17.4 (2020).
Tompa, Tamás, and Szilveszter Kovács. "Clustering-based fuzzy knowledgebase reduction in the FRIQ-learning." 2017 IEEE 15th International Symposium on Applied Machine Intelligence and Informatics (SAMI). IEEE, 2017.
Tompa, Tamás, and Szilveszter Kovács. "Determining the minimally allowed rule-distance for the incremental rule-base contruction phase of the FRIQ-learning." 2018 19th International Carpathian Control Conference (ICCC). IEEE, 2018.
Tompa, Tamás, and Szilveszter Kovács. "Heuristically accelerated FRIQ-learning." 20th Jubilee International Symposium on Intelligent Systems and Informatics (SISY 2022). IEEE, 2022.
Tompa, Tamás, and Szilveszter Kovács. "Tudásbázis redukálás a heurisztikusan gyorsított FRIQ-learning rendszerben." Production Systems and Information Engineering 11.2 (2023): 1-12.
Torrey, Lisa, and Jude Shavlik. "Transfer learning." Handbook of research on machine learning applications and trends: algorithms, methods, and techniques. IGI global, 2010. 242-264.
Vincze, Dávid, Alex Tóth, and Mihoko Niitsuma. "Antecedent redundancy exploitation in fuzzy rule interpolation-based reinforcement learning." 2020 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM).
Vincze, Dávid, and Szilveszter Kovács. "Fuzzy rule interpolation-based Q-learning." 2009 5th International Symposium on Applied Computational Intelligence and Informatics. IEEE, 2009.
Vincze, Dávid, and Szilveszter Kovács. "Rule-base reduction in Fuzzy Rule Interpolation-based Q-learning." Recent Innovations in Mechatronics 2.1-2. (2015): 1-6.
Watkins, Christopher JCH, and Peter Dayan. "Q-learning." Machine learning 8.3 (1992): 279-292.
