A Q-learning algorithm to generate shots for walking robots in soccer simulations

Credit: CM, Unsplash

RoboCup, originally called the J-League, is an annual robotics and artificial intelligence (AI) competition hosted by the International RoboCup Federation. During RoboCup, robots compete with other robot soccer tournaments.

The idea for the contest came about in 1992, when Professor Alan Mackworth of the University of British Columbia in Canada wrote a paper entitled ‘On Seeing Robots’. In 1993, a research team in Japan took inspiration from this article to organize the first robot soccer competition.

While RoboCup can be very entertaining, its main purpose is to showcase the advancements in robotics and AI in a real world. The robotic systems participating in the competition are the result of intensive research efforts by many researchers worldwide.

In addition to the real competition, computer scientists and roboticists can test their robot soccer calculation tools in the RoboCup 3D soccer simulation competition. This is essentially a platform that replicates the RoboCup environment in simulation and serves as a virtual “gym” for AI techniques and robotic systems designed for playing soccer.

Researchers from the Yantai Institute of Technology in China and the University of Rahjuyan Danesh Borazjan in Iran recently developed a new technique that could improve the ability of robots participating in football matches to shoot the ball while walking. This technique, presented in a paper published in Springer Link’s Journal of Ambient Intelligence and Humanized Computing, is based on a computational approach known as the Q-learning algorithm.

“One of the main goals of the teams participating in the RoboCup3D competition is the ability to increase the number of shots,” Yun Lin, Yibin Song and Amin Rezaeipanah, the three researchers who developed the technique, wrote in their paper. “The reason for this importance is that superiority over the opponent requires a powerful and accurate shot.”

Most techniques to generate shots in simulation are based on two approaches called inverse kinematics (IK) and point analysis. These are mathematical methods that can be used to create computer animations as well as in robotics to predict the joint parameters a robot needs to reach a certain position or perform an action.

“The assumption of these methods is that the positions of the robot and the ball are fixed,” the researchers explain in their paper. “However, this is not always the case for shooting.”

To overcome the limitations of previously proposed methods, Lin and his colleagues created a new shooting strategy based on a Q-learning algorithm, which can improve robots’ ability to shoot the ball while walking. Q-learning algorithms are model-free computational approaches based on reinforcement learning. These algorithms are particularly useful in cases where agents are trying to learn how to best navigate their environment or perform complex actions.

“A curved path is designed to move the robot toward the ball so that it is ultimately in an optimal position to shoot,” the researchers wrote in their paper. “In general, the vision receptor in RoboCup3D has noise. Therefore, robot movement parameters such as speed and angle are adjusted more accurately by the Q-learning algorithm. Finally, when the robot is in the optimal position relative to the ball and the target, the IK module is applied to the shooting strategy.”

Lin, Song and Rezaeipanah evaluated their Q-learning algorithm in a series of experiments and simulations. Remarkably, they found that robots could shoot the ball much better while walking than robots in most of the teams participating in the RoboCupSoccer competition and in Iran’s RoboCup3D competition. So in the end, it could significantly improve the performance of robots during RoboCup football matches.


A heuristic search algorithm to plan attacks in robot soccer


More information:
Generate running shooting for soccer simulation 3D competition using Q-learning algorithm. Journal of Ambient Intelligence and Humanized Computing(2021). DOI: 10.1007/s12652-021-03551-9

© 2021 Science X Network

Quote: A Q-learning algorithm to generate shots for walking robots in football simulations (2021, November 25) Retrieved November 25, 2021 from https://techxplore.com/news/2021-11-q-learning-algorithm-shots- robots- football.html

This document is copyrighted. Other than fair trade for personal study or research purposes, nothing may be reproduced without written permission. The content is provided for informational purposes only.

Leave a Comment