When Leo Wang arrived at Carnegie Mellon University from Hong Kong, he was already fascinated by robots. But it wasn’t until he joined the Robomechanics Lab led by Aaron Johnson(opens in new window) that his interest evolved into a mission: to teach quadrupedal robots to climb steep terrain using reinforcement learning.
“I’ve always dreamed of building general-purpose robots that could help in the home or navigate complex environments,” said Wang, now a sophomore in electrical and computer engineering. “Reinforcement learning felt intuitive, like how babies learn to walk through trial and error.”
A Summer Undergraduate Research Fellowship(opens in new window) (SURF) enabled Wang to begin training a robotic dog to climb simulated slopes as steep as 45 degrees. So far, his model has reached 36 degrees, and he’s optimistic about pushing further.
“There’s no hard-coded algorithm telling the robot how to move. It learns by itself, which is incredible to watch,” he said.
To achieve his goal, Wang built a custom simulation environment where the robot learns by trial and error. Each successful step earns a reward; each stumble is penalized. The process demands immense computing power, which the lab provides through access to high-performance machines.
“Reinforcement learning is computationally intensive,” Wang explained. “You need supercomputers to run thousands of iterations faster than real time.”
The challenge, he said, was designing the right rewards to motivate the robotic dog to climb properly and not exploit the simulator.
“I didn’t realize how hard it would be to build a realistic training environment. If the simulation isn’t accurate, the robot might learn behaviors that don’t transfer to the real world. Even small glitches can be exploited by the algorithm. For example, if he rewarded the robot dog according to its elevation from the ground, the robot might simply jump as high as possible and not learn to climb the hill in front of it at all. Thus, a better reward system would be one that gives higher rewards as the robot gets closer to the top of the hill,” he said.
Wang’s work is more than academic. He envisions real-world applications in search-and-rescue missions and environmental monitoring. Inspired by the agility of mountain goats, he hopes future robots will be able to traverse rugged terrain, identify footholds and operate autonomously in dangerous or remote areas.
“Robots can go where humans can’t. They can provide valuable help fighting forest fires, rescuing lost hikers or monitoring ecosystems from inaccessible cliffs,” he said.
His journey into reinforcement learning also reshaped his academic path. Initially focused on mechanical engineering, Wang switched majors after the fellowship gave him the chance to explore software and artificial intelligence.
“I spent a lot of time in high school building robots, but I hadn’t coded much. SURF opened the door to a whole new side of robotics for me,” he said.
Working in Johnson’s lab has been a source of immense fulfillment and inspiration for Wang.
“It’s an amazing feeling to be surrounded by people who share your passion,” he said. “We have weekly lab lunches and social events through which I learned so much from talking with Ph.D. students about their research and grad school experiences. I could see myself following a similar path.”
As for advice to students who are still thinking about applying for the fellowship, he said he would tell them to “go for it.”
“It’s a rare opportunity to dive deep into research and discover what you truly enjoy,” he said.
“Carnegie Mellon University is a private research university in Pittsburgh, Pennsylvania. The institution was originally established in 1900 by Andrew Carnegie as the Carnegie Technical School. In 1912, it became the Carnegie Institute of Technology and began granting four-year degrees.”
Please visit the firm link to site