Some of you may be interested in becoming Reinforcement Learning engineers in the industry. Some of you might want to apply Reinforcement Learning in your existing company, start a new Reinforcement Learning based company or just use it in your hobby projects. Therefore, I thought it would be interesting to explore how Reinforcement Learning is currently being used in the industry, so that you get some ideas and inspiration.
Let's first talk about companies that have a proven Reinforcement Learning use case. In the lecture notes, you will find links to all the articles that I am going to show you in this lesson.
We start with Google and this is a picture of one its giant data centers. In 2016, Google deployed a Reinforcement Learning system to optimize the cooling of their data centers and found that it reduces their energy use by 40%, which is quite a significant number. In 2018, Google has so much confidence in this algorithm that it effectively handed control to the algorithm, which is managing cooling at several of its data centers all by itself.
In the next article, we see the entrance to JP Morgan, a financial giant. In 2017, the news got out that JP Morgan and Chase has completed a successful trial of a Reinforcement Learning bot for financial trading in Europe and will soon roll this out in production. The Reinforcement Learning bot, called LOXM, was more efficient than traditional methods of buying and selling assets.
In 2019, Microsoft announced the launch of Reinforcement Learning based services called Decision and Personalizer on Azure, its cloud computing arm. Personalizer is a reinforcement learning service for recommendations or personalization designed to improve user engagement, so you can imagine it being used in ecommerce, news, social media etc. In specific, use of Personalizer in Microsoft stores and online systems has led to performance improvements of more than 40% in some instances, said Microsoft VP Scott Guthrie, who leads the company’s Cloud and AI Group.
The next example is a company that helps in drug discovery via a complex AI toolchain. You can see the toolchain here, and you can see that they use Reinforcement Learning for molecular design. They have 150 academic and industry collaborators worldwide.
The last example involves Reinforcement Learning based robots working in the warehouse. The company who created this robot is called Covariant.AI and they finished their first commercial installations of its AI-equipped robots in 2020. The robot picks boxes and bags of products for a German electronics retailer called Obeta.
Fascinating, right? So these were the use cases where Reinforcement Learning is currently being used in the industry with good success.
Now for some less proven cases. This means that the next examples would involve companies trialing Reinforcement Learning use cases but do not have an active deployment.
The first example is the video game maker Electronic Arts or EA. If you have played video games, you know that the computer opponents are often a bit dumb. This happens because they still rely on scripts and random number generators to program computer opponents. EA is currently trying to make more intelligent opponents by using Reinforcement Learning. So beware, very soon, you might never be able to beat the computer at its own game.
The last example is a startup called Wayve that's making selef driving cars based on Reinforcement Learning. Like we discussed in CartPole-v0, the car learns by taking actions in a real world environment. The driver ends the episode if the AI makes fatal mistakes and the car tries to maximize the rewards it gets in one episode. You see that just after a day of training, the car can follow a real world lane rather well.
To summarize, Reinforcement Learning is used for a diverse range of use cases from energy optimization, finance, engagement optimization, drug discovery, robotics to video game development and self driving cars. And I am sure there will be more use cases in the future. Maybe you will make one of them?
In the last lesson, we saw some high level skills demonstrated by Reinforcement Learning agents from the academia. In this lesson, we are going to take a look at some Reinforcement Learning capabilities used in the industry.
Reinforcement Learning based bots are good at optimization problems. Google used such bots to reduce their cooling energy bills by about 40% in their large data centers, which you see in this image.
They can also trade stocks in the financial market. J.P. Morgan uses them and says that their Reinforcement Learning trader called LOXM is more much more efficient than traditional methods of buying and selling.
You know how websites show us personalized content based on browsing history, right? This is another skill that has been demonstrated by Reinforcement Learning agent. Microsoft used one called Personalizer to boost sales in Microsoft Stores by 40%.
In the next article, you see a Reinforcement Learning robot working in a warehouse of a German retailer Obeta. These robots, developed by a company called Covariant.AI, picks bags and packages of products and puts them in the right place.
The final skill that I am going to show is the ability of self driving. Here is a video of a Reinforcement Learning based self driving car developed by a British company called Wayve driving in different weather conditions.
Once again, I have put a link to everything I showed you in the lecture notes. If any of them catches your interest, you can read about them in more detail.
This brings us to the end of the first Chapter. In the next Chapter, we will learn how to get high scores in the CartPole-v0 environment.
Focus
Uses of Reinforcement Learning in the industry
Content
Some of you may be interested in becoming Reinforcement Learning engineers in the industry. Some of you might want to apply Reinforcement Learning in your existing company, start a new Reinforcement Learning based company or just use it in your hobby projects. Therefore, I thought it would be interesting to explore how Reinforcement Learning is currently being used in the industry, so that you get some ideas and inspiration.
Let's first talk about companies that have a proven Reinforcement Learning use case. In the lecture notes, you will find links to all the articles that I am going to show you in this lesson.
We start with Google and this is a picture of one its giant data centers. In 2016, Google deployed a Reinforcement Learning system to optimize the cooling of their data centers and found that it reduces their energy use by 40%, which is quite a significant number. In 2018, Google has so much confidence in this algorithm that it effectively handed control to the algorithm, which is managing cooling at several of its data centers all by itself.
In the next article, we see the entrance to JP Morgan, a financial giant. In 2017, the news got out that JP Morgan and Chase has completed a successful trial of a Reinforcement Learning bot for financial trading in Europe and will soon roll this out in production. The Reinforcement Learning bot, called LOXM, was more efficient than traditional methods of buying and selling assets.
In 2019, Microsoft announced the launch of Reinforcement Learning based services called Decision and Personalizer on Azure, its cloud computing arm. Personalizer is a reinforcement learning service for recommendations or personalization designed to improve user engagement, so you can imagine it being used in ecommerce, news, social media etc. In specific, use of Personalizer in Microsoft stores and online systems has led to performance improvements of more than 40% in some instances, said Microsoft VP Scott Guthrie, who leads the company’s Cloud and AI Group.
The next example is a company that helps in drug discovery via a complex AI toolchain. You can see the toolchain here, and you can see that they use Reinforcement Learning for molecular design. They have 150 academic and industry collaborators worldwide.
The last example involves Reinforcement Learning based robots working in the warehouse. The company who created this robot is called Covariant.AI and they finished their first commercial installations of its AI-equipped robots in 2020. The robot picks boxes and bags of products for a German electronics retailer called Obeta.
Fascinating, right? So these were the use cases where Reinforcement Learning is currently being used in the industry with good success.
Now for some less proven cases. This means that the next examples would involve companies trialing Reinforcement Learning use cases but do not have an active deployment.
The first example is the video game maker Electronic Arts or EA. If you have played video games, you know that the computer opponents are often a bit dumb. This happens because they still rely on scripts and random number generators to program computer opponents. EA is currently trying to make more intelligent opponents by using Reinforcement Learning. So beware, very soon, you might never be able to beat the computer at its own game.
The last example is a startup called Wayve that's making selef driving cars based on Reinforcement Learning. Like we discussed in CartPole-v0, the car learns by taking actions in a real world environment. The driver ends the episode if the AI makes fatal mistakes and the car tries to maximize the rewards it gets in one episode. You see that just after a day of training, the car can follow a real world lane rather well.
To summarize, Reinforcement Learning is used for a diverse range of use cases from energy optimization, finance, engagement optimization, drug discovery, robotics to video game development and self driving cars. And I am sure there will be more use cases in the future. Maybe you will make one of them?
Links
https://www.technologyreview.com/2018/08/17/140987/google-just-gave-control-over-data-center-cooling-to-an-ai/ https://www.ft.com/content/16b8ffb6-7161-11e7-aca6-c6bd07df1a3c https://venturebeat.com/2019/05/02/microsofts-azure-cognitive-services-adds-reinforcement-learning-for-recommendations-and-doodle-recognition-ai/ https://www.wired.com/story/ai-helps-warehouse-bots-pick-new-skills/ https://www.youtube.com/watch?v=eRwTbRtnT1I https://thenextweb.com/artificial-intelligence/2018/03/15/you-cant-play-eas-newest-game-because-youre-not-a-robot/ https://insilico.com/
Alternate Content
In the last lesson, we saw some high level skills demonstrated by Reinforcement Learning agents from the academia. In this lesson, we are going to take a look at some Reinforcement Learning capabilities used in the industry.
Reinforcement Learning based bots are good at optimization problems. Google used such bots to reduce their cooling energy bills by about 40% in their large data centers, which you see in this image.
They can also trade stocks in the financial market. J.P. Morgan uses them and says that their Reinforcement Learning trader called LOXM is more much more efficient than traditional methods of buying and selling.
You know how websites show us personalized content based on browsing history, right? This is another skill that has been demonstrated by Reinforcement Learning agent. Microsoft used one called Personalizer to boost sales in Microsoft Stores by 40%.
In the next article, you see a Reinforcement Learning robot working in a warehouse of a German retailer Obeta. These robots, developed by a company called Covariant.AI, picks bags and packages of products and puts them in the right place.
The final skill that I am going to show is the ability of self driving. Here is a video of a Reinforcement Learning based self driving car developed by a British company called Wayve driving in different weather conditions.
Once again, I have put a link to everything I showed you in the lecture notes. If any of them catches your interest, you can read about them in more detail.
This brings us to the end of the first Chapter. In the next Chapter, we will learn how to get high scores in the CartPole-v0 environment.