Artificial Intelligence (AI) is slowly seeping into every aspect of our lives. And while the changes may not be prominent enough for all to see, you will notice small improvements that affect the overall experience. With the help of Machine Learning, researchers at Twitter have also made one some small, yet important tweaks to its photo-cropping AI Tool. Twitter will now be able to focus on the most important part of the photos in their newsfeed, and not just a random part of the image.
Like on all other social media channels, photographs and images have been an integral part of tweets as well. However, regular twitteratis can vouch for the fact that twitter hasn’t been the most friendly when it comes to highlighting images. We’re forced to overlook the slightly off-focus, oddly cropped thumbnails that appear on our newsfeed, and instead click to view the full image only if it’s an interesting read.
New Image-Cropping Method with Improved AI Features
Twitter has had the photo sharing feature since 2011, but the previous methodology used for image-cropping wasn’t viewer friendly. The reason for this was that Twitter’s previous AI was trained only for face detection, and if there weren’t any faces, it focused on the center of the image. This meant that any image with buildings or animals resulted in an off-focus thumbnail.
Recently, Twitter undertook some in depth research to overcome the limitations in its photo-cropping tool that led to awkwardly cropped preview images. Twitter researchers found two important AI tricks to detect the important parts of an image:
Identifying the Saliency of an Image
‘Saliency of an image’ is measured based on the part of an image that the human eye is most attracted to or that a person is most likely to focus on. The part of an image on which a person focuses is called a ‘highly salient region’. Academics have measured ‘saliency’ by using eye trackers that record those pixels that people fixated on while focusing on or looking at an image. People generally tend to focus on faces, text, animals and other objects and regions of high contrast. Using this data, neural networks and other algorithms can be trained to predict what people might like looking at.
Twitter will be using this feature to crop images so that it focuses on a region that the viewer will find most interesting. And twitter will do it at such speed that users will not experience any lag time while uploading images, everything will happen in real time.
Training Algorithms through Knowledge Distillation
The neural networks that deal with identifying the saliency of an image are usually too slow. As a result, these cannot be used for cropping the millions of images that are uploaded on twitter by the millisecond. So twitter found an alternative and made some optimizations to help its system perform 10 times faster that its standard method. This resulted in a highly advanced AI that can do intelligent cropping of photographs as and when they are being uploaded.
The development team at twitter used a technique called ‘knowledge distillation’. This enables to train their existing algorithms or data to quickly identify the most ‘salient’ parts of the photos or the area that catches the viewer’s eye the most.
Pruning of Images for Identification
The engineers at twitter also used a technique called ‘pruning’ to let the algorithm skip over features of the image that are not relevant for the identification. So, now twitter displays images that their previous algorithm could not detect properly, and these too in real time with better crops. This new AI tool also shows images of objects that were cropped previously as they could not ‘sit’ in the middle of the image but now appropriately cropped using the new tool. It also shows that the new AI feature recognizes text and can also adjust the crop to include a sign.
A Dramatic Feel to twitter Feeds
It’s a small yet important update that will dramatically improve the look and feel, and overall user experience of the twitter feed. This new update is being rolled out to all twitter users, and hopefully, it will put an end to all those awkwardly cropped thumbnail images.
This latest offering from twitter is now being available for twitteratis to use on desktop, iOS, and Android apps as reported by the company. So, next time when you use twitter, do not forget to thank the neutral networks for the enhanced images.