Summary:
- A new report highlights that bad bots account for 43% of web traffic within the airline sector, double the estimated average level across other industries;
- The Distill Research Lab Threat Research: How Bots Affect Airlines, provides a first industry-specific study of the damage caused by bad bots on the airline sector;
- While not all bots are bad and many are highly welcomed by companies, there are others that are unwanted and dangerous to the success of an organisation.
Bad bots obviously pose problems for every business with an online presence. Every website, mobile app, and the APIs that power them, are attacked by bots around the clock. According to the Distil Networks' annual Bad Bot Report, only 57.8% of web traffic is humans-the rest is bots. While not all bots are bad and many such as search engines are highly welcomed by companies, there are others that are unwanted and dangerous to the success of an organisation.
These comprise 21.8% of all web traffic, according to the Bad Bot Report. However, the new industry study highlights that the bot problem is considerably bigger issue for airlines. For the full calendar year of 2017, the proportion of bad bots amongst airlines was 43.9%, the study shows. Only one industry, Gambling, had a higher proportion of bad bot traffic.
The problem is bots are an essential tool within the airline industry to allow online travel agencies and aggregators to access latest flight and ticket information. While, this is usually done within fixed agreeable terms, it does open the door for unlawful access including unauthorised scraping, seat spinning (a particular problem in Asia Pacific where seats can be held for longer periods without payment), loyalty programme account takeover, and fraud.
CHART - Only one industry, Gambling, had a higher proportion of bad bot traffic than airlines
Source: Distill Research Lab's Threat Research: How Bots Affect Airlines
"Many of these problems alone are enough to have a significant financial impact on the airline. But collectively, these bot activities can add up to a significant headache for the business and especially the IT team, and left unaddressed may lead to poor website performance and even downtime," says the report.
Its analysis of 180 domains and 100 airlines over a 30-day period across Jul-2018 and Aug-2018 covered 7.4 billion individual requests. Shockingly, it discovered that humans accounted for just 5.42% of one European airline's total traffic - the remaining 94.58% was bots! On 51 of the domains, bots accounted for greater than 50% of all traffic-80% of these were from medium and large traffic sites. And comparing to the 21.8% global bad bot average across all industries, 94 airline domains exceed this average proportion of bad bot traffic.
The report highlights that nearly a third (31.40%) of bots on airline sites were classified as sophisticated. Only 15.70% were described as simple bots with the remaining (52.90%) showing moderate sophistication. Again, the level of sophisticated bots was up significantly on the all industries average of 19.70%. This, the report says, is explained by the arms race at play between bot operators and bot detection technology.
"Once bots are detected and blocked, the challenge to the bot operator is to create another bot to achieve the same goal. Because the financial viability of unauthorised OTAs and aggregators is based upon bots scraping airline data, the cycle continues ad infinitum," it says.
The bots generally claim a user agent associated with a desktop browser. Just 10.50% of bots on airlines identify as a user agent from a mobile device. While this proportion of mobile impersonators is currently small, the report says "it is consistently growing and this trend is expected to continue". The insight shows bad bots identified themselves as one of 270 unique user agents, almost half (48.87%) of all bad bots claim to be Chrome, hiding in plain sight by impersonating the most popular browser.
LEARN MORE... Download the full report...
Threat Research: How Bots Affect Airlines
USA is the leading source of bad bots on airlines responsible for 25.58% of this traffic. Singapore is in second place with 15.21% and China is third with 11.51%. Reflecting the global distribution of airlines, OTAs and aggregators, the number of countries hosting bot traffic is "high", says the report, and is "spread out across every region of the world".
Interestingly, the report highlights the consistency of bad bot traffic on airlines across a week period. In general, they are consistent in volume every day except for Friday where there is a peak of bad bot traffic at 18.17%. The Friday peak is seen across other industries too, but is much more pronounced with airlines. This can be explained by some airlines offering discounts on fares on Friday's and bots increasing activity to gather any new information.
It is clear that bots are a major issue for the airline sector. Even those sent from OTAs, aggregators and competitors to scrape prices and flight details are in themselves not malicious but the sheer volume of them on an airline website creates a multitude of business problems. So, what can be done? Well, as the report acknowledges, unfortunately, every site is targeted for different reasons, and usually by different methods, so there is "no one-size-fits-all bot solution".
It does however, suggest some proactive steps to start addressing the problem, such as to block or captcha outdated user agents/browsers; block known hosting providers and proxy services; block all access points; carefully evaluate traffic sources; investigate traffic spikes; monitor for failed login attempts; pay close attention to public data breaches; and evaluate a bot mitigation solution.