The Road to the Busbud Business API and Beyond

Today we’re proud to announce the release of the Busbud Business service. We’re really excited to open up solutions such as our realtime API and whiteprogram to partners around to world and enable them to connect and integrate our massive bus inventory.

 

 

For the Busbud engineering team, tasks and challenges involved in making this a reality started 8 months ago. In this article, you’ll get an overview of our technological stack and the challenges we faced building the API.

Building the next evolution of the bus industry
Busbud’s founding mission was to gather all the world’s bus information in one place. Since 2011, we’ve worked hard to integrate hundreds of bus operators into our partner network to offer travelers the most options when it comes to bus travel. As a result, Busbud today has the largest sellable bus inventory in the world.

Until recently, the only way to access this data was through our website. Once our mobile apps became available in 2015, travelers could also search and book bus tickets from their phone. Other teams at Busbud started playing around with our API to build side projects, like Bus vs Plane or the Dollar Bus Club. But it felt like something was missing.

Major travel booking sites offered flights, hotels, car rentals and even cruises, and yet, no bus, a mode of transport that we knew millions of people were using around the world each day. In order to truly make bus travel more accessible and to make full use of our inventory, we started planning Busbud Business.

Connecting businesses to the Busbud inventory

It’s no secret that APIs are a major aspect of the B2B world nowadays. Almost every major web company has an API that allows developers and other businesses to build products on top of their data and processing power.

In travel, companies like Expedia, Skyscanner and Google Flights all have APIs that power other travel websites with their flight and hotel inventory. But none of these companies powered bus.

So we set out to build a global API that would:

• Give individuals and businesses alike access to bus schedules, fares and ticket bookings from our inventory
• Be a well documented and complete set of APIs that gives developers access to modern technology
• And finally, provide a simpler integration process than connecting to individual bus operators one at a time

Let’s just say that we’ve just about seen it all when it comes to integrating with bus companies directly. While there are some standards in the flights industry such as the OpenTravel Alliance, such global and accepted standards don’t exist yet for the bus industry.

Our API allows businesses to avoid dealing with all the custom web services and formats and quickly integrate with Busbud knowing we always have ease of use and consistency in mind. We already did all the heavy lifting and continue to do so by integrating new operators on a daily basis.

 

 

Technology Stack Overview

Over the years we tried multiple languages, frameworks and technologies before getting to our current stack. We’re really happy with the decisions we made in the past years and believe we made the right technology choices.

Both of our backend and frontend stacks are developed in Node.js. Node is a great runtime for web development and using the same language for both backend and frontend development allows developers to jump from a project to another without too much effort. Given that our service spends most of its time fetching and formatting remote data (either from our departure cache or from our suppliers’ sites or APIs) Node’s event-driven and non-blocking model is a perfect fit. We stay up-to-date with the latest releases and use ES6 features as much as possible.

Our backend is based on a core API and a multitude of microservices. We love the flexibility and development efficiency microservices allow. We’re also able to build and deploy new services and features at record speed. We use HTTP and message queues for communication between these services.

Furthermore, we now maintain a fleet of Docker containers to connect with our supply partners. Using containers when integrating new partners allows us to use the right technology for each partner and easily scale resources for more popular partners. We’ll cover this part of our infrastructure in another blog post soon.

For data storage, we opted for PostgreSQL and Redis as our main data stores. We’re really happy with both and feel they’re great products with great communities.

We also maintain a database of every single departure that has recently passed through our system, and it’s now grown to over 150 million archived departures! This database has been a game changer for our marketing and analytics teams allowing them to complete extensive market analysis and provide powerful insights to our users, to the industry and to the media.

Finally, we use Fastly as our CDN provider. We love their product and try to use it as much as possible to cache content for both the API and the website. We use some of their unique features such as edge dictionaries to route our traffic effectively. We also instrument every request allowing us to better monitor and serve our users. Their great network, speed and reliability allow us to serve users and API consumers with a fast experience anywhere in the world.

We believe that our current stack is built on strong and modern technologies that will allow us to scale for years to come. Nevertheless, we keep an open mind and if a new proven and mature technology that is better suited to our own and our partners’ needs shows up, we’ll take a look and upgrade as we see fit.

 

Business API Challenges

Opening up our data and APIs to external partners brought some new and exciting challenges to our team. We had to make the right decisions from the get-go as we would not be able to make breaking changes to our API after releasing it to our first partners.

Caching

We introduced caching a long time ago at Busbud, but during this project, we definitely wanted to overhaul the system to support the expected increase in traffic and searches.

Our goal is always to serve departures with data as fresh as possible but we also need to protect our supply partners from the traffic we’re getting. We handle a lot more requests than most of the operator’s web services we deal with. To protect our partners’ services from these traffic surges, we have to make smart decisions about when we ask them for fresh data and when we can serve data we have in cache. Our system optimizes to improve the booking ratio to partners’ services.

To help us find the right settings for each data source we connect to, we relied on the departures archive mentioned earlier to build a statistical model representing the occurrence and frequency of price and availability changes per operator. This allows us to leverage usage patterns, not intuition, to consistently serve the right information.

Different search patterns

Another challenge for us in terms of caching was the variety of search patterns of our partners. On Busbud.com, our users usually buy their tickets within a few days of the departure date so most of the searches are within that time frame.

Some of our partners display bus results alongside flights results. The flight searches have different patterns, with people searching weeks and months in advance. To increase the cache hit ratio, we started sharing cache entries at the route level allowing searches made by Busbud or any partner to share cached data. We also have complex invalidation logic to quickly refresh data once we detect it is no longer valid. We are constantly improving our caching model and pushing the limits of what our CDN can do to make our data available globally with low latency.

Performance

Our team of Node.js developers work on many different projects, and as such, their collective experience was valuable in optimizing the Node based APIs. We use best practice debugging techniques such as CPU and memory profiling to identify the bottlenecks in our Javascript code.

We’ve created a set of tools to run distributed load tests against our production environment on demand. These tests measure the impact of small and big changes against our production infrastructure with real scenarios and at a large scale. Our API team always has performance in mind when it comes to new features and improvements. With this mindset, we’re able to serve partners with results as fast as possible with high availability.

We faced many other challenges that we will cover at another time. For example, our logging and monitoring infrastructure went from using a couple tools to being able to track every action made by a user or API partner. We also believe we should be the first ones to know when an operator or our own platform is having issues and invest regularly in our fast and efficient alerting.

Moving Forward

While increasing our supply remains one of our top priorities, we’re excited to open the Busbud Business service to the world. I hope this overview of our Busbud Business products gives you more context on the efforts that went into building a great experience for our partners. We’re excited to have more and more partners joining our network and hope you’ll be one of them!

 
Read more about the Busbud Business launch:

• Busbud Launches Bus Booking B2B Platform, Adds Former Expedia Managing Director as COO
• Busbud Business microsite
• Introducing our new COO, Sean Shannon by LP Maurice (CEO)
• About Busbud
• Contact: pr@busbud.com
• Work with Christophe at Busbud!