How to build a flight API: exploring our tech stack
The air travel industry is built on a complex web of legacy systems. Many of these systems have been around for decades and haven’t changed much.
In contrast, we joined the industry in 2017 and have grown quickly since then. Building a Flights API and related products from scratch gives us the opportunity to move faster and use the latest tools and systems.
Today, we’re an engineering team of more than 50 engineers in a company of nearly 100. Our tech stack represents our current business needs and sets us up for a future of disrupting travel.
A scalable infrastructure
Our customers are online travel sellers and we need to move quickly to meet and exceed their expectations. To do that, we need a technical infrastructure that lets us push code and new features to production without delays.
Continuous integration and continuous deployment
Working with capricious airline APIs and developing new features regularly means we need to deploy new versions of our code multiple times a day. To support fast iterations, we’ve built an efficient CI/CD pipeline.
Once a developer’s code has passed our test suite in CircleCI and has been approved by some of their colleagues they can merge their PRs. This merge will trigger our automatic deployment pipeline. It begins with a build of our production Docker container, which is then published to our private Google Container Registry. Once the image is available, we update the git tag which points to which commit should be running in production. ArgoCD picks up this update to deploy the corresponding container to our Kubernetes cluster managed with Google Kubernetes Engine.
This pipeline is designed for fast iterations and to keep our platform stable by blocking bad deployments with quality check gates.
Source-controlled infrastructure
Other than our CI/CD pipeline we define every piece of our infrastructure declaratively in our Terraform codebase, which is kept in a consistent state with Atlantis. We also keep an anonymised development replica of our database using Draupnir.
A back end built for consistency
Travel businesses face lots of ups and downs: seasonal changes in booking demand, unpredictable cancellations due to poor weather or staff shortages, and unexpected booking errors due to airline system maintenance or outages. We need a reliable back-end system to manage these operational difficulties and give customers consistency.
Core platform
Our backend is an Elixir umbrella project consisting of multiple isolated apps acting together to form the Duffel platform. Using an umbrella setup gives us the benefits of a monorepo: simpler code organisation and dependency management, better visibility, and a single build-and-deploy pipeline. It also gives us the benefit of mix build tooling which allows us to compile, test, and run all of the apps at once from the project root.
We use Postgres as our database – via CloudSQL on Google Cloud Platform – and PgBouncer to manage our database connection pool. For getting our bearings in the production database, we use SchemaSpy; and for all our analytical needs, Metabase is the tool of choice. It also comes with some neat data visualisation features to better understand product feature use.
When we want to test new features on the Duffel platform, we use feature toggles via an Unleash service. This is great for both gradual release and emergency control.
Observability
On the observability side, we use Prometheus and Thanos along with Grafana for metrics and reporting. If anything goes wrong in production, we get alerted via Sentry; and for urgent errors like major issues with our API, the on-call engineers will get a nice, friendly call from PagerDuty. To track and collaborate on incidents, we use incident.io directly in Slack, which is a great way to keep communication flowing and resolve things quickly.
Elixir libraries
While we aim to keep the number of external dependencies as low as possible, there are plenty of great libraries in the Elixir community which we’ve been very glad to use here at Duffel.
We use Phoenix’s routing and MVC functionality to marshal incoming API requests and serialise responses. An Ecto wrapper is used in all database interactions for its powerful query, schema definition, and data validation capabilities.
Our airline search workers use Broadway to pull requests from a work queue and execute them; and, to parse the tonnes of XML we receive in response, we use a combination of Saxy, SweetXml, and DataSchema which was written and open-sourced by Duffel engineer Adam Lancaster.
Swoosh is used for both internal and external mailing which was originally created by our CEO, Steve Domin and is now maintained by the community. We use the Unleash client library to check feature flags on the Unleash service.
To keep things consistent across the codebase, which is especially important as Duffel’s team grows, we use a combination of Credo to enforce good Elixir coding practices and mix format to enforce consistent style.
Testing
Testing is a core part of software development at Duffel. At the bottom of the testing pyramid, we use the ExUnit unit test framework and ExMachina for creating test data. We also use Hammox which was written and open-sourced by one of our engineers, Michał Szewczak, for generating test mocks which ensure consistency between behaviour typespecs and mock definitions.
Further up in the testing pyramid, we have an Elixir app containing end-to-end tests (named duffel_onera after the French aerospace lab) which checks booking flows with airline sandboxes to ensure service stability.
Finally, right at the peak of the pyramid, we have a great setup for local manual testing with Docker which is all we need to test features in most cases. We also use Draupnir which gives us real, anonymised, production data to test in a safe and risk-free environment.
When local testing doesn’t quite cut it for the more complex changes, we can deploy to our staging environment to be sure they’re working correctly, for example adding some new Kubernetes config.
A fast and accessible front end
Websites in the airline content distribution industry aren’t known for their polished user experience (UX), they often focus on function over usability. At Duffel, we offer a best-in-class UX that far exceeds others in the industry. To make this happen, we use a modern front-end tech stack to build and ship high-quality experiences quickly.
Programming languages and frameworks
We use TypeScript for all our front-end code, combined with React and Next.js. We’re really happy with Next as a tool as it abstracts away a lot of boilerplate for us.
For styling, historically we’ve used a mixture of CSS modules, styled-jsx, and Tailwind, but we quickly realised that we should standardise on one technology – we're in the process of migrating everything to CSS modules. So far we’ve completely removed Tailwind and our rule with components using styled-jsx is ‘if you touch it, migrate it’, which is helping us to migrate incrementally without grinding our delivery to a halt.
Front-end tooling
On the tooling side, we’re using Webpack (although Next handles most of our tooling for us) plus ESLint and Prettier for code formatting. We write snapshot tests using Jest and React Testing Library and end-to-end tests with Puppeteer.
We use Storybook as a source of truth for our components, and we subscribe to Chromatic which gets us visual testing with little effort.
We’ve also recently started a design system so that we can reduce duplicated effort across our front ends.
Collaborating with other teams
All our apps are hosted on Google Cloud Platform where we work closely with our colleagues on the infrastructure side.
Naturally, our front-end developers work very closely with the design team, who use Figma. We’re explicitly opposed to silos and ‘throwing things over the wall’ so the design team are all comfortable using GitHub, pull requests, and contributing code. Similarly, our front-end developers are comfortable using Figma and working with designers to find the best solution to a problem.
Join the Duffel team
Our engineering team is built on the principle of ‘small things compound’. We believe in working together to deliver high-quality work daily to create something bigger than the sum of its parts.
Read more engineering blog posts and explore open job opportunities. If we don’t have any new opportunities posted, feel free to submit an open application and we’ll send you a message if something relevant comes up.