Brewing Espresso At Linkedin

Whitepaper Detailed Summary

Before Espresso World

In the early days of LinkedIn, the data system was quite simple. It mainly used a Relational Database Management System (RDBMS), which is a type of database that organizes data into tables and supports data schema (the structure of the data), rich transactions (ensuring data consistency during updates), and can work at a large enterprise scale.

Here’s how it worked:

1. User Data: All the important user information like profiles, connections, etc., was stored in a single RDBMS with just a few tables.
2. Specialized Systems: In addition to the main RDBMS, LinkedIn had two other systems to handle special tasks.

• Full-text Search: One system allowed users to search through all the profiles in a fast and efficient way.
• Relationship Graph: Another system helped in efficiently tracking and traversing the complex web of user connections (like friends and contacts on LinkedIn).

3. Keeping Systems Updated: These two specialized systems needed to stay up-to-date with the latest information in the RDBMS. To do this, LinkedIn used a tool called Databus.

• Databus captured every change made in the RDBMS and sent those changes, in the exact order they happened, to the search and connection systems.
• This ensured that when a user updated their profile or added a new connection, both the search…