Explained: The Technology implementation behind launching Social Features [Zomato shares it all]
[Editorial notes: Zomato recently launched social features on the site and Ram Singla, Zomato CTO shares how the company implemented social features.]
It took us just over 2 weeks to enable socialfeatures on Zomato. The journey was packed with rapid iterations, complex engineering challenges and intense coding sessions.
There are two core challenges in adding social features to any app. The first challenge, is to engineer a platform that manages the interconnections between the users and disseminates the information between the connected users realtime. This challenge becomes harder and harder as the number of active users on the site and the interconnections increases. The second challenge, is to create an engaging user experience which makes it easy for user to connect, share and consume social information.
To engineer the social platform we kept our strategy simple and stupid: whenever possible we used a proven technology and used the right tool for the right job. Our stack for the social platform is amalgamation of several open-source technologies – Redis, MySQL, Krati Data Store, Apache Thrift, JavaEWAH Compressed Bitmaps, Faye, JRuby, NodeJS and PHP.
Redis servers collates realtime information about user activity on the site and disseminates that information down the social graph realtime. We store user activity list on per user basis in Redis. Redis notifies our PHP workers when the activity list is updated. PHP worker on update – find all the online subscribers of the activity; update their notifications list in Redis and publish messages to their channels on Faye Server. This architecture lets us perform lazy computation and help us save computation power for disseminating realtime information.
Apache Thrift, Krati Data Store, JavaEWAH Compressed Bitmaps and JRuby forms the part of our remote service which stores our social graph in high-performing persistant compressed bitmap format. Some of the highlights of this service are compact bitsets with extremely fast bitwise operations e.g. A random 10K cardinality bitset for arbitrary 32-bit integers takes less than 2MB of RAM compared to 512MB RAM consumed by uncompressed bitmap; can perform 10000 sets per second and 50000 membership checks per second on an index; can load a million cardinality index in less than 650 ms; can persist the same in 1500 ms and can traverse the bitset in less than 250 ms.
Faye messaging server running on Node.js help us deliver realtime notifications to the logged-in users in a predictable and consistent manner. Low latency and low overhead of Faye server helps us save hundred GBs in bandwidth. Our messaging setup consists of a master and two workers which load balance the subscribers. MySQL servers functions as the persistent store for all the user activity and user connections. We do not store transitional data in MySQL and use Redis for the same.
To provide the engaging user experience, we have baked Facebook deeply into our application. It takes less than 10 second to connect your Facebook account to the Zomato account and getting connected to your friends on Zomato. Eating out is social activity and sharing your experience with your friends provides an excellent story for your Facebook Timeline. Just by enabling one setting on your profile, your activity gets automatically posted onyour Facebook Timeline. On top of this, we provide both push and pull mechanisms to consume information from your network. When you search we seamlessly integrate recommendations from your network. And we push realtime notifications to keep you up to date with your network.
Recommended Read: How to: Build a Large and Scalable ECommerce Platform [by Amitabh Misra, Vice President and Head of Engineering at Snapdeal.com.]