Designing a Micro-blogging Platform — Scalable, Available, and Consistent

Introduction: 

Microblogging platforms have become a crucial part of our online social interactions. In this blog post, we will delve into the design considerations and architecture for creating a highly scalable, available, and consistent micro-blogging platform to meet the demands of a large user base.

Functional Requirements:

1. User Actions:
1. Users can create public or followers-only posts.
2. Following other users.
3. Like and comment on posts.
4. Reply to comments.
5. Edit and delete your own comments.
6. Sort comments based on time or likes.
7. View trending tweets/posts.

Non-functional Requirements:

1. High Availability:
1. The system should be available 24/7 to accommodate users from different time zones.
2. Low Latency:
1. Users should experience minimal latency when creating and viewing posts.
2. Eventual consistency is acceptable for some features.

Capacity Estimations:

1. Daily Active Users (DAU):
1. 10 million users.
2. Estimated Queries Per Second (QPS): 115.
2. Posts:
1. 5% create posts, 30% engage with posts, and the rest are viewers.

Storage Estimations:

• Post Storage:
  • 5TB per day.
  • Total storage for 5 years: 9125TB.

• Post Storage:
  • 5TB per day.
  • Total storage for 5 years: 9125TB.

Database Design:

1. User Table (SQL):
1. Unique identifier (Mobile/Email).
2. First/Last Name.
2. Post Table (SQL)
1. Post ID.
2. User ID.
3. Comment ID.
4. Media link (Stored in GCS/S3)
3. Comment Table (SQL):
1. Comment ID.
2. User ID.
3. Comment content.
4. Reply Table (SQL):
1. Reply ID.
2. Comment ID.
3. Reply content.
5. Connection Table (SQL):
1. User ID.
2. Followed User ID.
6. Like Table (HBase):
1. User ID.
2. Post ID.
3. Like count.

Data Storage:

1. GCS/S3:
1. For static media associated with posts.
2. Feed DB:
1. Batch job-based storage for the user feed.

APIs:

1. Create Post:
1. Endpoint: /create_post
2. Parameters: Token, Message, Media.
3. Returns: JSON.
2. Follow:
1. Endpoint: /follow
2. Parameters: Token, UserID.
3. Returns: Boolean.
3. Like:
1. Endpoint: /like
2. Parameters: Token, PostID, Count.
4. Comment:
1. Endpoint: /comment
2. Parameters: Token, PostID, Comment.
5. Reply:
1. Endpoint: /reply
2. Parameters: Token, CommentID, Reply.
6. Update:
1. Endpoint: /update
2. Parameters: Token, PostID, Operation (Edit/Delete).
7. Get Feed:
1. Endpoint: /get_feed
2. Returns: JSON (trending with posts).
8. Search:
1. Endpoint: /search
2. Returns: User/Post information.

High-Level Design:

Conclusion:

Designing a micro-blogging platform involves meticulous planning to handle the immense user base and their interactions. By incorporating scalable database structures, distributed storage, and efficient APIs, we can create a system that fulfills the functional and non-functional requirements, providing users with a seamless and engaging experience.