Yasin Engin Go Backend
Tolerex: Replicated Storage Prototype in Go
A learning project for practicing replication, secure node traffic, member detection, persistence, and metrics.
Problem
Tolerex explores how a replicated store behaves when a member becomes unavailable or a follower falls behind. Its present scope uses one fixed leader. Leader election and leader failover are not implemented, so the project is not presented as a generally fault-tolerant storage system.
Architecture
The system is organized around a leader node, follower nodes, a client-facing API, a heartbeat loop, and a persistent log/checkpoint path. The design keeps health monitoring separate from request handling so failures can be detected without blocking normal traffic.
Technologies
- Go for service implementation and concurrency control.
- gRPC for explicit service contracts between components.
- mTLS for authenticated node-to-node communication.
- Disk-backed checkpoints and logs for restart behavior.
What I Built
- Leader/follower replication flow with heartbeat-based health checks.
- Heartbeat-based member detection and crash-aware reads that can fall back to available replicas.
- Persistence layer for data and log checkpoints.
- Basic observability points for metrics, logs, and recovery timing.
Screenshots / Diagrams
GitHub Repository
What I Learned
- Separating failure detection from client request paths makes behavior easier to test.
- Distributed systems code needs simple, visible state transitions more than clever abstractions.
- Secure transport should be designed early because it affects local development, certificates, and deployment habits.
Future Improvements
- Add leader election before testing leader-loss recovery claims.
- Add repeatable tests for member loss, follower lag, and network partitions.
- Extend Prometheus metrics and add a small dashboard for replication and recovery timing.
- Document benchmark scenarios with dataset size, request pattern, and recovery target.