Building a Kafka Client in Rust


In my last post, we looked at the Rust library tokio, how its components fit together, and how we can use it to build a toy server. This time around, we'll go a step further and use the libary to build something a bit more useful: a client for Apache Kafka.

If you haven't worked with Kafka, you can think of it basically as an append-only log. It also happens to be distributed, partitioned, replicated, etc, but we're going to ignore all that for the time being and work with a single local instance.

We'll be focusing on the two fundamental operations you do with Kafka: produce (write to the log) and fetch (read from the log).