Kafka Command Generator

Quick Command Generator

Frequently Used Commands

Topics

Topic management and operations

kafka-topics --bootstrap-server localhost:9092 --list
List all topics
kafka-topics --bootstrap-server localhost:9092 --describe --topic my-topic
Describe topic details
kafka-topics --bootstrap-server localhost:9092 --create --topic my-topic --partitions 3 --replication-factor 1
Create new topic

Console Consumer

Consume messages from topics

kafka-console-consumer --bootstrap-server localhost:9092 --topic my-topic --from-beginning
Consume from beginning
kafka-console-consumer --bootstrap-server localhost:9092 --topic my-topic --group my-consumer-group
Consume with consumer group
kafka-console-consumer --bootstrap-server localhost:9092 --topic my-topic --property print.key=true --property key.separator=:
Consume with key-value display

Console Producer

Produce messages to topics

kafka-console-producer --bootstrap-server localhost:9092 --topic my-topic
Produce messages interactively
kafka-console-producer --bootstrap-server localhost:9092 --topic my-topic
Produce single message
kafka-console-producer --bootstrap-server localhost:9092 --topic my-topic --property parse.key=true --property key.separator=:
Produce key-value messages

Consumer Groups

Consumer group management

kafka-consumer-groups --bootstrap-server localhost:9092 --list
List all consumer groups
kafka-consumer-groups --bootstrap-server localhost:9092 --describe --group my-consumer-group
Describe consumer group
kafka-consumer-groups --bootstrap-server localhost:9092 --reset-offsets --group my-consumer-group --topic my-topic --to-earliest --execute
Reset offsets to earliest

Advanced Commands

Monitoring & Debugging

kafka-get-offsets --bootstrap-server localhost:9092 --topic my-topic
Get topic offsets
kafka-configs --bootstrap-server localhost:9092 --entity-type topics --entity-name my-topic --describe
Describe topic configs
kafka-leader-election --bootstrap-server localhost:9092 --election-type PREFERRED --all-topic-partitions
Trigger leader election

Performance Testing

kafka-producer-perf-test --topic my-topic --num-records 1000000 --record-size 1000 --throughput 10000 --producer-props bootstrap.servers=localhost:9092
Producer performance test
kafka-consumer-perf-test --bootstrap-server localhost:9092 --topic my-topic --messages 1000000
Consumer performance test

What is Apache Kafka?

Apache Kafka is a distributed event streaming platform capable of handling trillions of events a day. It is designed for high-throughput, fault-tolerant handling of real-time data feeds. Kafka is used for building real-time data pipelines and streaming applications. It combines the benefits of traditional messaging systems with the scalability and fault tolerance of distributed systems.

What are Kafka Tools?

Kafka Tools are command-line utilities that come with Apache Kafka for managing and monitoring Kafka clusters. These tools provide functionality for topic management, consumer group operations, performance testing, and cluster administration. They are essential for developers and operators working with Kafka clusters.

How to use this Kafka Command Generator

  1. Quick Command Generator: Use the input fields at the top to set your bootstrap server, topic name, and consumer group. All commands will automatically update with your values.
  2. Frequently Used Commands: Browse through categorized command cards (Topics, Console Consumer, Console Producer, Consumer Groups) to find the command you need.
  3. Dynamic Substitution: Commands automatically replace placeholders with your input values. For example, --bootstrap-server {bootstrap-server} becomes --bootstrap-server localhost:9092 when you enter "localhost:9092" in the bootstrap server field.
  4. Copy Commands: Click the copy button next to any command to copy it to your clipboard for immediate use in your terminal.
  5. Advanced Commands: Access monitoring, debugging, and performance testing commands for more complex Kafka operations.

Common Kafka Command Patterns

Basic Syntax: kafka-[tool] --bootstrap-server [server] [options]
Topic Operations: kafka-topics --bootstrap-server [server] --[action] --topic [topic]
Consumer Operations: kafka-console-consumer --bootstrap-server [server] --topic [topic]
Producer Operations: kafka-console-producer --bootstrap-server [server] --topic [topic]
Consumer Groups: kafka-consumer-groups --bootstrap-server [server] --group [group]
Performance Testing: kafka-[producer|consumer]-perf-test --bootstrap-server [server] --topic [topic]

Essential Kafka Concepts

  • Topics: Categories or feed names to which messages are published
  • Partitions: Topics are divided into partitions for parallel processing
  • Producers: Applications that publish messages to topics
  • Consumers: Applications that read messages from topics
  • Consumer Groups: Groups of consumers that work together to consume messages
  • Brokers: Kafka servers that store the data and serve client requests
💡 Pro Tips
  • • Use --from-beginning to read all messages from the start
  • • Add --property print.timestamp=true to see message timestamps
  • • Use --property key.separator=: for key-value message handling
  • • Monitor consumer lag with kafka-consumer-groups --describe