Kafka Installation Steps
Let's Begin........
Step 1: Download latest version from here
(venv) rajesh@rajesh:~/Rajesh/Softwares$ wget https://downloads.apache.org/kafka/3.3.1/kafka_2.13-3.3.1.tgz
...........
kafka_2.13-3.3.1.tgz 100%[=======================================================================>] 100.19M 5.33MB/s in 28s
2022-11-05 10:56:04 (3.54 MB/s) - ‘kafka_2.13-3.3.1.tgz’ saved [105053134/105053134]
(venv) rajesh@rajesh:~/Rajesh/Softwares$ ls -tlrh
total 101M
-rw-rw-r-- 1 rajesh rajesh 101M Oct 3 04:35 kafka_2.13-3.3.1.tgz
Step 2: Unzip the downloaded tgz file
(venv) rajesh@rajesh:~/Rajesh/Softwares$ tar -xzf kafka_2.13-3.3.1.tgz
(venv) rajesh@rajesh:~/Rajesh/Softwares$ ls -ltr|grep kafka
drwxr-xr-x 7 rajesh rajesh 4096 Sep 30 00:36 kafka_2.13-3.3.1
-rw-rw-r-- 1 rajesh rajesh 105053134 Oct 3 04:35 kafka_2.13-3.3.1.tgz
(venv) rajesh@rajesh:~/Rajesh/Softwares$ cd kafka_2.13-3.3.1/
Step 3: Start Kafka with ZooKeeper
Start the ZooKeeper service
(venv) rajesh@rajesh:~/Rajesh/Softwares/kafka_2.13-3.3.1$ bin/zookeeper-server-start.sh config/zookeeper.properties
...............
[2022-11-05 11:11:19,291] INFO binding to port 0.0.0.0/0.0.0.0:2181 (org.apache.zookeeper.server.NIOServerCnxnFactory)
Start the Kafka broker service
rajesh@rajesh:~/Rajesh/Softwares/kafka_2.13-3.3.1$ bin/kafka-server-start.sh config/server.properties
................
[2022-11-05 11:15:24,764] INFO Initiating client connection, connectString=localhost:2181 sessionTimeout=18000 watcher=kafka.zookeeper.ZooKeeperClient$ZooKeeperClientWatcher$@495ee280 (org.apache.zookeeper.ZooKeeper)
................
[2022-11-05 11:15:24,795] INFO Socket connection established, initiating session, client: /127.0.0.1:43696, server: localhost/127.0.0.1:2181 (org.apache.zookeeper.ClientCnxn)
[2022-11-05 11:15:24,980] INFO Session establishment complete on server localhost/127.0.0.1:2181, session id = 0x100036fac160000, negotiated timeout = 18000 (org.apache.zookeeper.ClientCnxn)
.................
[2022-11-05 11:15:27,503] INFO Awaiting socket connections on 0.0.0.0:9092. (kafka.network.DataPlaneAcceptor)
................
[2022-11-05 11:15:28,722] INFO [BrokerToControllerChannelManager broker=0 name=forwarding]: Recorded new controller, from now on will use broker rajesh:9092 (id: 0 rack: null) (kafka.server.BrokerToControllerRequestThread)
Step 4: List the topics
rajesh@rajesh:~/Rajesh/Softwares/kafka_2.13-3.3.1$ bin/kafka-topics.sh --list --bootstrap-server localhost:9092
Step 5: Create Topic to Store Events
rajesh@rajesh:~/Rajesh/Softwares/kafka_2.13-3.3.1$ bin/kafka-topics.sh --create --topic producer-consumer-demo --bootstrap-server localhost:9092
Created topic producer-consumer-demo.
rajesh@rajesh:~/Rajesh/Softwares/kafka_2.13-3.3.1$ bin/kafka-topics.sh --list --bootstrap-server localhost:9092
producer-consumer-demo
Step 6: Write Some events into the Topic Each line you enter will result in a separate event being written to the topic.
rajesh@rajesh:~/Rajesh/Softwares/kafka_2.13-3.3.1$ bin/kafka-console-producer.sh --topic producer-consumer-demo --bootstrap-server localhost:9092
Hello
Step 7: Read the events from the Topic
rajesh@rajesh:~/Rajesh/Softwares/kafka_2.13-3.3.1$ bin/kafka-console-consumer.sh --topic producer-consumer-demo --from-beginning --bootstrap-server localhost:9092
Hello