Apache Kafka 4.1.1 Installation (KRaft Mode, No ZooKeeper) on CentOS 8

1. Install Java 17 (Required for Kafka 4.x)

sudo dnf install -y java-17-openjdk java-17-openjdk-devel

Verify:

java -version

2. Create Kafka User

sudo useradd kafka -m
sudo usermod -s /bin/bash kafka

3. Download Kafka

wget https://dlcdn.apache.org/kafka/4.1.1/kafka_2.13-4.1.1.tgz
tar -xvzf kafka_2.13-4.1.1.tgz
sudo mv kafka_2.13-4.1.1 /opt/kafka
sudo chown -R kafka:kafka /opt/kafka

4. Create Data Directory

sudo mkdir -p /opt/kafka/data
sudo chown -R kafka:kafka /opt/kafka

5. Configure KRaft Mode

Edit the file:

sudo nano /opt/kafka/config/server.properties

Add/update:

process.roles=broker,controller
node.id=1
controller.quorum.voters=1@localhost:9093

listeners=PLAINTEXT://:9092,CONTROLLER://:9093
controller.listener.names=CONTROLLER
inter.broker.listener.name=PLAINTEXT

log.dirs=/opt/kafka/data

6. Generate UUID for Storage

UUID=$(/opt/kafka/bin/kafka-storage.sh random-uuid)
echo $UUID

7. Format Storage

sudo -u kafka /opt/kafka/bin/kafka-storage.sh format   -t $UUID   -c /opt/kafka/config/server.properties

Expected:

Formatting metadata directory /opt/kafka/data with metadata.version 4.1-IV1.

8. Test Kafka Manually

sudo -u kafka /opt/kafka/bin/kafka-server-start.sh /opt/kafka/config/server.properties

If Kafka starts, press Ctrl + C to stop.

9. Fix SELinux (CentOS 8)

Check:

getenforce

If Enforcing, temporarily allow:

sudo setenforce 0

Permanent:

sudo nano /etc/selinux/config
# set:
SELINUX=permissive

10. Create Systemd Service

sudo nano /etc/systemd/system/kafka.service

Add:

[Unit]
Description=Kafka KRaft Server
After=network.target

[Service]
User=kafka
Environment="JAVA_HOME=/usr/lib/jvm/java-17-openjdk"
Environment="PATH=/usr/lib/jvm/java-17-openjdk/bin:/usr/bin:/bin"
ExecStart=/opt/kafka/bin/kafka-server-start.sh /opt/kafka/config/server.properties
ExecStop=/opt/kafka/bin/kafka-server-stop.sh
Restart=on-failure
LimitNOFILE=100000

[Install]
WantedBy=multi-user.target

Reload & Start:

sudo systemctl daemon-reload
sudo systemctl start kafka
sudo systemctl enable kafka
sudo systemctl status kafka

11. Test Kafka

Create Topic

/opt/kafka/bin/kafka-topics.sh --create   --topic test-topic   --bootstrap-server localhost:9092

List Topics

/opt/kafka/bin/kafka-topics.sh --list --bootstrap-server localhost:9092

Producer

/opt/kafka/bin/kafka-console-producer.sh --topic test-topic --bootstrap-server localhost:9092

Consumer

/opt/kafka/bin/kafka-console-consumer.sh --topic test-topic --from-beginning --bootstrap-server localhost:9092

Completed

Kafka 4.1.1 is now running in KRaft mode without ZooKeeper.