{"id":1064,"date":"2021-04-02T13:00:09","date_gmt":"2021-04-02T12:00:09","guid":{"rendered":"http:\/\/www.igfasouza.com\/blog\/?p=1064"},"modified":"2021-04-26T14:34:34","modified_gmt":"2021-04-26T13:34:34","slug":"kraft-aka-kip-500-and-raspberry-pi","status":"publish","type":"post","link":"http:\/\/www.igfasouza.com\/blog\/kraft-aka-kip-500-and-raspberry-pi\/","title":{"rendered":"KRaft aka KIP-500 and Raspberry PI"},"content":{"rendered":"

\"\"<\/p>\n

How goes the battle?<\/b><\/p>\n

It is now possible to run Apache Kafka without Apache ZooKeeper! KRaft (aka KIP-500)<\/a> mode Early Access Release is available to download<\/a>.<\/p>\n

This is another blog about Kafka and Raspberry PI, where I want to show how I did a simple KRaft test.<\/p>\n

If you\u2019re not familiar with Kafka, I suggest you have a look at my previous post What is Kafka<\/a> and I suggest you check my blog about how I created a Raspberry PI Kafka cluster<\/a>.<\/p>\n

First, let’s download the new beta Kafka version.<\/p>\n

1
<\/div><\/td>
wget<\/span> https:\/\/<\/span>github.com\/<\/span>apache\/<\/span>kafka\/<\/span>archive\/<\/span>refs\/<\/span>tags\/<\/span>2.8.0-rc0.zip<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n

You can follow the GitHub readme. The first step is to generate an ID for your new cluster, using the kafka-storage tool<\/p>\n

1
<\/div><\/td>
.\/<\/span>bin\/<\/span>kafka-storage.sh random-uuid<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n

Format your storage directories.<\/p>\n

1
<\/div><\/td>
.\/<\/span>bin\/<\/span>kafka-storage.sh format -t<\/span>  -c<\/span> .\/<\/span>config\/<\/span>kraft\/<\/span>server.properties<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n

Start the Kafka server on each node.<\/p>\n

1
<\/div><\/td>
.\/<\/span>bin\/<\/span>kafka-server-start.sh .\/<\/span>config\/<\/span>kraft\/<\/span>server.properties<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n

Create a topic.<\/p>\n

1
<\/div><\/td>
.\/<\/span>bin\/<\/span>kafka-topics.sh --create<\/span> --topic<\/span> igor --partitions<\/span> 1<\/span> --replication-factor<\/span> 1<\/span> --bootstrap-server<\/span> localhost:9092<\/span><\/div><\/td><\/tr><\/tbody><\/table><\/div>\n

Producer<\/p>\n

1
<\/div><\/td>
kafka-console-producer.sh --broker-list<\/span> localhost:9092<\/span> --topic<\/span> igor<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n

Consumer<\/p>\n

1
<\/div><\/td>
kafka-console-consumer.sh --bootstrap-server<\/span> localhost:9092<\/span> --topic<\/span> igor --from-beginning<\/span><\/div><\/td><\/tr><\/tbody><\/table><\/div>\n

Depending on each Raspberry Pi version you are using you will need to change the bin\/kafka-server-start.sh<\/em> with<\/p>\n

1
<\/div><\/td>
export<\/span> KAFKA_HEAP_OPTS<\/span>="-Xmx256M -Xms128M"<\/span><\/div><\/td><\/tr><\/tbody><\/table><\/div>\n

\"\"
\nPicture 1: Terminal Logs<\/small><\/p>\n

And that is it. Now you are ready to play with Kafka without Apache ZooKeeper on your Raspberry PI.<\/p>\n

I came up with the idea to use a simple REST interface application to produce and consume Kafka data.<\/p>\n

\"\"
\nPicture 2: Kafka REST application. Created with excalidraw.com<\/small><\/p>\n

It can be any webapp, it can use Quarkus, Micronaur, Spring or any other framework and I can run in a simple Raspberry PI zero. With that, I can integrate everything in my house with the Raspberry PI Kafka cluster.<\/p>\n

KRaft is in Early Access and is not a production-ready product, but is perfect to test with Raspberry PI at home. No Zookeeper means less memory, less disk space and fewer steps to configure your cluster.<\/p>\n","protected":false},"excerpt":{"rendered":"

How goes the battle? It is now possible to run Apache Kafka without Apache ZooKeeper! KRaft (aka KIP-500) mode Early Access Release is available to download. This is another blog about Kafka and Raspberry PI, where I want to show… Continue Reading →<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":1065,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[25],"tags":[7],"class_list":["post-1064","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-kafka","tag-kafka"],"_links":{"self":[{"href":"http:\/\/www.igfasouza.com\/blog\/wp-json\/wp\/v2\/posts\/1064","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.igfasouza.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.igfasouza.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.igfasouza.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.igfasouza.com\/blog\/wp-json\/wp\/v2\/comments?post=1064"}],"version-history":[{"count":6,"href":"http:\/\/www.igfasouza.com\/blog\/wp-json\/wp\/v2\/posts\/1064\/revisions"}],"predecessor-version":[{"id":1110,"href":"http:\/\/www.igfasouza.com\/blog\/wp-json\/wp\/v2\/posts\/1064\/revisions\/1110"}],"wp:featuredmedia":[{"embeddable":true,"href":"http:\/\/www.igfasouza.com\/blog\/wp-json\/wp\/v2\/media\/1065"}],"wp:attachment":[{"href":"http:\/\/www.igfasouza.com\/blog\/wp-json\/wp\/v2\/media?parent=1064"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.igfasouza.com\/blog\/wp-json\/wp\/v2\/categories?post=1064"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.igfasouza.com\/blog\/wp-json\/wp\/v2\/tags?post=1064"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}