Note: After upgrade to 2.2.1, we have upgraded the testnet to 2.3.1 recently by following the same process. In the previous section, we have written down the steps to install a Kafka cluster on test1/test2/test3 VMs. The installed version is 2.0.0, which was the latest at the time of the installation. The current latest version is 2.2.1, and we are going to upgrade our test cluster to the latest version and see if the data file is compatible. Read More »

While working on one of the streams processing applications running in a Docker container to access local Kafka instance on the docker host, I had a lot of trouble getting the connection done right. First, I tried to use the host IP address 192.168.1.144:9092 to connect to the Kafka server and it doesn’t work as expected. The issue is that it works in some instances but not others. Then, I switched to the host name freedom:9092 and the same issue is still there. Read More »

During the development cycle, chances are you want to reset the entire Kafka cluster to the original state like a refreshed installation. Here are the steps to achieve that. Stop Applications If consumers or producers are connecting to your Kafka cluster, you need to stop them. For Taiji-chain cluster, we need to stop the chain-writer, chain-reader and token-reader on each Kafka node. For the light-portal cluster, we need to stop test1, test2, test3 and test4 docker-compose. Read More »

When using message driven approach for service to service communication on top of light-eventuate-4j framework, there are several security concerns that must be covered. For most customers, these issues might be minor but for banks, health care, military and government they are major concerns that must be addressed. The following discussion is based on Kafka as it is our first choice of messaging broker; however, the concept applies to other software package. Read More »

Apache Kafka is a distributed streaming platform developed by Apache Software Foundation and written in Java and Scala. Light uses Kafka for its messaging broker for event-based frameworks (light-eventuate-4j, light-tram-4j, and light-saga-4j). In this document, we will show you how to install Kafka 2.0 on Ubuntu 18.04 VMs to form a three nodes cluster. Prerequisites You will need three KVM boxes with 32GB memory each and to have sudo access to all the VMs. Read More »