org.apache.kafka.clients.NetworkClient Bootstrap broker (id: -3 rack: null) disconnected when connec

I am trying to consume messages from a kafka topic on Amazon MSK (Kafka Version 2.2.1) using NiFi ConsumeKafka_2_6 processor (NiFi version 1.12.1).In ConsumeKafka processor I am getting the warning as "org.apache.kafka.clients.NetworkClient [Consumer clientId=console-consumer-34567-5, groupId=nifi-consumer] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected".What can be the cause of this issue?Is this due to version mismatch of kafka client used or due to connectivity issues.

3 Answers

It's a connectivity problem or you configured the Nifi processor incorrectly

You're not connecting to MSK, you're trying to connect Nifi to itself

Bootstrap broker localhost:9092

Also, yes, you might have some issues using the 2_6 consumer with a 2.2 cluster, but very minor as the core functions should still work

From aws cli we use the JVM truststore to talk to the MSK cluster. So from NiFi application also I was supposed to use that. I have configured the security protocol as SSL and configured JVM truststore in SSLContextService. Then I was able to consume messages from Kafka.

There can be multiple reason leading to this error. In my case, AWS MSK IAM auth enabled, I was missing the client.properties file while running kafka-console-consumer.sh script --consumer.config path/to/client.properties

ncG1vNJzZmirpJawrLvVnqmfpJ%2Bse6S7zGiorp2jqbawutJobW5tZ26FcYGOqKmgZZGlrqS0xGaimp6blnqkuMiepa2rXaOytcPOq6KcpJmau7V5waimraukp66xecGrpqSdomK2pXmSZqmam5tiu7a4y2aboquT