1. Home
  2. Know how
  3. When will a Consumer gets down?
  1. Home
  2. Videos
  3. When will a Consumer gets down?

When will a Consumer gets down?

When a consumer gets down:

The possible causes that a consumer stop are when SparkJobs and Yarn application stops working.

1. To Check the SparkHB:
  • ps -ef | grep SparkHB

This will tell the existing status of the Spark consumer running on the Application with a PID.

If the SparkHB is not working it will return no job running

This will also stop the running Yarn application

To Start the consumer and yarn application again normally,

  • cd /usr/hdp/filegps_release/FILEGPSV42/
  • ./scripts/consumers.sh
2. Troubleshooting of Consumers:

These might be a chance of downtime if the Spark2, Yarn and also the dependent services stop. Then the services needed to be started from the Ambari and check for Status of Spark and Yarn application

Note: Only one Yarn and SparkHB will be running from the FileGPS Application.

  • ps -ef | grep Kafka

If Kafka is not running the data will not be pushed to Spark Streaming.

So, to start the Kafka service, go to

  • Go to Ambari and restart the Kafka services.
  • telnet 10.5.0.X 6667

Port 6667 is the default port for Kafka. When it is deployed as a service for Ambari. If 6667 port is listening by Internal IP of Server where Kafka is Deployed, This ensures that Kafka service is working properly.

Note: Port 6667 can be configurable upon requirment.

Updated on July 26, 2019

Was this article helpful?

Related Articles

Leave a Comment