Student Reviews
( 5 Of 5 )
1 review
Video of 12.2. Flume Agents in Apache Hadoop course by CloudxLab Official channel, video No. 65 free certified online
Flume agents are independent daemon processes. Flume agent consists of three parts: Source, Channel and Sink.
- A source receives data from the data generators
Examples - Tweets, Web server logs, Click event data in any application, etc
- A channel is a transient store which receives the events from the source and buffers them till they are consumed by the sinks. It acts as a bridge between the sources and the sinks.
Examples - File System channel, memory channel, etc
- A sink consumes data from the channels and delivers it to the destination.
Examples - HDFS, HBase etc
When the rate of incoming data from the source exceeds the rate at which it can be written to the destination, flume channel acts as a mediator between the source and sink by buffering the data.
Flume agents run on every machine where we want to collect the data.
As displayed in the image, in our previous use case, flume agents will be installed on every web server where data is being produced. A data collector collects data from agents and pushes it to a centralized data store.
Flume agents can be arranged in arbitrary topologies. As shown in the image, the source is consuming data from the sink and the same sink data is getting consumed by multiple sources.
This Big Data Tutorial will help you learn HDFS, ZooKeeper, Hive, HBase, NoSQL, Oozie, Flume, Sqoop, Spark, Spark RDD, Spark Streaming, Kafka, SparkR, SparkSQL, MLlib, and GraphX from scratch. Everything in this course is explained with the relevant example thus you will actually know how to implement the topics that you will learn in this course.
Let us know in the comments below if you find it helpful.
In order to claim the certificate from E&ICT Academy, IIT Roorkee, visit https://bit.ly/cxlyoutube
________
Website https://www.cloudxlab.com
Facebook https://www.facebook.com/cloudxlab
Instagram https://www.instagram.com/cloudxlab
Twitter http://www.twitter.com/cloudxlab