Student Reviews
( 5 Of 5 )
1 review
Video of 6.3. YARN Architecture in Apache Hadoop course by CloudxLab Official channel, video No. 36 free certified online
Let us try to understand the architecture of YARN. The objective of YARN is to be able to execute any kind of workfload in distributed fashion.
All of the components that you see in the diagram are software programs.
It has one resource manager. Each machine has a node manager. The node managers create containers to execute the programs.
Application Masters are the programs per application that are executed inside the containers. Examples of Application Masters are Mapreduce AM and Spark AM.
The flow goes like this:
A client submits an application to the resource manager. The resource manager consults with the node managers and creates an Application Master for the client inside a container.
Now, the application master registers itself with resource manager so that the client can monitor the progress and interact with the application master.
Application Master then requests Resource Manager for containers.
The resource manager consults and negotiates with Node Managers for the containers.
Node Managers create containers and launch app in these containers.
The node managers talk back to the resource manager about the resource usage.
To check the status, the clients can talk to Application Master.
On completion, Application Master deregisters itself from the resource manager and shuts down.
It can be better understood if we think of YARN as a country where Resource Manager is the president.
And clients are organizations like Microsoft or Shell who want their work to be done by the way of establishing offices.
Each node is a state such as California and the node manager is governor of the state.
The container is the place for the office. The application master is the head office.
Though it looks like over simplification, it would make us understand how YARN works.
Microsoft wants to get their offices to be setup, so they contact the president, the president talks to governors of the states and then the head office is established. After wards to get more offices established in various states, head office would request president and the president would discuss with governors and offices would be established.
This Big Data Tutorial will help you learn HDFS, ZooKeeper, Hive, HBase, NoSQL, Oozie, Flume, Sqoop, Spark, Spark RDD, Spark Streaming, Kafka, SparkR, SparkSQL, MLlib, and GraphX from scratch. Everything in this course is explained with the relevant example thus you will actually know how to implement the topics that you will learn in this course.
Let us know in the comments below if you find it helpful.
In order to claim the certificate from E&ICT Academy, IIT Roorkee, visit https://bit.ly/cxlyoutube
________
Website https://www.cloudxlab.com
Facebook https://www.facebook.com/cloudxlab
Instagram https://www.instagram.com/cloudxlab
Twitter http://www.twitter.com/cloudxlab