Gebruik de code voor 20% korting op alle trainingen -->

Black Friday

BLACK20%FRIDAY
Training: Hadoop Ecosystem
Hadoop
29 uur
Engels (US)

Training: Hadoop Ecosystem

Snel navigeren naar:

  • Informatie
  • Inhoud
  • Kenmerken
  • Meer informatie
  • Reviews
  • FAQ

Productinformatie

In deze Hadoop training / cursus leert u het Hadoop ecosystem kennen. Zo komen de meest gangbare open source componenten aan bod, maar leert u ook Hadoop te installeren. Later in de cursus komt data repository (HDFS, Flume, Sqoop) en data factory (Hive, Pig, Oozie) uitgebreid aan bod.

Onderwerpen die onder andere aan bod komen zijn repository components, YARN, HDFS, NameNode and DataNode, Flume, MySQL, MapReduce Java API, Hive, Pig, Oozie en nog veel meer.

Inhoud van de training

Hadoop Ecosystem

29 uur

Ecosystem for Hadoop

  • start the course
  • describe supercomputing
  • recall three major functions of data analytics
  • define Big Data
  • describe the two different types of data
  • describe the components of the Big Data stack
  • identify the data repository components
  • identify the data refinery components
  • identify the data factory components
  • recall the design principles of Hadoop
  • describe the design principles of sharing nothing
  • describe the design principles of embracing failure
  • describe the components of the Hadoop Distributed File System (HDFS)
  • describe the four main HDFS daemons
  • describe Hadoop YARN
  • describe the roles of the Resource Manager daemon
  • describe the YARN NodeManager and ApplicationMaster daemons
  • define MapReduce and describe its relations to YARN
  • describe data analytics
  • describe the reasons for the complexities of the Hadoop Ecosystem
  • describe the components of the Hadoop ecosystem

Installation of Hadoop

  • start the course
  • recall the minimum system requirements for installation
  • configure the start-up shell and yum repositories
  • install the Java Developers Kit
  • setup SSH for Hadoop
  • recall why version 2.0 was significant
  • describe the three different installation modes
  • download and install Apache Hadoop
  • configure Hadoop environmental variables
  • configure Hadoop HDFS
  • start and stop Hadoop HDFS
  • configure Hadoop YARN and MapReduce
  • start and stop Hadoop YARN
  • validate the installation and configuration
  • recall the structure of the HDFS command
  • recall the importance of the output directory
  • run WordCount
  • recall the ports of the NameNode and Resource Manager Web UIs
  • use the NameNode and Resource Manager Web UIs
  • describe the best practices for changing configuration files
  • recall some of the most common errors and how to fix them
  • access Hadoop logs and troubleshoot Hadoop installation errors
  • to install and configure Hadoop and its associated components

Data Repository with HDFS and HBase

  • start the course
  • configure the replication of data blocks
  • configure the default file system scheme and authority
  • describe the functions of the NameNode
  • recall how the NameNode operates
  • recall how the DataNode maintains data integrity
  • describe the purpose of the CheckPoint Node
  • describe the role of the Backup Node
  • recall the syntax of the file system shell commands
  • use shell commands to manage files
  • use shell commands to provide information about the file system
  • perform common administration functions
  • configure parameters for NameNode and DataNode
  • troubleshoot HDFS errors
  • describe key attributes of NoSQL databases
  • describe the roles of HBase and ZooKeeper
  • install and configure ZooKeeper
  • instause the HBase command line to create tables and insert datall and configure HBase
  • instause the HBase command line to create tables and insert datall and configure HBase
  • manage tables and view the web interface
  • create and change HBase data
  • provide a basic understanding of how Hadoop Distributed File System functions

Data Repository with Flume

  • start the course
  • describe the three key attributes of Flume
  • recall some of the protocols cURL supports
  • use cURL to download web server data
  • recall some best practices for the Agent Conf files
  • install and configure Flume
  • create a Flume agent
  • describe a flume agent in detail
  • use a flume agent to load data into HDFS
  • identify popular sources
  • identify popular sinks
  • describe Flume channels
  • describe what is happening during a file roll
  • recall that Avro can be used as both a sink and a source
  • use Avro to capture a remote file
  • create multiple-hop Flume agents
  • describe interceptors
  • create a Flume agent with a TimeStampInterceptor
  • describe multifunction Flume agents
  • configure Flume agents for mutliflow
  • create multi-source Flume agents
  • compare replicating to multiplexing
  • create a Flume agent for multiple data sinks
  • recall some common reasons for Flume failures
  • use the logger to troubleshoot Flume agents
  • configure the various Flume agents

Data Repository with Sqoop

  • start the course
  • describe MySQL
  • install MySQL
  • create a database in MySQL
  • create MySQL tables and load data
  • describe Sqoop
  • describe Sqoop's architecture
  • recall the dependencies for Sqoop installation
  • install Sqoop
  • recall why it's important for the primary key to be numeric
  • perform a Sqoop import from MySQL into HDFS
  • recall what concerns the developers should be aware of
  • perform a Sqoop export from HDFS into MySQL
  • recall that you must execute a Sqoop import statement for each data element
  • perform a Sqoop import from MySQL into HBase
  • recall how to use chain troubleshooting to resolve Sqoop issues
  • use the log files to identify common Sqoop errors and their resolutions
  • to use Sqoop to extract data from a RDBMS and load the data into HDFS

Data Refinery with YARN and MapReduce

  • start the course
  • describe parallel processing in the context of supercomputing
  • list the components of YARN and identify their primary functions
  • diagram YARN Resource Manager and identify its key components
  • diagram YARN Node Manager and identify its key components
  • diagram YARN ApplicationMaster and identify its key components
  • describe the operations of YARN
  • identify the standard configuration parameters to be changed for YARN
  • define the principle concepts of key-value pairs and list the rules for key-value pairs
  • describe how MapReduce transforms key-value pairs
  • load a large text book and then run WordCount to count the number of words in the text book
  • label all of the functions for MapReduce on a diagram
  • match the phases of MapReduce to their definitions
  • set up the classpath and test WordCount
  • build a JAR file and run WordCount
  • describe the base mapper class of the MapReduce Java API and describe how to override its methods
  • describe the base Reducer class of the MapReduce Java API and describe how to override its methods
  • describe the function of the MapReduceDriver Java class
  • set up the classpath and test a MapReduce job
  • identify the concept of streaming for MapReduce
  • stream a Python job
  • understand YARN features and components, as well as MapReduce and its classes

Data Factory with Hive

  • start the course
  • recall the key attributes of Hive
  • describe the configuration files
  • install and configure Hive
  • create a table in Derby using Hive
  • create a table in MySQL using Hive
  • recall the unique delimiter that Hive uses
  • describe the different operators in Hive
  • use basic SQL commands in Hive
  • use SELECT statements in Hive
  • use more complex HiveQL
  • write and use Hive scripts
  • recall what types of joins Hive can support
  • use Hive to perform joins
  • recall that a Hive partition schema must be created before loading the data
  • write a Hive partition script
  • recall how buckets are used to improve performance
  • create Hive buckets
  • recall some best practices for user defined functions
  • create a user defined function for Hive
  • recall the standard error code ranges and what they mean
  • use a Hive explain plan
  • understand configuration option, data loading and querying

Data Factory with Pig

  • start the course
  • describe Pig and its strengths
  • recall the minimal edits needed to be made to the configuration file
  • install and configure Pig
  • recall the complex data types used by Pig
  • recall some of the relational operators used by Pig
  • use the Grunt shell with Pig Latin
  • set parameters from both a text file and with the command line
  • write a Pig script
  • use a Pig script to filter data
  • use the FOREACH operator with a Pig script
  • set parameters and arguments in a Pig script
  • write a Pig script to count data
  • perform data joins using a Pig script
  • group data using a Pig script
  • cogroup data with a Pig script
  • flatten data using a pig script
  • recall the languages that can be used to write user defined functions
  • create a user defined function for Pig
  • recall the different types of error categories
  • use explain in a Pig script
  • install Pig, use Pig operators and Pig Latin, and retrieve and group records

Data Factory with Oozie and Hue

  • start the course
  • describe metastore and hiveserver2
  • install and configure metastore
  • install and configure HiveServer2
  • describe HCatalog
  • install and configure WebHCat
  • use HCatalog to flow data
  • recall the Oozie terminology
  • recall the two categories of environmental variables for configuring Oozie
  • install Oozie
  • configure Oozie
  • configure Oozie to use MySQL
  • enable the Oozie Web Console
  • describe Oozie workflows
  • submit an Oozie workflow job
  • create an Oozie workflow
  • run an Oozie workflow job
  • describe Hue
  • recall the configuration files that must be edited
  • install Hue
  • configure the hue.ini file
  • install and configure Hue on MySQL
  • use the Hue File Browser and Job Scheduler
  • configure Hive daemons, Oozie, and Hue

Data Flow for the Hadoop Ecosystem

  • start the course
  • describe the data life cycle management
  • recall the parameters that must be set in the Sqoop import statement
  • create a table and load data into MySQL
  • use Sqoop to import data into Hive
  • recall the parameters that must be set in the Sqoop export statement
  • use Sqoop to export data from Hive
  • recall the three most common date datatypes and which systems support each
  • use casting to import datetime stamps into Hive
  • export datetime stamps from Hive into MySQL
  • describe dirty data and how it should be preprocessed
  • use Hive to create tables outside the warehouse
  • use pig to sample data
  • recall some other popular components for the Hadoop Ecosystem
  • recall some best practices for pseudo-mode implementation
  • write custom scripts to assist with administrative tasks
  • troubleshoot classpath errors
  • create complex configuration files
  • to use Sqoop and Hive for data flow and fusion in the Hadoop ecosystem

Kenmerken

Docent inbegrepen
Bereidt voor op officieel examen
Engels (US)
29 uur
Hadoop
90 dagen online toegang
HBO

Meer informatie

Doelgroep Netwerkbeheerder, Softwareontwikkelaar, Databasebeheerders
Voorkennis

Er is geen specifieke voorkennis vereist.

Resultaat

Na het volgen van deze training bent u bekend met het gehele ecosystem van Hadoop en weet u hoe u deze kunt installeren.

Positieve reacties van cursisten

Training: Leidinggeven aan de AI transformatie

Nuttige training. Het bestelproces verliep vlot, ik kon direct beginnen.

- Mike van Manen

Onbeperkt Leren Abonnement

Onbeperkt Leren aangeschaft omdat je veel waar voor je geld krijgt. Ik gebruik het nog maar kort, maar eerste indruk is goed.

- Floor van Dijk

Training: Leidinggeven aan de AI transformatie

Al jaren is icttrainingen.nl onze trouwe partner op het gebied van kennisontwikkeling voor onze IT-ers. Wij zijn blij dat wij door het platform van icttrainingen.nl maatwerk en een groot aanbod aan opleidingen kunnen bieden aan ons personeel.

- Loranne, Teamlead bij Inwork

Hoe gaat het te werk?

1

Training bestellen

Nadat je de training hebt besteld krijg je bevestiging per e-mail.

2

Toegang leerplatform

In de e-mail staat een link waarmee je toegang krijgt tot ons leerplatform.

3

Direct beginnen

Je kunt direct van start. Studeer vanaf nu waar en wanneer jij wilt.

4

Training afronden

Rond de training succesvol af en ontvang van ons een certificaat!

Veelgestelde vragen

Veelgestelde vragen

Op welke manieren kan ik betalen?

Je kunt bij ons betalen met iDEAL, PayPal, Creditcard, Bancontact en op factuur. Betaal je op factuur, dan kun je met de training starten zodra de betaling binnen is.

Hoe lang heb ik toegang tot de training?

Dit verschilt per training, maar meestal 180 dagen. Je kunt dit vinden onder het kopje ‘Kenmerken’.

Waar kan ik terecht als ik vragen heb?

Je kunt onze Learning & Development collega’s tijdens kantoortijden altijd bereiken via support@icttrainingen.nl of telefonisch via 026-8402941.

Background Frame
Background Frame

Onbeperkt leren

Met ons Unlimited concept kun je onbeperkt gebruikmaken van de trainingen op de website voor een vast bedrag per maand.

Bekijk de voordelen

Heb je nog twijfels?

Of gewoon een vraag over de training? Blijf er vooral niet mee zitten. We helpen je graag verder. Daar zijn we voor!

Contactopties