Course Details

We offer value based learning for the training program that has been designed and the course contents are listed below:

Writing a MapReduce Program

• The MapReduce Flow
• Basic MapReduce API Concepts
• Writing MapReduce Drivers, Mappers and Reducers in Java
• Writing Mappers and Reducers in Other Languages Using the Streaming API
• Speeding Up Hadoop Development by Using Eclipse
• Demo: Writing a MapReduce Program
• Differences Between the Old and New MapReduce APIs

Unit Testing MapReduce Programs

• Unit Testing
• The JUnit and MRUnit Testing Frameworks
• Writing Unit Tests with MRUnit
• Demo: Writing Unit Tests with the MRUnit Framework
• Delving Deeper into the Hadoop API
• Using the ToolRunner Class
• Decreasing the Amount of Intermediate Data with Combiners
• Demo: Writing and Implementing a Combiner
• Setting Up and Tearing Down Mappers and Reducers by Using the Configure and Close Methods

Writing Custom Partitioners for Better Load Balancing

• Demo: Writing a Partitioner
• Accessing HDFS Programmatically
• Using The Distributed Cache
• Using the Hadoop API’s Library of Mappers, Reducers and Partitioners

Practical Development Tips and Techniques

• Strategies for Debugging MapReduce Code
• Testing MapReduce Code Locally by Using LocalJobReducer
• Writing and Viewing Log Files
• Retrieving Job Information with Counters
• Determining the Optimal Number of Reducers for a Job
• Creating Map-Only MapReduce Jobs
• Demo: Using Counters and a Map-Only Job

Data Input and Output

• Creating Custom Writable and WritableComparable Implementations
• Saving Binary Data Using SequenceFile and Avro Data Files
• Implementing Custom Input Formats and Output Formats
• Issues to Consider When Using File Compression
• Demo: Using SequenceFiles

Common MapReduce Algorithms

• Sorting and Searching Large Data Sets
• Performing a Secondary Sort
• Indexing Data
• Demo: Creating an Inverted Index
• Computing Term Frequency — Inverse Document Frequency
• Calculating Word Co-Occurrence
• Joining Data Sets in MapReduce Jobs
• Writing a Map-Side Join
• Writing a Reduce-Side Join
• Machine Learning and Mahout
• Introduction to Machine Learning
• Using Mahout
• Demo: Using a Mahout Recommender