Learning Apache Spark 2

• Get an overview of big data analytics and its importance for organizations and data professionals
• Delve into Spark to see how it is different from existing processing platforms
• Understand the intricacies of various file formats, and how to process them with Apache Spark.
• Realize how to deploy Spark with YARN, MESOS or a Stand-alone cluster manager.
• Learn the concepts of Spark SQL, SchemaRDD, Caching and working with Hive and Parquet file formats
• Understand the architecture of Spark MLLib while discussing some of the off-the-shelf algorithms that come with Spark.
• Introduce yourself to the deployment and usage of SparkR.
• Walk through the importance of Graph computation and the graph processing systems available in the market
• Check the real world example of Spark by building a recommendation engine with Spark using ALS.
• Use a Telco data set, to predict customer churn using Random Forests.

Spark juggernaut keeps on rolling and getting more and more momentum each day. Spark provides key capabilities in the form of Spark SQL, Spark Streaming, Spark ML and Graph X all accessible via Java, Scala, Python and R. Deploying the key capabilities is crucial whether it is on a Standalone framework or as a part of existing Hadoop installation and configuring with Yarn and Mesos.

The next part of the journey after installation is using key components, APIs, Clustering, machine learning APIs, data pipelines, parallel programming. It is important to understand why each framework component is key, how widely it is being used, its stability and pertinent use cases.

Once we understand the individual components, we will take a couple of real life advanced analytics examples such as ‘Building a Recommendation system’, ‘Predicting customer churn’ and so on.

The objective of these real life examples is to give the reader confidence of using Spark for real-world problems.