Back to Blog
Spark scala5/21/2023 ![]() This step gives back a key value list> as a sorted key-value pair output. Sorting: This step is in charge of transforming the merge step input to a sorted key-value pair (sorted based on keys).This step returns an unsorted list like (Key, List(Value)). It can be viewed as an aggregate function applied to all keys. Merging: This step combines key-value pairs having the same keys.It takes a list of outputs from the mapper module and performs the following two sub-steps on every key-value pair. It essentially performs the following two sub-steps: It is is the second step in MapReduce Algorithm. ![]() The output from this mapping module is a key-value pair like (Key, Value) 2.Shuffle Function It takes smaller datasets and performs computation on each of them
0 Comments
Read More
Leave a Reply. |