Merge Dataframes In R


The merge function in R allows you to combine two data frames, much like the join function that is used in SQL to combine data tables. Merge, however, does not allow for more than two data frames to be joined at once, requiring several lines of code to join multiple data frames.

Merge Dataframes In R

This post explains the methodology behind merging multiple data frames in one line of code using base R. We will be using the Reduce function, part of Funprog in base R v3.4.3. Funprog contains a suite of higher order functions which provide simple alternatives to laborious, long winded coding solutions.

In R you use the merge function to combine data frames. This powerful function tries to identify columns or rows that are common between the two different data frames. How to use merge to find the intersection of data The simplest form of merge finds the intersection between two different sets of data. Data Wrangling in R: Combining, Merging and Reshaping Data. # The basic syntax for merge is merge(x, y, by, by.x, by.y), where 'x' and 'y' # are the respective data sets, 'by' is the column(s) to merge by (assuming the # column names match between the two data sets), and 'by.x' and 'by.y' are also # columns to merge by in the event the.

The merge function

Merge Dataframes In R

As described, merge is essentially the “join” of the R world. Whilst this post is not about the fine workings of merge, I will give a brief introduction.

Merge takes two data frames, x and y, and combines them based on one or more shared columns. Rows are combined where the data of these shared columns are equal, meaning we can combine columns from different data frames that refer to the same piece of data. For instance, take the following two data frames:

It is clear that the two data frames are referring to the same characters, however it may be more useful to us if the two were combined into a single data frame. This is where merge comes in. Merge takes the following structure:

Here, we are looking to combine the height and gender data frames where the character columns are equal. To continue the SQL analogy, x is the left-hand table, y is the right-hand table, and merge is the LEFT JOIN operation. The “by” component is our “ON” clause. For example:

Running this merge function gives us the following output:

This is the result we were expecting, but what if we introduce a third data frame?

Sadly, merge does not allow us to simply add our eyeColour data frame as a third input (we only have x and y parameters available). That’s where Reduce comes in.


The Reduce function

Merge Data Frames In R By Multiple Columns

Merge Dataframes In R

Reduce takes a function and sequentially applies it to a given list of inputs, in our case a list of data frames. For example, imagine we have a function f which accepts two arguments, and a list of objects (a, b, c). Then Reduce(x, list(a, b, c)) would perform the following action:

f(a, f(b, c))

where the function x is first applied to data frames b and c, and is then applied to data frame a and the output of the first application of x. This allows us to avoid running and saving x(b, c), like this:

Applying Reduce to merge


In merge we have an example of a function that performs an action on two inputs. Reduce takes two parameters; f which stands for function and x which represents a vector. Reduce will sequentially apply the function f to the list x.

In our example, the function that we want to apply is merge, and the vector which we want to apply it to is a list of our data frames. First off, let’s try the following:

Perfect! But what if we wanted to specify the parameters within our merge function call? Well, we could define our own function which merges two data frames with specified parameters:

R Merge Multiple Data Frames

Here, we have specified our f as a custom function, which takes two parameters and applies the merge function to them. Within this custom function, we have specified our by parameter, which may be necessary for longer or more complex uses of Reduce.

Further reading

How To Merge Dataframes In R

The function that we passed to Reduce is known in the world of functional programming as a lambda function, or an anonymous function; a single use function that is not named and saved. Functional programming is a principle around which R is built, and can provide many smart and elegant ways to achieve things that would otherwise require large amounts of coding. We may explore more of the functional programming features of R in future blog posts, however for now the following link provides a nice overview of the most used techniques:

Combine Data In R

by Jon Willis

Merge A List Of Dataframes In R

  • 2019
    • 27 Feb 2019 5 reasons why Microsoft became Gartner’s market leader for BI 27 Feb 2019
  • 2018
    • 14 Dec 2018 8 insights from the SDR 2017-18 Dashboard 14 Dec 2018
    • 23 Nov 2018 What is a Dashboard? 23 Nov 2018
    • 31 Aug 2018 Plotly in R: How to make ggplot2 charts interactive with ggplotly 31 Aug 2018
    • 16 Aug 2018 Making the most of box plots 16 Aug 2018
    • 24 Jul 2018 Plotly in R: How to order a Plotly bar chart 24 Jul 2018
    • 11 Apr 2018 Machine learning in the housing sector 11 Apr 2018
    • 5 Mar 2018 How Useful Are Traffic Light Scorecards for Performance Management? 5 Mar 2018
    • 16 Feb 2018 How to merge multiple data frames using base R 16 Feb 2018
    • 8 Feb 2018 The beginner's guide to time series forecasting 8 Feb 2018
    • 24 Jan 2018 R Shiny vs. Power BI 24 Jan 2018
  • 2017
    • 18 Oct 2017 What is predictive analytics? 18 Oct 2017
    • 19 Sep 2017 Performance Management Case Study 19 Sep 2017
  • 2016
    • 15 Aug 2016 Fundamentals of a good performance framework 15 Aug 2016