Skip to main content

17  Dependency management for smoother collaboration

Many of you probably work mostly on your own, but as you move through your career (whether in academia or industry), you will need to—and perhaps want to—directly collaborate much more with others.

Tip

Here, “directly collaborate” means contributing to a shared project directly (like editing the same files), rather than discussion- or planning-based collaborations (and definitely not the “emailing-files-around” type of collaboration).

Even though different types of collaboration (e.g. meetings, brainstorming, real-time co-writing) form the basis for almost all research-based work (and probably most non-research-based work), direct collaboration quickly becomes unmanageable when relying on traditional academic “workflows” (like emailing files around). That’s when you’ll need tools designed for collaboration, such as Git and GitHub. But that’s just the starting point; many other factors related to workflows and processes must be considered to collaborate more effectively.

This session focuses on using more automated ways to structure data analysis projects and facilitate smoother collaboration.

17.1 Learning objectives

  1. Identify potential actions to take that streamline collaboration on a data analysis project.
  2. Explain what a “project-oriented” workflow is, what project-level R dependency management is, and why these concepts are important for collaborative and reproducible analyses.
  3. Describe the difference between workflow dependencies and build dependencies, and apply functions in the usethis R package to implement these dependency management concepts.

17.2 📖 Reading task: Project-level package dependency management

Briefly highlight the meaning of “project-oriented workflow”, the informal way of tracking dependencies, and why it’s not optimal.

Time: ~8 minutes.

Note

This first session is more conceptual and is heavier on the reading and explanation, but is important for the next sessions.

One of the first things to consider when working collaboratively on a data analysis project (and probably other types of projects too) is what software everyone will need in order to use or contribute to your project. This starts out at the highest level: Are you using R or some other software for the analysis? Since this is an R workshop, we’re assuming the software will be R! 😜

The next consideration is determining which packages your project will depend on to produce the results. When working collaboratively with others (or even with your future self), you’ll need a clear way to track the project’s package dependencies, and, ideally, have a simple and efficient method to install or update them.

Part of the approach covered in this workshop requires that you follow a “project-oriented” workflow when working on, well, a project. In order to know how to track your project’s package dependencies, you need to first know, what a “project” is and how do we work around it. Since the introduction workshop’s first session on the Management of R Projects, we’ve consistently taught and used this workflow-style. In fact, it is embedded into the use of the R Projects via the .Rproj files and in the use of the here package. So, we’re already following this approach, which will make it easier to track the package dependencies for our project.

The R Project text is made of metal, and the `{here}` package wears a helmet, appearing secure and robust and ready to go skateboarding. Meanwhile `setwd()` is made of glass, covered in casts and band-aids, looking much more fragile.

Cartoon of combining R Projects with the here package, compared to the common approach of using setwd(). Artwork by @allison_horst.

Let’s start with the AdvancedR3 project, created in Chapter 12, that includes the cv-risk dataset. We have code in the data-raw/cv-risk.R file already that uses some packages. Let’s assume that your project will be more complex than this and that you will eventually need some collaborators to contribute to your project who are experts in, for instance, metabolomics data processing and in statistical analysis of high-dimensional data. You know you will end up needing to use other packages. You also know that you all need some way of tracking which packages are used so that when others join and contribute to the project, they can as seamlessly as possible install or update the packages your data analysis project needs. There are a few ways of “tracking” package dependencies.

The simplest, but most primitive way is to always make sure to use library() at the top of each R script for each package that the R script uses. We can call this “informal” dependency management.

Let’s review the advantages and disadvantages of this form of dependency management:

  • Advantage:

    • It’s the easiest to conceptually understand and to use.

  • Disadvantages:

    • It doesn’t track project-level dependencies very well, since multiple scripts might use the same packages, while some might use different ones. This means that you can’t easily and quickly install or update all the packages used in your project. You will probably have to go through each R script manually and install each package manually.

    • It doesn’t track the versions of the packages your project depends on, so if a package gets updated and it breaks something, you might not be able to figure out how to quickly fix that issue, especially for those deadline crunches.

You might have seen a similar “informal” approach where scripts starts with code that looks like this:

if (!require("packagename")) {
  install.packages("packagename")
}

This code checks whether a package exists (i.e., is installed), and if not, it installs it. But! This is not an optimal method to track packages because require() won’t load the package if it doesn’t find it. As a result, you will have to rerun the script a few times if you don’t have that package installed; first, to install the package, then to load it. Plus, sometimes you may need to restart the R session after installing a package in order for R to detect it properly. So, this is not a very efficient way to track dependencies either.

What’s the alternative, then? One alternative is a more formal way of managing your dependencies, which is what we’ll cover in the next section.

CautionSticky/hat up!

When you’re ready to continue, place the sticky/paper hat on your computer to indicate this to the teacher 👒 🎩

17.3 📖 Reading task: Formal dependency management

Reinforce that we track dependencies via the DESCRIPTION file and usethis::use_package(), as well as the easy of installing and updating with pak::pak().

Time: ~3 minutes.

The most common form of a more formal way of managing dependencies— at least based on R packages and projects found on GitHub—makes use of the DESCRIPTION file and usethis::use_package(). We covered this style of dependency management in the intermediate workshop. We will also use this approach during this workshop, but we’ll expand on it a lot more. We use and recommend it for managing dependencies because many tools and workflows rely on it, making it well-integrated with R projects.

Let’s take a look on the advantages and disadvantages of this way of tracking dependencies:

  • Advantages:

    • Relatively easy to conceptually understand, since you can directly view the packages used in the project by opening the DESCRIPTION file and looking at the contents.

    • Because it is widely used, there are many processes already built around making use of tracking dependencies this way. For instance, when creating an R package, you need to track package dependencies using this approach.

    • Installing packages is as easy as opening the project and running pak::pak() in the Console, which will install all the packages listed in the DESCRIPTION file.

    • Adding packages that your project will depend on is as easy as writing usethis::use_package("packagename") in the Console.

  • Disadvantages:

    • Like the previous method, it doesn’t automatically keep track of the versions of the packages you are using. Even though it is possible to add a specific version to the DESCRIPTION file, pak will install the newest version of the package.

    • Your project might still rely on a package that is installed on your computer which influences your project. It could be that it isn’t obvious as a dependency, so you forget to include it. Meaning, you are still responsible for adding the project dependencies to the DESCRIPTION file.

Tip

There is an even more formal and structured way of tracking dependencies using the renv package, which we covered in previous versions of this workshop. It is a great package to use, especially if you are working on a project with several others or a project that requires tracking not only the packages but their specific versions. We don’t teach it anymore because it is quite complex and difficult to understand. It is also not really necessary for the majority of projects.

As we work on the project during the workshop and realize we need to use a specific package, we will continue using usethis::use_package() to add it to the DESCRIPTION file.

CautionSticky/hat up!

When you’re ready to continue, place the sticky/paper hat on your computer to indicate this to the teacher 👒 🎩

17.4 Adding packages from the data-raw/cv-risk.R script

Walk through the help pane when looking at ?usethis::use_package.

Before we continue, let’s first make sure to add and commit all the files in the AdvancedR3 project to save the current state of the files to the Git history. Open the Git interface by either typing with or with the Palette (, then type “commit”) or by going to the Git pane and clicking the “Commit” button. Then, write a commit message and click the “Commit” button in this window.

Next, we’ll update the DESCRIPTION file with the packages we use in the data-raw/cv-risk.R script. Let’s open the data-raw/cv-risk.R file and find which packages we use. We want to look for either packages listed in library() or for any functions explicitly called from their package with ::.

It can help to use the “Find in files” feature in RStudio to look for all places that have either :: or library. Use either Edit -> Find in Files ..., or with or with the Palette (, then type “find files”) to open up the search popup.

We should see two packages when searching for ::: fs and here. We will use usethis::use_package() to add these packages to our dependencies, but first, let’s review what this function does. Go to the R Console and use the ? help on the function:

?usethis::use_package

There are three arguments you can give this function: the package name, the “type” (we’ll cover this shortly), and the version. For now, we will only use the package argument. This is where we put the name of the package we want to set as a dependency.

In the Console, run:

usethis::use_package("fs")
✔ Setting active project to
  "/home/runner/work/r-cubed-advanced/r-cubed-advanced".
usethis::use_package("here")

We’ll see some notes about it being added to the Imports field in the DESCRIPTION file. We’ll explain what this means later in this session. While you could manually add these packages to the DESCRIPTION file, this helper function also checks that the package is installed, will install it if necessary, and correctly places the package in the right location in the DESCRIPTION file, so we strongly recommend using it.

Once we’re done, let’s open the Git interface with or with the Palette (, then type “commit”) (or going to the Git Pane and clicking the “Commit” button) and check what has been changed. Notice the DESCRIPTION file and where the packages have been added in the file.

Next, we’ll commit those changes to the Git history with a descriptive message.

17.5 Two types of dependencies: Build and workflow

Verbally explain the content of the next few paragraphs (don’t need to show the website on the projector).

When you work on a research project that involves data analysis, you’ll likely use packages in two different ways:

  1. Packages that directly contribute to data wrangling, analysis, plotting, and making the manuscript. These types of packages are generally called “build” dependencies. Packages like dplyr or tidyr are usually build dependencies, since you use them directly in scripts for processing data.
  2. Packages that assist you in doing your work but aren’t directly used for data analysis. These types of packages would be called “workflow” or “development” dependencies. usethis is usually considered a workflow dependency because you usually only use it in the Console and don’t need to use it in a script.

A useful guideline to help you determine whether a package is a build or a workflow dependency is:

  • If you write and use functions from the package within an R script that does something to the data or analysis, then it is likely a build dependency.
  • If you only ever use functions from a package in the Console, then it is probably a workflow dependency.

So, now we know that the dependencies we added in the exercise are actually build dependencies, since we use them directly in our project (in data-raw/cv-risk.R) to process the data.

The way you add packages to the DESCRIPTION file is slightly different depending on what type of dependency it is. For build dependencies, we use the function we’ve already used before: usethis::use_package("packagename"). For workflow dependencies, it’s the same function with a small twist. Our first workflow dependency is usethis, which we will use many times throughout the workshop.

But before we get into adding usethis to the DESCRIPTION file, let’s tweak our personal (also called the global) .Rprofile to make our lives a bit easier. We’ll add a setting so we don’t have to type usethis:: before each function.

Run the usethis::use_usethis() function to open your own (user) .Rprofile:

Console
usethis::use_usethis()

Then, copy and paste the code below into your .Rprofile, at the bottom of the file.

What this code does is that when you start an interactive R session (like when you open RStudio), it will automatically load the usethis package without you having to type library(usethis). It means you don’t need to always use usethis:: before any of its functions when you use them. This is template code, so we don’t really need to understand the details of what it is doing, but briefly, the interactive() function checks if you are running R interactively (like in RStudio with the Console open). The suppressMessages() function prevents R from printing messages when loading the package and the require() function loads the package if it is installed, otherwise it does nothing.

Let’s restart R with or with the Palette (, then type “restart”). Now, we’re ready to use use_package() to add usethis as a workflow dependency. The only difference between adding a build compared to a workflow dependency is that for workflow dependencies, we need to add a second argument to use_package() to specify that it is a workflow dependency. The second argument is "suggests", like so:

Console
use_package("usethis", "suggests")
Tip

If you open the help file for use_package() you’ll see that the second argument we used above is called type, and “Imports” is its default value. So, if you don’t specify the second argument, the package will be added under “Imports” in the DESCRIPTION file.

So, “imports” corresponds to build dependencies, while “suggests” are equivalent to what we call workflow dependencies.

Open the Git interface and see that under Suggests: in the DESCRIPTION file is usethis. Let’s commit these changes, with or with the Palette (, then type “commit”).

You’ve used styler during the pre-workshop tasks and we’ll use it more throughout the workshop. Which dependency do you think this is? It’s a workflow dependency, since we will only be using it in the Console or Command Palette. Let’s add it to the project DESCRIPTION as a workflow dependency:

Console
use_package("styler", "suggests")

For the information block below, mention it to the learners but you don’t need to go over it. Especially mention the second part of the tip.

Tip

When you come back to a project after a few months or if you start collaborating on a project, you can run pak::pak() to install all types of dependencies, both workflow and build. Neat! 🎉

Before we move on, let’s make sure to commit everything to the Git history with or with the Palette (, then type “commit”).

17.6 Summary

  • Track your project package dependencies with the DESCRIPTION file and usethis.

  • Install the dependencies necessary for your project with pak::pak().

  • Differentiate between build and workflow dependencies and how to add them to the DESCRIPTION file.

17.7 Survey

Please complete the survey for this session:

Feedback survey! 🎉

17.8 Code used in session

This lists some, but not all, of the code used in the section. Some code is incorporated into Markdown content, so is harder to automatically list here in a code chunk. The code below also includes the code from the exercises.

?usethis::use_package
usethis::use_package("fs")
usethis::use_package("here")
usethis::use_usethis()
use_package("usethis", "suggests")
use_package("styler", "suggests")