Setup a Monorepo with PNPM workspaces and speed it up with Nx!

In this article we're going to have a deep dive into setting up a new monorepo using PNPM workspaces that hosts a Remix application as well as a React-based library. We will learn how to run commands with PNPM, how to run them in parallel and finally we're going to add Nx for a more sophisticated task scheduling, including command caching and more.

Important: If you are already familiar with the setup and configuration of a new PNPM workspace, feel free to skip to the part where we add Nx later in the article.

Prefer a video walkthrough?

Table of Contents

· Initialize a new PNPM workspace
· Setting up the Monorepo structure
· Adding a Remix application
· Create a Shared UI library
· Consuming our shared-ui package from the Remix app
· Running commands with PNPM
· Speeding up with Nx
· Installing Nx
· Running tasks with Nx
· Configure Caching
· Fine-tuning the caching
· Reusing Cache Input Globs
· Defining task dependencies (aka build pipeline)
· Running just what changed
· Additional features
· Dynamic Terminal Output
· Project Graph Visualization
· Conclusion

Initialize a new PNPM workspace

To get started, let's make sure you have PNPM installed. The official docs have an installation page with detailed instructions. I also recommend using something like Volta in particular if you have to deal with multiple different versions of NPM/PNPM and node versions.

Let's create a new folder named pnpm-mono, cd into it and then run pnpm init to generate a top-level package.json. This will be the root package.json for our PNPM monorepo.

It is probably also handy to initialize a new Git repository such that we can commit and backup things as we progress in the setup:

At this point let's also create a .gitignore file to immediately exclude things like node_modules and common build output folders.

Setting up the Monorepo structure

The structure of a monorepo might vary depending on what you plan to use it for. There are generally two kinds of monorepo:

• package centric repositories which are used for developing and publishing a cohesive set of reusable packages. This is a common setup in the open source world and can be seen in repositories such as Angular, React, Vue and many others. Those repos are characterized by most commonly having a packages folder and which are then commonly published to some public registry such as NPM.
• app centric repositories which are used mainly for developing applications and products. This is a common setup in companies. Such repos are characterized in having an apps and packages or libs folder, where the apps folder contains the buildable and deployable applications, while the packages or libs folder contains libraries that are specific to one or multiple applications that are being developed within the monorepo. You can still also publish some of these libs to a public registry.

In this article we're going to use the "app centric" approach, to demonstrate how we can have an application that consumes packages from within the monorepo.

Create an apps and packages folder within pnpm-mono:

Now let's configure PNPM to properly recognize the monorepo workspace. Basically we have to create a pnpm-workspace.yaml file at the root of the repository, defining our monorepo structure:

Adding a Remix application

We should now be ready to add our first application. For this example I picked Remix but you can really host any type of application in here, it won't really matter.

Info: We use the normal Remix installation & setup procedure here which you can find on their docs page.

Since we want to have the app within the apps folder, we need to cd into it:

You will be asked for an app name. Let's just go with "my-remix-app" which we'll be using for the rest of this article. Obviously feel free to use a different one. In addition, the Remix setup process is also going to ask you a couple of questions that customize the exact setup. The particular options are not really relevant for our article here, so feel free to choose whatever best suits your needs.

You should have now a Remix app, within the apps/my-remix-app folder or whatever name you chose. Remix has already a package.json with corresponding scripts configured:

Usually, in a monorepo you want to run commands from the root of the repository to not have to constantly switch between folders. PNPM workspaces have a way to do that, by passing a filter argument, like:

Now it happens (at the writing of this article) that Remix's default package.json doesn't have a name property defined which PNPM wants to run the package. So let's define one in the apps/my-remix-app/package.json:

You should now be able to serve your Remix app in dev-mode by using:

Create a Shared UI library

Now that we have our app set up, let's create a library package that can be consumed by our application.

Next, let's create a package.json with the following content (you can also use pnpm init and adjust it):

Note, we declare it as private because we don't want to publish it to NPM or somewhere else, but rather just reference and use it locally within our workspace. I also removed the version property since it is not used.

As the technology stack I've chosen to go with React (so we can import it in Remix) and TypeScript (because it can almost be considered a standard nowadays). Let's install these dependencies from the root of the workspace:

By passing --filter shared-ui to the installation command, we install these NPM packages locally to the shared-ui library.

Info: Be aware that this might potentially cause version conflicts if the React/TypeScript version used by the library package and the consumer (e.g. our app) differs. Adopting a single version policy, where you move the packages to the root of the monoreopo, is a possible solution for that.

Our first component will be a very simple Button component. So let's create one:

We also want to have a public API where we export components to be used outside of our shared-ui package:

For sake of simplicity we just use the TypeScript compiler to compile our package. We could have some more sophisticated setup for bundling multiple files together etc with something like Rollup or whatever you prefer using, but that's outside the scope of this article.

To create the desired compilation output create a packages/shared-ui/tsconfig.json file with the following configuration.

In a monorepo it is good practice to extract the common config part into a higher-level config (e.g. at the root) and then extend it here in the various projects. This to avoid a lot of duplication across the various monorepo packages. For the sake of simplicity I kept it all in one place here.

As you can see the outDir points to a package-local dist folder. So we should add a main entry point in the shared-ui package's package.json:

Finally, the actual build consists of deleting some residual folders from the previous output and then invoking the TypeScript compiler (tsc). Here's the complete packages/shared-ui/package.json file:

Use the following command to run the build from the root of the PNPM workspace:

If the build succeeds, you should see the compiled output in the packages/shared-ui/dist folder.

Consuming our shared-ui package from the Remix app

Our shared-ui library is ready so we can use it in the Remix application hosted within the apps folder of our repository. We can either manually add the dependency to Remix's package.json or use PNPM to add it:

This adds it to the dependency in the apps/my-remix-app/package.json:

workspace:* denotes that the package is resolved locally in the workspace, rather than from some remote registry (such as NPM). The * simply indicates that we want to depend on the latest version of it, rather than a specific one. Using a specific version really just makes sense if you're using external NPM packages.

To use our Button component we now import it from some Remix route. Replace the content of apps/my-remix-app/app/routes/index.tsx with the following:

If you now run the Remix app again you should see the button being rendered.

If you happen to get the following error, then it is because you need to build shared-ui first

To build that, run

Why? This is due to the symlinks PNPM creates in order to be able to reference and resolve local dependencies. By adding shared-ui: "workspace:*" to Remix's package.json you instruct PNPM to add a symlink to Remix's node_modules folder.

PNPM creates a symlink in the local node_modules folder to be able to import local packages

Running commands with PNPM

PNPM comes with handy features to run commands across the monorepo workspace. We have already seen how to scope commands on single packages using the --filter :

You can also run a command recursively on all the packages in the workspace using the -r flag. Imagine for instance running the build for all projects.

Similarly you can parallelize the run by using --parallel

Speeding up with Nx

PNPM workspaces come with some basic facilities for running tasks on the monorepo packages, even in parallel. As the monorepo grows, you might want to have a more sophisticated approach that allows to

• run tasks on only the packages that changed
• advanced caching based on file contents to not run anything that has already been computed previously
• remote distributed caching to speed up your CI

This is exactly where Nx can help. It is optimized for monorepo scenarios and comes with an advanced task scheduling mechanism. We still rely on the package installation and package linking mechanism that PNPM workspaces provide us, but use Nx instead to run our tasks in the most efficient way.

Installing Nx

Since Nx will be used for running operations across the entire monorepo workspace we're going to install it at the root level package.json.

That's it.

Running tasks with Nx

Nx uses the following form to run your commands:

target is the NPM script in this specific case you want to execute.

Let's try to run the build for our shared-ui package using the following command:

This produces the following output

Nx automatically finds shared-ui and runs the build script defined in packages/shared-ui/package.json.

Similarly, to launch our Remix app, run npx nx dev my-remix-app.

We can also run commands in parallel across the projects with:

Or selectively specify projects with

Note I'm prefixing the commands with _npx_ which runs the Nx executable in the _node_modules_ folder. In this way I don't have to install _nx_ globally. If you prefer doing that, feel free to do so.

Configure Caching

One of the main benefits of adding Nx to our PNPM workspace is speed via caching. Computation caching is a feature where different inputs (source files, env variables, command flags, etc.) are collected and a hash computed & stored in a local folder. Next time you run the command again, Nx looks for a matching hash, and if it finds one it just restores it. This includes restoring the terminal output as well as build artifacts (e.g. JS files in dist folders).

Not all operations are cacheable, only side-effect free ones are. For example, if you run an operation with the same inputs, it reliably always has to produce the same output. If as part of that operation you call some API for instance, it wouldn't be cacheable because the result of that API might vary given the same input parameters.

In order to enable caching, let's configure our cacheable operations. To do that we create an nx.json at the root of our workspace with the following content

Note the cacheableOperations array where we specify build and test . You can add more such as linting.

Having enabled this, if we now run our Remix app build the first time it is executed just as normal and we'll see it takes roughly 1s.

If you re-run the same command, it will now be pulled out of the cache and take only a few milliseconds.

You can also see that from the terminal output mentioning "existing outputs match the cache, left as is" as well as at the end "Nx read the output from the cache instead of running the command for 1 out of 1 tasks."

Having caching in place can drastically improve command execution times. It also gets even more useful if the cache is remotely distributed so that it can be shared with CI as well as other developer machines. In the case of Nx this can be done by enabling Nx Cloud, which comes with 500 hours saved/month for free (no credit card required) and unlimited hours for open source projects.

Fine-tuning the caching

By default the caching mechanism takes all project-level files as an input. We might want to distinguish though which files are being considered based on the target that we execute. Example: you might not want to invalidate the cache for the build task if only spec files for unit testing got changed.

To illustrate this on our example, run npx nx build my-remix-app twice, such that the caching gets activated. Next, change the README.md of the Remix project (apps/my-remix-app/README.md). If you re-run the Remix app build the cache will be invalidated due to the change of the README file. This might definitely not be a desirable operation.

We can fine-tune the caching by adding a targetDefaults node in the nx.json and define that the default input for the build target should exclude *.md files.

With this change, MD files would not be considered as part of the cache input whenever you run the build task.

Note that all path globs are relative to the root of the workspace. This avoids confusion as the inputs could also be defined at the project level in the _package.json_ (more here). You can use the interpolation variables _{projectRoot}_ and _{workspaceRoot}_ do distinguish whether the path should be targeting the project specific files or workspace level files.

Reusing Cache Input Globs

You can also go a step further as you might re-use this glob for excluding markdown files also for a hypothetical test target. You can do so by extracting the glob into a namedInputs property:

By adding ^ in front of the namedInput we indicate that this should also apply for changes in any dependencies of the project.

Defining task dependencies (aka build pipeline)

We have seen previously that when running our Remix dev server, but not having compiled the dependent shared-ui package first, we got an error when running our Remix app.

To fix it, we had to manually build shared-ui first. Normally you want to avoid this, which is exactly why Nx comes with a targetDefaults definition (often also denoted as the "build pipeline").

We can define such task dependencies in nx.json at the root of the workspace in the targetDefaults property.

As the first dependency we want to define that whenever we run the build target on a project, all the build targets of its dependent projects should be executed first. We can express that by adding an additional dependsOn property to the build task definition:

Similar as we have seen in the definition of the inputs, the ^ here denotes that the target should be run on all dependent projects. If you remove the ^, then the target would be invoked on the same project. That can be useful if you have a prebuild step that always needs to be invoked.

Next, we also want to define a targetDefault for our Remix dev command, such that first the build on all dependent packages (e.g our shared-ui) is run.

Here's the entire nx.json file again as a reference point:

If we now run npx nx build my-remix-app we can see that Nx first runs tasks on dependent projects, and only then runs the command we invoked.

Nx highlights dependent projects being built, but it keeps the main attention to the current task at hand without distracting

Running just what changed

In addition to providing caching, Nx also allows to just run what changed in a given branch with respect to a base branch by using the so-called "affected command".

You can use any target you have defined in your workspace. For example

• npx nx affected:build
• npx nx affected:test
• npx nx affected:lint
• npx nx affected:publish

How does this work? Nx builds a project graph based on the structure and dependencies among packages in your monorepo workspace. Let's assume the following hypothetical graph:

Potential graph of a monorepo workspace

Whenever we run the affected commands on a branch, Nx compares all the commits and relative changes with the base branch. By default that is main, but you can fine-tune that in the nx.json file:

If lib2 gets changed in our feature branch, running tests against the workspace using affected:test would only run tests for lib2 and appB.

Affected projects if "lib2" gets changed

Be aware however, if we run affected:build and we defined a dependency in our nx.json indicating that dependent projects need to be built first as well (see section "Defining task dependencies"), then affected:build would build

• lib3
• lib2
• appB

It would not build lib1 or appA though.

Additional features

Besides speed and task scheduling improvements, we also get some additional features by adding Nx to our PNPM workspace. Let's explore some:

Want to automate the creation of packages?

Once you have a good setup for a package, you obviously want to replicate that as you create new ones. The usual approach: copy & paste and then remove all stuff that's not needed.

That's tedious and potentially error prone. Nx has a concept of "generators", basically code scaffolding which allows you to generate new packages in the monorepo rather than copy & pasting old ones.

If that sounds interesting, here's a walkthrough:

Dynamic Terminal Output

Running tasks in parallel with PNPM results in quite a messy terminal output. The logs are hard to parse as the messages from the different commands being executed in parallel are interleaved.

When using Nx to run tasks you get a dynamic terminal that shows just what is necessary and most relevant to the current command that has been executed. Running the same parallel build task results in the following output when using Nx:

Terminal output of Nx dynamically showing the parallel tasks being computed as well as the ones that already succeeded

Project Graph Visualization

This launches an interactive visualization of the workspace's project graph with some advanced capabilities of filtering, debugging your workspace structure and more.

Nx project graph visualization of our PNPM workspace

As a side-note: you can run the project graph on any PNPM workspace, even if you don't have Nx installed. Running _npx nx graph_ should work.

Conclusion

We did it! Here are some of the things we covered:

• how to setup a PNPM based monorepo workspace
• create a Remix and shared React library within a PNPM monorepoe
• how to run different commands with PNPM
• how to add Nx & incrementally adopt it in the monorepo
• benefits and features that come with adding Nx to a PNPM workspace

You can find an example of such setup on the Nx Recipe GitHub repository:
https://github.com/nrwl/nx-recipes/tree/main/pnpm-workspace

Learn more

• 🧠 Nx Docs
• 👩‍💻 Nx GitHub
• 💬 Nx Official Discord Server
• 📹 Nrwl Youtube Channel
• 🥚 Free Egghead course