4. Road to Nix - The module system: Introduction

In the previous chapter we introduced the nixpkgs library and some of the simple utilities it provides. One of the things it also provides is the module system. Let’s see what it’s all about!

Note: Sometimes in this chapter some features might seem useless. It will become clear why they are so useful when, in future chapters, we introduce NixOS and how it works.

The module system

The module system is a group of utilities allowing us to merge various so-called “modules” into a final configuration. These modules can declare configurable options and define their values. It is used to allow users to use Nix as a language to configure a tool, the most notable of which is NixOS.

Let’s make an example and say we want to make a tool which sends a greeting message whenever we reboot our computer. We could create the following modules:

We then let the user create a module that sets the value of enableGreeting and greetingMessage. Once they do that, we collect all of these modules together and pass them through the appropriate nixpkgs library function to create the final configuration. This final configuration will be an attribute set containing enableGreeting, greetingMessage and finalCommand with their values. We could then simply evaluate the value of finalCommand and run the output at every reboot. Thanks to its value depending on the value of the other two, it will already contain the correct command, all done in Nix!

A practical example

Understanding all of this without having seen one bit of Nix code about the module system is probably pretty hard. To better understand it, let’s create the example we just described above!

The module system is quite big, to the point where it’s split across three different sub-libraries: modules, options and types. The most important function in the module system is lib.modules.evalModules, or just lib.evalModules. It’s the entry point of the module system: the function to which we pass our modules and from which we get the final configuration.

It takes an attribute set as argument. This attribute set contains some parameters to be used by lib.evalModules. One of these, the most important, is modules: a list of modules to be merged together into the final configuration. Let’s try it out with an empty list of modules!

Note: In the following examples I will not include the code to import lib, you can find it in the previous chapter.

lib.evalModules {
  modules = [ ];
}

This is the output (not strictly evaluated):

 nix-instantiate --eval no-modules.nix
{ _module = <CODE>; _type = "configuration"; class = null; config = <CODE>; extendModules = <CODE>; options = <CODE>; type = <CODE>; }

It’s an attribute set containing various attributes. Here are the two most important for now:

We can use the -A config flag in the nix-instantiate command to access the config attribute:

 nix-instantiate --eval no-modules.nix -A config
{ }

As you might have expected, it’s an empty attribute set. This makes sense considering we passed an empty module list, meaning we didn’t declare or define any options.

Anatomy of a module

Before we can actually make the lib.evalModules function useful for us, we have to learn about the structure of a module and how to create one.

A module in Nix can be one of three things:

Notice how all three of these in the end produce an attribute set, which is the actual content of the module. This attribute set can contain the so-called “top-level” attributes, such as:

These are the other top-level attributes, but they won’t be discussed for now, as they are not important at the moment:

Let’s write a module and test it out! We’ll declare an option under options by using the lib.options.mkOption function and the lib.types.str type. We’ll then set its value in the config attribute.

let
  # In this case, we use an attribute set
  myModule = {
    # Declare `myFirstOption` in `options`.
    # Remember that a.b = x is the same as a = { b = x; };
    options.myFirstOption = lib.mkOption {
      type = lib.types.str;
    };

    # Define its value in `config`.
    config.myFirstOption = "Hello, World!";
  };
in
lib.evalModules {
  modules = [ myModule ];
}

If we now run the command to evaluate the config attribute strictly, this is what we get:

 nix-instantiate --eval --strict first-call.nix -A config
{ myFirstOption = "Hello, World!"; }

Our option is in the final configuration! Let’s dive a bit deeper in the library attributes we used.

lib.options.mkOption

This function is used to declare options in the options attribute set. It takes an attribute set containing various attributes as argument. Some of the most used attributes are:

If you’re interested in the other options and what they do, you can read the documentation for lib.options.mkOption here.

lib.options.mkOption is actually a very simple function, so much that we can just look at its source code to understand what its output is:

mkOption =
  {
    default ? null,
    defaultText ? null,
    example ? null,
    description ? null,
    relatedPackages ? null,
    type ? null,
    apply ? null,
    internal ? null,
    visible ? null,
    readOnly ? null,
  } @ attrs:
  attrs // { _type = "option"; };

It returns the attribute set we provided as input, but it updates it with the _type attribute. It’s important to note that it also checks we didn’t accidentally pass in additional attributes (as mentioned in the first chapter, argument destructuring is strict in checking).

You might be confused by the fact that it does almost nothing. How are we declaring the option then? Well, it’s simple: the heavy work is actually done inside of lib.evalModules and the functions it calls! It reads your modules, parses them, and does what you expect it to.

lib.types.str

We used lib.types.str as type when declaring the option. Let’s look at its source code to understand what it is:

str = mkOptionType {
  name = "str";
  description = "string";
  descriptionClass = "noun";
  check = isString;
  merge = mergeEqualOption;
};

It seems to use another function: lib.types.mkOptionType. It gives it a name and description for the type, along with check and merge functions. The check will be used to ensure the value to which the option is set is indeed a string. We won’t learn about lib.types.mkOptionType for now, but we’ll learn more about types later.

Note: This will be discussed in future chapters, but, as a spoiler, your NixOS configuration.nix is a module!

Building our example

Now that we roughly learned what a module looks like, we can start to build our example and learn even more about modules and what they can do. Let’s start with the simple task, which is to create a module that declares the enableGreeting and greetingMessage options:

{
  options = {
    # Note that there is a shorter way to do exactly this:
    # `enableGreeting = lib.mkEnableOption "the greeting system";`
    # would achieve the same as what we do below.
    enableGreeting = lib.mkOption {
      type = lib.types.bool;
      default = false;
      example = true; # For documentation
      description = "Whether to enable the greeting system."; # For documentation
    };

    greetingMessage = lib.mkOption {
      type = lib.types.str;
      example = "Greetings, fellow NixOS user!"; # For documentation
      description = "The greeting message to send."; # For documentation
    };
  };
};

Simple enough! Now, in order to make the module that declares and defines finalCommand based on the other configuration options, we need to take a look at modules that are functions. We can for example create a module like this:

{ ... }:

{
  /** My module's attributes */
}

That leaves the question: what attributes do we receive in the argument? Many actually! For now, we’ll just focus on one of them: config. It is one of the arguments we receive and its value is the final configuration output. Yes! The config we receive is exactly the attribute set that we evaluate when running nix-instantiate --eval --strict our-file.nix -A config: part of the output of the lib.evalModules function call. This is possible thanks to Nix’s laziness and recursion mechanics, as explained in the second chapter.

If we use it carefully, we can avoid infinite recursion errors and achieve our goal:

# Another useful argument we receive is `lib`! If this module was in another file
# and was included via path, we could use it to access `lib` anyways.
{ config, lib, ... }:

{
  # Declare the `finalCommand` option.
  options.finalCommand = lib.mkOption {
    type = lib.types.str;
    example = "echo \"Hello there!\""; # For documentation
    description = "The command to execute at every reboot."; # For documentation
  };

  # Define its value.
  config.finalCommand = if config.enableGreeting then
      # The `lib.escapeShellArg` function makes sure the greeting is escaped properly!
      "echo ${lib.escapeShellArg config.greetingMessage}"
    else
      # The `true` command does nothing and exits successfully
      "true";
}

Let’s try to pass these modules to lib.evalModules:

let
  firstModule = {
    options = {
      enableGreeting = lib.mkOption {
        type = lib.types.bool;
        default = false;
        example = true;
        description = "Whether to enable the greeting system.";
      };

      greetingMessage = lib.mkOption {
        type = lib.types.str;
        example = "Greetings, fellow NixOS user!";
        description = "The greeting message to send.";
      };
    };
  };

  secondModule =
    { config, ... }:
    {
      options.finalCommand = lib.mkOption {
        type = lib.types.str;
        example = "echo \"Hello there!\"";
        description = "The command to execute at every reboot.";
      };

      config.finalCommand =
        if config.enableGreeting then
          "echo ${lib.escapeShellArg config.greetingMessage}"
        else
          "true";
    };
in
lib.evalModules {
  modules = [
    firstModule
    secondModule
  ];
}

Let’s try to evaluate the full value of config:

 nix-instantiate --eval --strict all-declarations.nix -A config
error:
        while evaluating the attribute 'greetingMessage'

        while evaluating the attribute 'value'
         at /nix/store/7g9h6nlrx5h1lwqy4ghxvbhb7imm3vcb-source/lib/modules.nix:927:9:
          926|     in warnDeprecation opt //
          927|       { value = addErrorContext "while evaluating the option `${showOption loc}':" value;
             |         ^
          928|         inherit (res.defsFinal') highestPrio;

        while evaluating the option `greetingMessage':

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: The option `greetingMessage' was accessed but has no value defined. Try setting the option.

Looks like we got an error. The reason for this error is that we tried to evaluate config strictly, which asked for the value of greetingMessage. We never set a default for greetingMessage, nor did we set a value for it. Thankfully lib.evalModules informs us of this problem with the following part of the error message:

error: The option `greetingMessage' was accessed but has no value defined. Try setting the option.

This error doesn’t happen if we just evaluate enableGreeting though!

 nix-instantiate --eval --strict all-declarations.nix -A config.enableGreeting
false

That’s because the evaluation of enableGreeting does not depend on the evaluation of the values of the other options and Nix uses lazy evaluation, as seen in the previous chapters.

Writing the user configuration

Let’s now write the configuration file a user might write. We’ll call it configuration.nix:

{
  enableGreeting = true;
  greetingMessage = "Hey, don't forget how awesome this is!";
}

You might notice how we removed the config in config.enableGreeting and config.greetingMessage. We can do this in the module system because all modules that don’t have the options and config top-level attributes are considered to almost only contain attributes that go in config. This feature is simply a shortcut to avoid always having to write config. Note that you can still use imports and the other top-level attributes. For example, even though almost everything else is going in config, we can still use imports in this module:

{
  imports = [
    # This is another module that we include
    {
      enableGreeting = true;
    }
  ];
  greetingMessage = "Hey, don't forget how awesome this is!";
}

We can now edit our previous file to include the path ./configuration.nix as a module:

/** --- snip --- */
lib.evalModules {
  modules = [
    firstModule
    secondModule
    ./configuration.nix
  ];
}

Let’s try to evaluate the full value of config:

 nix-instantiate --eval --strict all-declarations.nix -A config
{ enableGreeting = true; finalCommand = "echo 'Hey, don'\\''t forget how awesome this is!'"; greetingMessage = "Hey, don't forget how awesome this is!"; }

It works! Our software could also just evaluate the config.finalCommand value to get the command to run at every reboot:

 nix-instantiate --eval --strict all-declarations.nix -A config.finalCommand
"echo 'Hey, don'\\''t forget how awesome this is!'"

A few more details

Now that we’ve seen a practical example for the usage of lib.evalModules, let’s take a deeper look at its features. One of those is that options can be in nested attribute sets:

# This is a module
{
  options.a.b.c.d = lib.mkOption {
    type = lib.types.str;
  };

  config.a.b.c.d = "Deeply nested...";
}

The module system knows that options.a is not an option because it doesn’t have the _type attribute set to option. It therefore assumes that it’s simply a “category” for more options or more “sub-categories”.

In case it was not already clear, options and their definitions can be split across multiple modules:

let
  module1 = {
    options.a.b.c.d = lib.mkOption {
      type = lib.types.str;
    };
  };

  module2 = {
    config.a.b.c.d = "Deeply nested...";
  };
in (lib.evalModules {
  modules = [
    module1
    module2
  ];
}).config # Evaluates to { a.b.c.d = "Deeply nested..."; }

In addition, by default, all option definitions must be for a valid option (meaning an option that was properly declared). The following throws an error for example:

(lib.evalModules {
  modules = [
    {
      config.notExisting = "Does this work?";
    }
  ];
}).config

Here it is:

 nix-instantiate --eval problematic.nix
error:
        while evaluating the attribute 'config'
         at /nix/store/7g9h6nlrx5h1lwqy4ghxvbhb7imm3vcb-source/lib/modules.nix:336:9:
          335|         options = checked options;
          336|         config = checked (removeAttrs config [ "_module" ]);
             |         ^
          337|         _module = checked (config._module);

        while calling the 'seq' builtin
         at /nix/store/7g9h6nlrx5h1lwqy4ghxvbhb7imm3vcb-source/lib/modules.nix:336:18:
          335|         options = checked options;
          336|         config = checked (removeAttrs config [ "_module" ]);
             |                  ^
          337|         _module = checked (config._module);

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: The option `notExisting' does not exist. Definition values:
       - In `<unknown-file>': "Does this work?"

       It seems as if you're trying to declare an option by placing it into `config' rather than `options'!

This check ensures that the user creates their configuration in the way that was intended.

Types

Let’s take a quick look at some (but far from all) of the types that the library gives us. You can find the full list here. Note that while previously we were looking at the nixpkgs manual, this time it’s the NixOS manual.

Basic types

There are some basic types which are pretty much self-explanatory. Some examples (but not all) are:

Union types

These types allow values that match at least one of the types specified. Their lib.types value is a function that can be used with other types. For example, there is lib.types.nullOr. This signifies that the value may either be null or the type passed as argument:

lib.types.nullOr lib.types.str # Value should be null or of type str

They include the following types:

Composed types

Composed types are similar to union types. Their lib.types attribute is a function that takes another type as argument in some way. Good examples are lib.types.listOf t and lib.types.attrsOf t. For the former, the value must be a list of elements of type t. For the latter, the value must be an attribute set where the values of the attributes are of type t.

Merging definitions

Now that we’ve seen the basics of the module system and learned about some types, we can finally talk about one of the main features of lib.evalModules: multiple definitions can be merged together. Remember how in the definition of lib.types.str we noticed that there was a merge function? The same option can be defined multiple times and the definitions merged using that function. In the case of lib.types.str, the merge function was lib.options.mergeEqualOption, which only allows merging if the values are the same. While that may not be so useful, many types allow merging in more interesting ways.

Let’s for example take a look at lib.types.attrsOf lib.types.int. According to the merge function of lib.types.attrsOf, multiple definitions that contain different attributes are merged together in an attribute set that contains all of the defined attributes:

(lib.evalModules {
  modules = [
    {
      options.attrs = lib.mkOption {
        type = lib.types.attrsOf lib.types.int;
      };
    }
    {
      attrs.first = 1;
    }
    {
      attrs.second = 2;
    }
  ];
}).config
# Evaluates to { attrs = { first = 1; second = 2; }; }

Many types are merged in the way you would expect them to. For example lib.types.listOf t merges its definitions by concatenating the lists.

Overriding values

Sometimes you might be in a situation where a module you cannot control (for example one that is included by default) sets a value for an option. If you add your own different value in your modules, the module system will try merging them. If you simply want to override the other value, you can use the lib.mkOverride function. Its source code is very simple, so we can just look at it to understand how to use it:

mkOverride = priority: content:
  { _type = "override";
    inherit priority content;
  };

It takes a priority and your value as arguments (via currying). It then returns an attribute set with a _type attribute, the priority and your value. Just like for lib.mkOption, the function can be so simple because most of the logic is handled by lib.evalModules and the various functions it uses.

The priority tells the module system whether your definition is more or less important than other definitions. The values with the lowest priority are used. If there are multiple, they will be merged. Option definitions that don’t use lib.mkOverride have a default priority of 100.

There are functions such as lib.mkForce which you can use to avoid having to specify the priority yourself. Here is the definition of lib.mkForce:

mkForce = mkOverride 50;

It can be used like this:

(lib.evalModules {
  modules = [
    {
      options.overriding = lib.mkOption {
        type = lib.types.int;
      };
    }
    {
      overriding = 1; # Uses default priority: 100
    }
    {
      overriding = lib.mkForce 2; # Uses priority 50
    }
  ];
}).config
# Evaluates to { overriding = 2; }

Note: The default value of an option is simply an option definition with priority 1500.

Summary

There is much more to learn about the module system, which is why its explanation is split across multiple chapters of this guide. Before we can continue learning more about the module system by creating a custom mini-NixOS (almost) from scratch, we must take a detour to learn about derivations, which are the topic of the next chapter. See you there!