NixOS configuration restructuring #38
Labels
No Label
Contributions welcome
Service packaging
bug
duplicate
enhancement
help wanted
invalid
question
wontfix
No Milestone
No project
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: SelfPrivacy/selfprivacy-nixos-config#38
Loading…
Reference in New Issue
There is no content yet.
Delete Branch "%!s(<nil>)"
Deleting a branch is permanent. Although the deleted branch may exist for a short time before cleaning up, in most cases it CANNOT be undone. Continue?
Simplified diagram of NixOS configuration with its static and variable dependencies. Many services are omitted (in order not to clutter the diagram) and represented as "some service X.nix".
So, the variable inputs are:
hardware-configuration.nix
(which is not clear howgenerated bynixos-infect
)userdata.json
(which is generated bynixos-infect
at first and by selfprivacy-api service further)tokens.json
(which is generated by selfprivacy-api service)userdata.json
andtokens.json
are generated by selfprivacy-api, but it's not clear when exactly. Most likely we should expect that each API request may result in updated files.Flakes integration variant 1 (pure)
In this case local flake.nix example (incomplete):
The git work directory does not reside on the NixOS machine. But
/etc/nixos/flake.nix
(and eventually/etc/nixos/flake.lock
) does reside.Commands to run on NixOS machine:
Flakes integration variant 2 (impure)
The git work directory resides on the NixOS machine.
Commands to run on NixOS machine:
In the pure case custom user overlays can be placed on NixOS machine and imported the same way as
hardware-configuration.nix
(as an argument to a function from configuration repository).Flakes integration variant 3 (stateless)
Another way exists. Maybe it's the best, because allows to keep the minimum possible number of files on a NixOS machine.
This commit contains such example. Build with:
assuming, that the actual
userdata
andhardware-configuration.nix
are in your current directory.Idea for automatic upgrades:
Source: https://stephank.nl/p/2023-02-28-using-flakes-for-nixos-configs.html#using-hosted-git.
What is the current upgrade algorithm for nixpkgs input? And how is it expected to work with flakes?
Current related research task -- how to determine which commit is the current running NixOS system built from?
How to switch a NixOS machine to a configuration, stored in a remote git repository using variant №3?
Run this on a machine, where
NIXOS_CONFIG_COMMIT
is your desired commit sha1:You can replace
switch
totest
orbuild
, etc if you need othernixos-rebuild
actions.How to get commit sha1 of the currently running NixOS configuration?
This is
plannedconsidered, but doesn't work in conjunction with--override-input
method.configuration switching workflow using variant №1
initialize a local git repository
put (or update) these files:
flake.nix
hardware-configuration.nix
userdata.json
commit changes in the local git repository:
$ git add . && git commit -m "<some changes in userdata/hardware-configuration.nix/flake.nix>"
get latest config from selfprivacy git:
$ nix flake lock --update-input selfprivacy-nixos-config --commit-lock-file
# nixos-rebuild switch --flake git+file://${REPO_PATH}
Step 0 needs to be executed only during the deployment (i.e. nixos-infect).
Step 1 is mostly needed for changes in user configuration (so that we have the whole history and can provide rollbacks hermetically). Changes to
flake.nix
are not needed normally (exceptions are: moving or adding more user json files). Changes tohardware-configuration.nix
are rare.Step 2 is needed to fix the current state of all components required for the build. "Dirty" builds are not allowed to maintain reproducibility and traceability. So,
nixos-version
will return its origin git commit sha1 of the local repository:[local git commit] => [selfprivacy-nixos-config commit] => [official nixpkgs commit]
Step 3 is meant to update (or create if missing) the
flake.lock
file.flake.lock
file contains URLs and hashes of inputs, required for building. Hence, the configuration is fully pinned (locked) to exact software snapshots and reproducible.Step 4 is for building and applying the configuration.
how to
nixos-rebuild
a NixOS configuration from a flake function right nowflake https://git.selfprivacy.org/SelfPrivacy/selfprivacy-nixos-config.git?ref=flakes returns a function, which cannot be built alone, hence requires
userdata
andhardware-configuration
arguments to be passed from a local machine.prerequisites
Nix installed (multi-user installation) with flakes enabled.
basic steps
Put on a local machine this
flake.nix
(along withuserdata
andhardware-configuration.nix
):Override git branch (only because currently master branch has no flakes):
Build NixOS configuration:
further steps
Update
selfprivacy-nixos-config
input. Just run this again:In the future
nix flake lock --update-input selfprivacy-nixos-config
will be enough, when flakes get merged into master branch.Clone
selfprivacy-nixos-config
repository locally for rapid development build iterations:Return back to remote SelfPrivacy repository builds:
After today's meeting with @inex, the decision was made to use variant #1.
Benefits of the variant #1 are:
have all resources to reproduce a build of each past configuration, which variable files must be either:
/etc/selfprivacy-config-source
)This means giving User an opportunity to revert any changes, go back and continue configuration from any point in time (given the system remains accessible via API, otherwise restoring from backups is needed).
not letting
userdata.json
inside /nix/store (in case local git repository is not used)Although, its configuration fields are still used in the build (thus get into /nix/store in one way or another), it allows us to migrate quicker, because
userdata.json
must not be freed from sensitive data.potential ways to merge configuration with User supplied NixOS modules
traceability of all build ingredients including a sha1 of
selfprivacy-nixos-config
(which is not possible when using--override-input
as in variant #3)SP modules (extensions/bundles/packages/etc)
SP module (preliminary naming) is a Nix attribute set or a Nix flake, containing:
configuration.nix
SP module can be downloaded from a remote git or HTTP server location.
SP modules remote references must be specified somehow to be included in the configuration build.
If reading references from a JSON file, the
builtins.readFile
must be called from the top-level flake (e.g./etc/nixos/flake.nix
). Otherwise, such errors occur:hub
If we maintain an SP modules hub containing many SP modules, it can be used as a a single flake input or each module can be added separately using the
dir
parameter, like this:git+https://git.selfprivacy.org/SelfPrivacy/selfprivacy-hub?dir=mailserver
.transitive dependencies control
If we import some Nix code it can bring lots of dependencies including its own nixpkgs (either via flake inputs or builtins.fetchTarball, etc). In case it is a flake, we can override nixpkgs with our own using the
follows
attribute. However, it needs to be done for each input regardless of its depth in the dependency tree:Or we can directly edit
flake.lock
with some scripts like these:follows
to all mutual dependencies)In case of pure imports without using flake inputs like this:
there is no way to override such dependencies.
Nix flakes have not yet established a way to define an output schema definition. But it seems we do not depend on such feature. We can use any output attributes we want.
This article shows that configuration parts can be put right into
nixosModules
flake attribute and merged with original one by putting them as a list intoextendModules
attribute of original configuration.This can be used for SP modules. However, adding to
modules
looks to be enough.Although, we also need to pass https://nixos.wiki/wiki/Flakes documentation
Latest local flake configuration example with SP modules prototype: https://git.selfprivacy.org/alexoundos/selfprivacy-nixos-top-level.
Developing auto-rollback mechanism.
The diagram above is considered useless in case we configure auto-upgrades.
But in case a User adds a new custom SP module, it might be useful.