Monitoring File Access for Dummies

Say you work for Mashley Faddison, an online dating site which handles sensitive user information. Say you suspect somebody is fiddling with files that should best be left alone. Say you want to catch said somebody in the act…

File monitoring means you can make your GNU/Linux operating system do something if a certain file, group of files, or directory get modified in any way. If a new file is created in a certain place, or another gets deleted, or a user even opens something else just to take a peek, you can know instantly, even if you are away from your desk. You can also have your system trigger an automatic action as a response.

There used to be a cool little framework called incron for GNU/Linux that sat atop the kernel’s inotify subsystem. It allowed you to pick files or directories to monitor and link changes to actions you defined. Back in the day I wrote a comprehensive tutorial on how to use incron and was going to revisit it when I discovered it had been discontinued.

It turns out incron has succumbed to The Steamroller!!!, aka systemd. systemd is now the prefered way to monitor paths and files, so I decided to explore this new fangled malarkey and find out how it held up to the old incron way of doing things.

Before we get deeper into this, let me state three things: Firstly, I have no gripe with systemd. Scratch that: I had no gripe with systemd. But I will expound on my newly acquired gripes, but I’ll vent much later on in this article. Because, hey, you’re here to get stuff solved, right? You’re here to learn how to set up you file monitoring and not read me grumble.

Secondly, even if you think systemd is way over your head, do yourself a favour and hang in there. It’s an essential piece in most modern GNU/Linux system and this article may be a good gateway in. I’m not going to pull you down into all the intricacies of the framework, in fact you will only see a small, manageable subset of the tools. These tools will allow you to pull off a simple and straightforward file monitoring system and I’ll explain everything and every step in detail. Console yourself with the thought that, before writing this article, I had virtually no idea of how to make something like this work either.

Finally, systemd hackers and professional sysadmins in the know, if you are reading this, please note that this is not exactly a systemd tutorial. I will only touch open the parts we need to get the very specific job of file monitoring done, deliberately leaving out everything else. Also note that by no stretch of the mind am I a systemd expert, so I may be using its means… er… imaginatively. If you think I should be doing something differently or that you can do things better than me, by all means share your wisdom in the comments below.

UPDATE FOR ALL THE VISITING SECURITY EXPERTS: Thanks for dropping by. Seriously. But note this is not exactly a security tutorial either. I am well aware there are better tools for the job. In fact, that is exactly what I say down in the conclusion. This is an introduction to messing about with systemd units, especially .path units from the perspective of an end user. The spying-on-file-activity is an excuse to give the project some purpose.

With all that cleared up, let’s dig in.

systemd killed the incron star

The first thing you have to know about systemd is that it relies on a series of configuration files called units that define how and when services get started and stopped. “Services” in this context are scripts or programs that, as their name implies, serve the apps you, as a user, interact with. If you need to print from LibreOffice, for example, you’ll probably go through a CUPS server. The cupsd daemon that allows you to access the printer is a service and it is controlled by systemd. If you are running an SSH server on a remote machine and you use it to access and work with the host, the SSH daemon sshd is a service controlled by systemd.

You can probably see already where this is going: the automatic action we talked about above, that thing you want the system to fire when someone meddles with your files, will be a service. You tell systemd how and when to start and stop a service with a .service file (“unit”) that usually lives in the /etc/systemd/system/ directory.

Your first unit, you can call it mymonitor.service, will look like this:

[Unit]
Description= Starts the logging script
Documentation= man:systemd.service

[Service]
Type=oneshot
ExecStart=/home/[yourusernamehere]/bin/mymonitor.sh

A unit, as you can see, can be divided into several sections. The title of each section is enclosed in square brackets ([...]) and each section contains one or more option-value pairs. In the example above, you have a [Unit] section, which usually contains a description of what the unit does, a link to documentation, say a man page, other units it needs to activate so the daemon can run correctly, etc. To see what other options you can include in the [Unit] section check out the systemd.unit man page (man systemd.unit).

The [Service] section tells systemd the name and location of the actual program or script that needs to be run. It can also contain the Type option that tells systemd how the daemon will be run. In this case, we want to run it once and exit (a oneshot), because we’re going to be calling it every time a certain event happens. We don’t want it hanging around in memory or anything like that. You can read up about other directives that you can use in the [Service] section in the systemd.service man page.

But, look here! You just wrote your first unit! Save it in /etc/systemd/system/mymonitor.service. You will need root privileges to do this (we’ll be seeing how to do all this as a regular, non-root user a bit later in the article), and, when you’re done, let’s move on to the actual executable that will do the dirty work.

Say you want to log when someone messes with your files. The actual daemon could be a two-line Bash script that looks like this:

#!/bin/bash
 echo `date` 'Hey! Someone touched your stuff!' >> /home/[yourusernamehere]/fileMonitor.log

This script writes into a fileMonitor.log, a file that resides in your /home directory, a line with the date and time and a message that says “Hey! Someone touched your stuff!” every time it’s called.

Save that as mymonitor.sh in your /home/[yourusernamehere]/bin directory.

Set up

systemctl st

systemctl stat

systemctl status mymo

systemctl status mymonitor.service

As a regular, non-root user you can check on the state of your service with:

systemctl status mymonitor.service

This will show something like what you can see below.

You may be tempted to try and run this service. If you have a passing knowledge of systemd and the systemctl utility, you will be tempted to execute

systemctl start mymonitor.service

That would be the normal way of firing up a service. But mymonitor.service is lacking an [Install] section. Although we’ll talk about this section later, you must know that without it, a unit cannot be directly enabled or started. This is fine, because, remember, our monitoring service must be triggered by a system event, namely, when a file or directory gets fiddled with. It is not intended to be started by hand or when the OS boots.

Let’s get the triggering mechanism up and running.

Apart from regular service units, systemd also has path units and, as the systemd.path man page states,

A unit configuration file whose name ends in “.path” encodes information about a path monitored by systemd, for path-based activation.

which is exactly what you want.

Your path unit, which you’ll call mymonitor.path, would look something like this:

[Unit]
Description= Triggers the service that logs changes.
Documentation= man:systemd.path

[Path]
PathModified=/home/[yourusernamehere]/my_monitored_dir/

[Install]
WantedBy=multi-user.target

Save that into /etc/systemd/system/mymonitor.path.

mymonitor.path contains three sections. First up is the [Unit] section you saw above. No mysteries there.

Then you have a [Path] section. This is specific for .path units. This section contains information about what has to be monitored and what changes to look out for.

It can also contain a Unit= option that tells systemd the the unit to fire when the monitored change actually takes place. A propos of this, that the .path unit and .service unit share the same name in this tutorial is not a coincidence: unless told otherwise (with the Unit= directive mentioned above), systemd will assume that the unit it has to activate when the event PathModified= is triggered shares the same name as the .path file. So, .path and .service units with the same names = no need for Unit= directive.

Note that apart from PathModified=, you can monitor different things, such as PathExists=, PathExistsGlob=, PathChanged=, and DirectoryNotEmpty=. The difference between each trigger can be subtle, so check carefully what each does by reading the systemd.path man page. You rill use the PathModified= directive because it more or less covers everything, so it will be triggered even when a file is just looked at.

Finally you have the [Install] section which tells systemctl (NOT systemd) what environment the unit needs to run. In this case, it needs a multi-user level of execution (i.e., systemd’s equivalent of runlevel 3 for us old fogies). This is useful when you want to make your service start at runtime: until the multi-user environment is set up, systemctl will not start the service.

Now you have your .path unit, go ahead and create the my_monitored_dir/ directory in your home directory. Once you’re done, you can get down to business.

Run, service, run

Now you are ready start your monitoring service. Make yourself root and execute

systemctl start mymonitor.path

and then

systemctl status mymonitor.path

to check mymonitor.path has started correctly. You should see something like what’s shown on the right.

Let’s try it out. Try creating a file in my_monitored_dir/:

touch my_monitored_dir test

And then take a look in fileMonitor.log. You should see something like what is shown below.

If anything goes wrong and your service is not working, check the status of both your units. If you run

systemctl status -l mymonitor.path
systemctl status -l mymonitor.service

you’ll get a breakdown of the errors that may have stopped your service from working.

Improvements

The first thing you can do to improve your monitoring service is have it start when the computer boots. To do this execute

systemctl enable mymonitor.path

as root.

If you ever want to take mymonitor.path off the list of services that run automatically at boot, just do the opposite:

systemctl disable mymonitor.path

The second thing you can do is skip all the root thing and actually run your file monitor as a regular user. To do this, copy the mymonitor.* files you created earlier into your home directory under .config/systemd/user/ (if this subdirectory doesn’t exist, create it). Now you can run all of the above as a regular, non-root user with systemctl --user .... To start the service, for example, you’d use:

systemctl --user start mymonitor.path

And to check its status, you’d do:

systemctl --user status mymonitor.path

and so on.

Finally, just logging is lame! You can have your script email or text you on your phone when an intruder tries to mess with your stuff. You’ll know immediately your files are being tampered with! Or you could have your script take the server offline to keep it protected. The possibilities are endless.

Caveats

Although using systemd to monitor files and directories for changes is not hard, it is also far from ideal. For starters, when watching a directory, there is no way to know which file or subdirectory triggered the action, and there is also no way to know what event did the triggering — was a file deleted, created, read from, or written to?

Something else that is quite annoying is that, although you can observe several directories and files by adding more lines to your .path file, it would be nice to, instead of having to hard code them into the unit, be able to pass them through using an environmental variable. But, no, the [Path] directives will only take explicit, full and absolute routes to whatever it is you want to watch.

Conclusion

If I understand the philosophy of systemd correctly, one of the things the developers sought to achieve was to keep the declarative part of services separate from their logic. Maybe in the grand plan of things that is desirable.

However, for the task explained here, and unless I am missing something, this seems to make the current systemd tools pretty underpowered and a bit clunky compared to the deprecated incrond-related and other more modern utilities.

Talking of which, if this is intentional, if systemd’s architects wanted to delegate real file monitoring functions to other tools, why did they bother implementing the lacklustre path system at all? As I say, I think I am missing something here.

That said, systemd does provide a quick and rather painless way of setting up basic file/directory monitoring, and, as there are other tools that provide more granular control and feedback, you can combine the former with the latter to set up a really sophisticated monitoring system quite easily.

Cover Image: Looking through the Tube by Mateusz Stachowski for FreeImages.com