Processes with brackets in `ps` output

Have you ever wondered why some processes have brackets around their names?

$  ps ax
  PID TTY      STAT   TIME COMMAND
    1 ?        Ss     0:03 /usr/lib/systemd/systemd --switched-root --system --deserialize 24
    2 ?        S      0:00 [kthreadd]
    4 ?        I<     0:00 [kworker/0:0H]
    6 ?        I<     0:00 [mm_percpu_wq]
...

The COMMAND column shows the command that was used to start that process and all its arguments in a string. By reading the ps(1) man page, you will find out that the brackets mean the arguments weren’t available. But where do they come from exactly?

First, let’s find out where ps itself comes from:

$ rpm -qf /usr/bin/ps
procps-ng-3.3.10-15.fc27.x86_64

Ah, the /proc filesystem!

This is an file-based interface to internal data structures in the kernel. Each process gets a directory under /proc with a bunch of files that make it easy to retrieve that information. We are interested here in the cmdline file:

Table 1-1: Process specific entries in /proc
..............................................................................
 File		Content
 clear_refs	Clears page referenced bits shown in smaps output
 cmdline	Command line arguments
 cpu		Current and last cpu in which it was executed	(2.4)(smp)
 cwd		Link to the current working directory
 environ	Values of environment variables
 exe		Link to the executable of this process
 fd		Directory, which contains all file descriptors
 maps		Memory maps to executables and library files	(2.4)
 mem		Memory held by this process
 root		Link to the root directory of this process
 stat		Process status
 statm		Process memory status information
 status		Process status in human readable form
 wchan		Present with CONFIG_KALLSYMS=y: it shows the kernel function
		symbol the task is blocked in - or "0" if not blocked.
 pagemap	Page table
 stack		Report full stack trace, enable via CONFIG_STACKTRACE
 smaps		an extension based on maps, showing the memory consumption of
		each mapping and flags associated with it
 numa_maps	an extension based on maps, showing the memory locality and
		binding policy as well as mem usage (in pages) of each mapping.

Now that we know where this information is coming from, let’s dive into the procps-ng source code.

By using your favorite code editor or simply grep, you will find out that the read_unvectored() function is called to read the contents of /proc/%u/cmdline and will return zero when there’s nothing in it. The fill_cmdline_cvt() function then calls escape_command() with the ESC_BRACKETS flag, which adds the brackets we see in ps’ output.

You can check for yourself that such processes really don’t have arguments:

$ stat /proc/2/cmdline
  File: /proc/2/cmdline
  Size: 0         	Blocks: 0          IO Block: 1024   regular empty file
Device: 4h/4d	Inode: 202         Links: 1
Access: (0444/-r--r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
...

But why not? Most commonly, these will be kernel threads implementing helper functions, specific subsystems, work queues, etc. These tasks will register as system time in top.

They can also be processes which were execve‘ed with an empty list of arguments; or even a process that overwrote its argv[] with empty data. These occur less often.

If you’re curious to understand what these processes do in more detail, see below. This list includes links to useful information about each process.

The Documentation directory in the Linux kernel sources is an invaluable resource. And there’s always the source code itself if you’re in doubt about something (thanks open source!).