Ugh! - Software, Music and more…

Setting up linux as a audio/MIDI workstation is not a trivial task. On this page i try to collect knowledge on the issue. These pages will describe the steps involved in a distribution independent way. But note that i myself run Ubuntu Linux which might shine through.

These are the major steps in getting reliable low latencies and tight midi timing:

• Allowing ordinary users to run tasks SCHED_FIFO
• Setting up the audio part of the system
• Setting up the Midi part of the system

But before we dive into the details of these some introductory words on Audio processing, MIDI processing, POSIX scheduling classes, and realtime kernels are needed.

Basics

Scheduling

The first thing i would like to discuss is scheduling classes. A typical PC’s CPU is a serial device. It executes one command after the other. The illusion that many programs run at the same time on a computer, called “multitasking”, is produced by switching from one program to the next very fast. The individual program does not need to keep track of this though. It is done transparently by the operating system. Some modern systems have multicore CPUs or even multiple CPUs. In this case it is actually possible to run more than one program truly at once (limited by the number of available CPUs). As soon as the number of programs running exceeds the number of CPUs available we are back to above situation. So this doesn’t really change anything.

As a CPU really is a serial device which only can run one program at a time (before switching to the next one) it is a resource that needs to be managed. Different programs should be allowed to use more CPU time than others, Different users should be allowed to use more or less CPU time than others, etc. For this reason POSIX (and Linux implements large parts of the POSIX specification) provides means of telling the operating system how much CPU time to give to each program when they compete for it (There might be times when a program is idle, waiting e.g. for keyboard presses. At these times it does not compete for CPU time). The mechanism most commonly known to Linux and UNIX users is the “nice” value. This value can be set for each process and it says how “nice” it is, i.e. how willing it is to give up the CPU. A large negative nice value means much reluctance to giving up the CPU and a large positive number means great willingness to do so.

How exactly this mechanism is implemented varies greatly from one POSIX system to the next and even between different versions of the LINUX kernel. And most of it all, this mechanism makes no guarantees at all about which process gets the CPU. The different kernels might employ very different strategies to provide an optimal usage of the resource CPU for many common usecases. Realtime processing is not one of these ;)

Realtime Scheduling

Realtime processing means roughly that there are deadlines which must not be missed. Take for example an application processing some audio. The soundcard records buffers filled with audio, hands these to the program, the program processes the buffer and hands it back to the soundcard which then plays it back. The buffer size is usually a constant as is the samplerate. This means that for each buffer, the time available to process it is exactly the time until the next buffer arrives (which is buffersize/samplerate seconds if we specify buffersize in samples without taking the channel count into account). Failing to meet this deadline means that a buffer could not processed in time and thus the soundcard can not playback the missing buffer generating a click in the sound.

For this reason, the POSIX specification also includes different scheduling mechanisms. The mechanism mentioned in the previous section implements a scheduling class called SCHED_OTHER. Never mind the name :) Basically all processes on a normal Linux system run in that scheduling class. But there are also others suitable for certain other usecases:

• SCHED_FIFO
• SCHED_RR
• SCHED_BATCH
• SCHED_RR

The one we are interested right now is SCHED_FIFO as this class is perfectly suited for realtime processing in audio/MIDI systems. The central property is that all processes running in the SCHED_FIFO class get the CPU before any SCHED_OTHER tasks and they can hold it as long as they want until they voluntarily give it up (e.g. by waiting for user input or the next soundcard buffer). This is perfect for realtime purposes because no SCHED_OTHER task can “disturb” SCHED_FIFO tasks (this is not totally true though. We will come to this in a bit). The SCHED_FIFO scheduling class has something similar to SCHED_OTHER’s nice values. Each task has a prority from the range 1 to 100 (or 0 to 99, depending on how you count). If two SCHED_FIFO processes compete for the CPU the one with the higher prio wins and if they have the same prio, the one who asked first is served first.

Interrupts

All that’s said about scheduling until now is true for the case that the programs involved don’t communicate with the outside world. As soon as communication with the outside world is done there has to exist a mean to

• Send data out to the outside world
• Receive data from the outside world

The CPU does not communicate with the outside world directly. It uses peripheral devices like network adapters, IDE controllers, etc., for this purpose. These devices are typically connected to the CPU via the PCI bus (or AGP or whatever). If the CPU wishes to send out data it simply writes the data on the PCI bus using some protocol to make sure the data reaches the right device and be done with it. But how does it receive data? One way that wastes precious resources is called polling. The CPU simply checks from time to time whether any device has data for it. This is not the optimal solution in many cases. Another scheme is used. When a network adapter receives a packet of data it notifies the CPU of this by means of a so called “interrupt”. Each PC system has a number of “interrupt lines” which are typically numbered. These are special connections on the e.g. PCI bus which PCI devices can use to signal the CPU that they want its attention. The CPU reacts by directly jumping to an interrupt handler routine that he operating system can “install” (the CPU maintains a list of memory addresses, one for each interrupt, containing the address of the corresponding interrtupt handler routines).

It is important to understand that interrupts truly interrupt the normal scheduling and the interrupt routines installed by the operating system can be rather large and complex. In a normal non-realtime-preemption linux kernel all interrupt routines have priority even over SCHED_FIFO processes. That means that even if your system is perfectly setup with audio and midi programs using the appropriate SCHED_FIFO scheduling, etc., you cannot be sure that there will not be any interruptions because there are still the interrupts. Some network adapter driver’s interrupt routines can be several tens of milliseconds long. Enough to cause dropouts and crackles due to missed deadlines in an audio program that tries to achieve latency in the tens of milliseconds.

This is one of the biggest strengths of the linux kernel with realtime preemption: The interrupt handlers are SCHED_FIFO tasks and the user can set their priorities. This way one can set all non-audio interrupts to priorities lower than the audio processing program and thus eliminate this source of deadline misses.

Kernel Latencies

There exists another source of possible deadline misses for realtime programs: Kernel latencies. Once a task has made a system call no other task is allowed to run unless the system call blocks and puts the process to sleep (this is an oversimplification. It depends on the kind of system call and on whatever is going on on the system at the moment). So if the system call takes a long time to do its thing this might cause our realtime audio processing program to miss its processing deadline even though the system call was made by some low priority SCHED_OTHER process.

The realtime preemption kernel comes to the rescue here again. It has reduced kernel latencies to the order of microseconds (and even nanoseconds for some system calls) thus eliminating this remaining source of possibly missed deadlines for realtime programs.

Realtime Preemption vs. Realtime Scheduling

I want the reader to understand that realtime scheduling and realtime preemption are two different but related things. Realtime scheduling enables processes to use the SCHED_FIFO scheduling class and the realtime preemption patches make the interrupt handlers SCHED_FIFO (so they can get lower prios than some SCHED_FIFO user processes) and reduce the kernel latencies.

Allowing ordinary Users to run SCHED_FIFO Processes

On a normal linux system usually only root is allowed to make processes SCHED_FIFO. But there are several mechanisms to also allow non-root, ordinary users to do this. The oldest of these is the realtime-lsm, a kernel module that was simple to use. This was rejected by the kernel developers due to the lack of flexibility. They implemented a scheme using the proven ulimit and related mechanisms and introduced a new limit in the kernel: rtprio. Modern distros allow to set this limit via the normal logon mechanism using PAM. On my ubuntu linux system this file is called

and it looks like this:

This allows normal users on my system to run processes in the SCHED_FIFO class with a maximal priority of 95. If your distribution does not support this mechanism, there is a little program called set_rtlimits or something similar to achieve this.

Setting up the Audio Part of the System

The central part of most linux audio systems is the Jack Audio Connection Kit (http://jackaudio.org). It is an audio server designed from the ground up to provide functionality to interconnect different applications without adding additional latency overhead.

Setting up the MIDI part of the System