Designing a Linux Resource Manager in C++

By the fervor of Linus Torvalds, there does not exist any immediate C++ or OOP interfaces to operating system services. Consequently, it is sometimes necessary to wrap a logical set of Linux system calls in a C++ wrapper.

In this post, I will demonstrate a standard process of wrapping the resource limitation and usage system calls in a ResourceManager singleton service while utilizing some nifty C++ tricks such as CRTP.

Motivation

A procedural style offers the same behavior as an OOP-style would; however, OOP makes optimal use of abstractions by providing common design patterns. Consequently, refactoring will be made possible.

Because this tutorial primarily regards the design of the resource manager, I will not provide implementation details beyond the header declarations; however, the full source can be found on my GitHub repository.

Linux Resource Limitation and Usage Specifications

Before designing a C++ wrapper, we must begin by noting some of the nuances in the Linux specification. In particular, the system calls associated with resource limitations and usage can be found in the Linux manual pages as getrlimit(), setrlimit() and getrusage(). The manual pages describe them as follows:

The getrlimit() and setrlimit() system calls get and set resource limits respectively. Each resource has an associated soft and hard limit, as defined by the rlimit structure.

getrusage() returns resource usage measures for who, which can be one of the following: RUSAGE_SELF, RUSAGE_CHILDREN, RUSAGE_THREAD. The resource usages are returned in the structure pointed to by usage.

Although these system calls are relatively straightforward, they also require a few constants and structures: rlimit and usage. The constants such as RLIMIT_CPU simply define the type of resource to limit; these constants can simply be referred to in the manual. The structures are defined as follows:

struct rlimit {
	rlim_t rlim_cur;  /* Soft limit */
	rlim_t rlim_max;  /* Hard limit (ceiling for rlim_cur) */
};

struct rusage {
	struct timeval ru_utime; /* user CPU time used */
	struct timeval ru_stime; /* system CPU time used */
	long   ru_maxrss;        /* maximum resident set size */
	long   ru_ixrss;         /* integral shared memory size */
	long   ru_idrss;         /* integral unshared data size */
	long   ru_isrss;         /* integral unshared stack size */
	long   ru_minflt;        /* page reclaims (soft page faults) */
	long   ru_majflt;        /* page faults (hard page faults) */
	long   ru_nswap;         /* swaps */
	long   ru_inblock;       /* block input operations */
	long   ru_oublock;       /* block output operations */
	long   ru_msgsnd;        /* IPC messages sent */
	long   ru_msgrcv;        /* IPC messages received */
	long   ru_nsignals;      /* signals received */
	long   ru_nvcsw;         /* voluntary context switches */
	long   ru_nivcsw;        /* involuntary context switches */
};

// necessary for struct rusage
struct timeval {
	time_t         tv_sec      /* seconds */
	suseconds_t    tv_usec     /* microseconds */
};

These structures can easily be transformed into classes. However, before we design the resource manager, let us describe the difference in programming styles that we expect.

Programming Styles

These structures along with the aforementioned system calls entails a procedural style of resource management. Consider the following procedural code:

struct rlimit cpu;
cpu.rlim_cur = 3;
cpu.rlim_max = 3;
setrlimit(RLIMIT_CPU, &cpu, NULL);

This takes quite a few lines that is often easily reduced to a single line by an OOP style:

ResourceManager.set_resource_limit(ResourceLimit(RLIMIT_CPU, 3, 3));

Although obviously longer as a single line, this does not logically conflict with an OOP programming paradigm. There is no performance optimization by changing the style to OOP; however, placing the code into a wrapper enables us to handle the code as a cross-cutting concern and therefore enable refactoring when the time comes. Recall that the primary benefit is making use of design patterns that OOP offers.

The Structural Design

Before we implement this wrapper, we must consider the high-level design. Allow us to begin with a driver program as our test case which will outline how we would like to use the interface. For simplicity, we will use main as our driver.

int main(int argc, char* argv[])
{
	// Acquire a single instance of the manager
	ResourceManager& manager = ResourceManager::instance();

	// Set resource limits
	manager.set_resource_limit(ResourceLimit(RLIMIT_CPU, 3, 3));
	manager.set_resource_limit(ResourceLimit(RLIMIT_RTTIME, 3, 3));
	manager.apply_limits();

	// Do some computation?
	fib(31);

	// Obtain a resource usage report
	ResourceUsage usage = manager.get_resource_usage(RUSAGE_SELF);

	// Print the resource usage report
	cout << "User Time (sec): " << usage.utime().seconds << endl;
	cout << "User Time (usec): " << usage.utime().microseconds << endl;
	cout << "System Time (sec): " << usage.stime().seconds << endl;
	cout << "System Time (usec): " << usage.stime().microseconds << endl;

	return 0;
}

Alright, so we have shown how we would like to use this interface. We will now consider the objects in use.

ResourceManager

Service that applies and tracks our resource limits.

ResourceLimit

Descriptor of a resource limit.

ResourceUsage

Descriptor of a resource usage report.

We must now consider a few nuances to our objects. First, we will look at the ResourceManager. In particular, it is only logical to have one, global instance of a ResourceManager because it applies to the entire process. Consequently, we should use the Singleton pattern. Further, because the manager tracks the currently applied resource limits and there exists a finite (and uniquely identifiable) set of resource limits, we will use a std::set data structure provided by STL.

The singleton design pattern ensures that only one instance of a class is instantiated throughout the lifetime of a program.

Since ResourceLimit and ResourceUsage are simply descriptors, it is easy enough to implement them as straightforward classes. Operationally, ResourceLimit should be able to apply() itself. ResourceUsage should be immutable because its data is consistently changing relative to the speed of our processor; consequently, it is not easily observable.

Now that we have outlined the structure, we will begin implementing the entire component.

Descriptor Design

We will begin with the obvious, descriptor implementation. Specifically, we will begin with the ResourceLimit implementation. The header file should proceed as follows:

#include "sys/resource.h"

class ResourceLimit
{
public:
	// constructors
	ResourceLimit(int);
	ResourceLimit(int, rlim_t, rlim_t);
	ResourceLimit(int, rlimit&);

	// accessors
	rlim_t soft_limit() const;
	rlim_t hard_limit() const;

	// native accessors
	const rlimit& to_rlimit() const;

	// operational functions
	void get_limit();
	void apply();

	// comparisons
	bool operator==(const ResourceLimit&) const;
	bool operator<(const ResourceLimit&) const;
private:
	int resource_id_;
	rlimit limit_;

	bool applied_;
};

As we can see, the necessary accessors are provided and the system-provided rlimit structure is hidden within the class. Further, the comparison operators are provided for when the class is used in the ResourceManager's set.

Before we begin designing the ResourceUsage class, we must define a structure to replace the system-provided timeval structure.

/**
 * An alternative representation of timevals
 */
class TimeParts
{
public:
	long seconds;
	long microseconds;

	// constructors
	TimeParts(long seconds, long microseconds) :
		seconds(seconds), microseconds(microseconds) {}

	TimeParts(timeval tv) :
		seconds(tv.tv_sec), microseconds(tv.tv_usec) {}
};

Now that we have redefined a class for timeval, we begin the ResourceUsage class which is, of course, immutable.

Immutable objects are non-modifiable after creation. They are necessary for implementing systems in which side effects are circumvented.

#include "sys/resource.h"
#include "sys/time.h"

/**
 * A descriptor of the resource usage of the
 * current process. This object is immutable
 * after construction.
 */
class ResourceUsage
{
public:
	// constructors
	ResourceUsage(int);
	
	// time accessors
	TimeParts utime() const;
	TimeParts stime() const;

	// accessors
	long maxrss() const;
	long ixrss() const;
	long idrss() const;
	long isrss() const;
	long minflt() const;
	long majflt() const;
	long nswap() const;
	long inblock() const;
	long oublock() const;
	long msgsnd() const;
	long msgrcv() const;
	long nsignals() const;
	long nvcsw() const;
	long nivcsw() const;
private:
	int who_;

	rusage usage_;
};

This descriptor also directly models the structure provided by the system; however, note that this design enables absolute immutability. With C-style structures, the data fields would be modifiable which is absolutely unnecessary.

Now that we have designed the descriptors, we can proceed with the design of the manager.

Resource Manager Design

We begin by noting that that the ResourceManager must obviously have access to the ResourceLimit and ResourceUsage descriptors and begin by including their headers into the source. Furthermore, we include the necessary resource management headers provided by the system and the set data structure provided by STL as we have intended.

#include "ResourceLimit.h"
#include "ResourceUsage.h"

#include "Singleton.h"

#include "sys/resource.h"
#include "sys/time.h"

#include <set>

Next, we design the manager to provide the same operational functionality that the Linux system-calls do; however, we design it in the context of sets of resource limits.

class ResourceManager
{
public:
	// mutators
	bool set_resource_limit(const ResourceLimit&);

	// accessors
	ResourceLimit get_resource_limit(int) const;
	ResourceUsage get_resource_usage(int) const;

	// operational functions
	void apply_limits();
private:
	std::set<ResourceLimit> resource_limits_;
};

This manager enables us to set defer resource limit application until necessary. Additionally, because we use a set data structure, no duplicate resource limits will overwrite each other. Now, we are only missing a single feature: the singleton.

Singleton Application

Although we can use the standard approach by manually embedding the operational functionality of a singleton into our ResourceManager, we can use a modern C++ idiom, the curiously recursive template pattern (CRTP). Specifically, the singleton will have a template parameter which will be our derived class. The singleton will then inject the singleton operational functionality into the derived class.

/**
 * Guarantees that only a single instance of an object will exist
 * throughout the lifetime of the program.
 */
template <class Derived>
class Singleton
{
public:
	Singleton(const Singleton&) = delete;
	Singleton& operator=(const Singleton&) = delete;

	static Derived& instance()
	{
		if (instance_ == nullptr)
			instance_ = new Derived();
		return *instance_;
	}
protected:
	Singleton() {}
	static Derived* instance_;
};

template <class Derived>
Derived* Singleton::instance_ = nullptr;

Here, notice that the singleton also uses some C++11 features where we delete the copy constructor and the copy assignment operator. Furthermore, notice that singleton operational functionality can now simply be inherited.

Finally, we can then inherit from our singleton,

class ResourceManager : public Singleton<ResourceManager>
{
public:
	// mutators
	bool set_resource_limit(const ResourceLimit&);

	// accessors
	ResourceLimit get_resource_limit(int) const;
	ResourceUsage get_resource_usage(int) const;

	// operational functions
	void apply_limits();
private:
	std::set<ResourceLimit> resource_limits_;
};

Note that it will be even easier to implement further singleton functionality if necessary; however, this component no further singletons.

At this point, our design is complete! Do not forget that the full source code is available in my GitHub repository.

Conclusion

It is easy to see that there was much duplication in the descriptor classes; however, it is a necessary evil to provide further constraints that classes provide such as immutability. Furthermore, use of the constructor enables quick initialization of objects as we have seen the driver.

Furthermore, utilizing OOP in C++ enables use of design patterns such as the singleton through CRTP. This allows us to design the resource manager as a cross-cutting concern and think of it as a logical abstraction rather than as an implementation detail.