Intro to using GNU/Linux for C++ Programming - Part 3: GNU Make



GNU Make

GNU Make manual

This next section should reassure those who saw the above gcc commands as being a cumbersome way to do things compared to using a graphical IDE. Fortunately, the utility GNU Make is here for that purpose and in fact it's general purpose enough that you could use it to manage any project, not necessarily code, that needs to update files based on the contents of certain other files changing.

When you run 'make', the utility is going to look for a makefile in the current directory. This could be named 'makefile', 'GNUmakefile', or 'Makefile', the latter being the most common (the capitalization is to group it with other special 'non-source' files like 'README', etc.). This file will inform make of the dependencies in your project, and the 'recipes' to update them.

Make Rules

We will start with a basic example:
$ nano Makefile
$ cat Makefile
all: runme

runme: mymain.o mycode.o
    g++ -Wall -g -o runme mymain.o mycode.o
clean:
    rm -f *.o
    rm -f runme
$ make clean
rm -f *.o
rm -f runme
$ make
g++    -c -o mymain.o mymain.cc
g++    -c -o mycode.o mycode.cc
g++  -Wall -g -o runme mycode.o mymain.o
$ ls
Makefile  mycode.cc  mycode.hh  mycode.o  mymain.cc  mymain.o  README  runme
This makefile divides into three 'rules', each indicated by a target file: all, runme, and clean. After 'target:' comes an optional list of dependencies or 'prerequisites'; if these files do not exist make will attempt to create them or find a rule to make each prerequisite. The indented lines below the rule headers constitute the 'recipe' used update the target indicated. Each of these lines will be issued in order in the shell. Keep in mind recipe lines have to be indented otherwise it is an error.

In the above example, when the 'make' command is issued with no arguments, it defaults to the 'all' rule. Here, we specified prerequisite for 'all' as another target, 'runme'. Make will check to see if a file with that name already exists and whether it is up to date based on its dependencies. (If no file exists the recipe will always be run).

Looking at the rule for runme we have two prerequisites 'mymain.o' and 'mycode.o'. We did not specify rules for mymain.o, but make knows that a .o file requires the compilation of a source file (.c, .cpp, .cc etc.), and so looks and finds mymain.cc.

As a result of running make the first time, you can see that each object file was created with g++. However since we didn't define a rule for .o files, we don't get the same options we are using for the executable recipe (-Wall and -g).

When the prerequisite object files for runme are up to date, make proceeds to the recipe line immediately following which creates the file 'runme' as output with a g++ command.

See how make behaves when changes are made to the source files:
$ make
make: Nothing to be done for `all'.
$ touch mymain.o
$ make
g++  -Wall -g -o runme mycode.o mymain.o
$ sudo nano mycode.cc
$ make
g++    -c -o mycode.o mycode.cc
g++  -Wall -g -o runme mycode.o mymain.o
First is the result you see if everything is up to date. Next we updated the timestamp mymain.o manually. We didn't actually change any source code yet, but make sees this file is newer than runme and recompiles/links it. After that we update some source code, and when make runs it sees this and recompiles mycode.o (and leaves the mymain.o alone since it is up to date) and then recompiles the executable 'runme'.

Since the rule 'all' only has the one prerequisite, it is the same as explicitly using the command 'make runme'. Because make checks for a file with a name matching the target, ie 'runme', it is important to output the file runme in the recipe to avoid recompiling: if the file isn't present, the recipe will always be run.

In light of the above, the third rule 'clean' is self explanatory. It is a 'phony' target that really is just a quick way to execute the recipe it specifies, without the need of any dependencies. The rm option '-f' ('force') is used to ignore the error generated when no file can be found to remove.
$ make clean
rm -f *.o
rm -f runme
Note if there happens to be a file named 'clean' in the directory, nothing will be done. This is probably not going to be an issue but you can add the line '.PHONY: clean' somewhere above the rule definition in the makefile to make it always work.

make Variables

Using the above makefile, if we wanted to make changes to the executable name or the dependencies, we would have to make changes in more than one place for each change. To avoid this we can define variables and use those variables throughout the file:
$ cat Makefile
# define the variables

CC = g++
CFLAGS = -Wall -g
EXEC = runme
OBJECTS = mymain.o mycode.o

# define the rules

all: $(EXEC)

$(EXEC): $(OBJECTS)
    $(CC) $(CFLAGS) -o $@ $^

clean:
    rm -f *.o
    rm -f $(EXEC)
The syntax '$(variable)' dereferences the variable to the value previously assigned. In the recipe line for our executable there are two automatic variables introduced: '$@' refers to the target of the rule, and '$^' to all the prerequisites. Using automatic variables is optional; in this case it works the same as if the line was:
$(CC) $(CFLAGS) -o $(EXEC) $(OBJECTS)
Note that it is conventional to match the flags variable to the utility variable, but it is a standard exception to see 'CFLAGS' instead of 'CCFLAGS'.

Pattern Rules

At this point we still haven't defined a rule for compiling object files from their sources; it's being handled automatically for us. To make our own we will use a 'pattern rule':
$ cat Makefile
# define the variables

CC = g++
CFLAGS = -Wall -g
CPPFLAGS =
LDFLAGS =
EXEC = runme
OBJECTS = mymain.o mycode.o

# define the rules

all: $(EXEC)

$(EXEC): $(OBJECTS)
    $(CC) $(LDFLAGS) $(CFLAGS) -o $@ $^

%.o: %.cc
    $(CC) -c $(CPPFLAGS) $(CFLAGS) -o $@ $<

clean:
    rm -f *.o
    rm -f $(EXEC)
In a pattern rule, the (non-empty) part of the target filename corresponding to the stem character, '%', is substituted everywhere '%' appears in the prerequisites. This would fail if you were to specify a %.o file when a matching %.cc file did not already exist since make has no implicit rule for making such a file. Notice that a new automatic variable is used here: $< will only resolve to the first prerequisite if there is more than one provided.

You may encounter the syntax
.cc.o:
    $(CC) -c $(CPPFLAGS) $(CFLAGS) -o $@ $<
which has the same meaning but is considered obsolete.

Don't be thrown off by the extra variables CPPFLAGS, and LDFLAGS added to the makefile. When they are assigned to nothing, they will not affect anything, but they are there by convention to provide an easy way to specify Preprocessor and Linker options. You may have noticed that the '-c' option wasn't included in the CFLAGS variable. The make manual specifies that if an option is required for a recipe to work, it should appear explicitly in the recipe, rather than allowing the user to accidentally break things by changing the flags variable.

Substitution Reference, Pattern Substitution, and Wildcards

Wildcards are already used above in the clean target; '*.o' specifies all the files in the directory with extension '.o'. Be aware that if you try assigning a wildcard like this to a variable, the variable literally becomes that wildcard (*.o), not the expanded list of files. It will be expanded in some cases, but usually this will cause unforseen problems. Instead, use the wildcard function to assign variables:
$cat Makefile
# define the variables

CC = g++
CPPFLAGS =
CFLAGS = -Wall -g
LDFLAGS =
EXEC = runme
HEADERS = $(wildcard *.hh)
SOURCES = $(wildcard *.cc)
OBJECTS = $(SOURCES:.cc=.o)

# define the rules

all: $(EXEC)

$(EXEC): $(OBJECTS)
    $(CC) $(LDFLAGS) $(CFLAGS) -o $@ $^

%.o: %.cc $(HEADERS)
    $(CC) -c $(CPPFLAGS) $(CFLAGS) -o $@ $<

clean:
    rm -f *.o
    rm -f $(EXEC)
This makefile will now automatically assign OBJECTS as a list of .o files based on the existing .cc source files in the directory. To change all the .cc extensions in SOURCES to .o extensions, we are using a syntax called 'substitution reference': $(variable:suffixa=suffixb). This form requires that these replacements are at the ends of the filenames. Alternatively, you can use patterns by including a '%' stem: 
OBJECTS = $(SOURCES:%.cc=%.o)
The result is the same as above. An alternative way of defining OBJECTS: 
OBJECTS = $(patsubst %.cc, %.o, $(wildcard *.cc))
The function patsubst replaces the pattern of the first argument with the pattern in the second argument where it occurs in the text provided as a third argument: $(patsubst pattern,replacement,text). Note that a change to any header will result in a recompilation of all source files. The solution is to create explicit rules for source files and specify only the headers they include as prerequisites.


Debug and Release Configurations

 




If you are working on a piece of code you may want to build with debug by default, but have a rule to compile a release version as well:





$ cat Makefile
# define the variables

CC = g++
CFLAGS = -Wall
CPPFLAGS =
LDFLAGS =
DOUT = runme_debug
ROUT = runme
HEADERS = $(wildcard *.hh )
SOURCES = $(wildcard *.cc )
DOBJECTS = $(SOURCES:.cc=.do)
ROBJECTS = $(SOURCES:.cc=.o)

# define the rules

all: $(DOUT)

release: $(ROUT) 

$(DOUT): CFLAGS += -g
$(DOUT): CPPFLAGS += -DDEBUG=1
$(DOUT): $(DOBJECTS)
    $(CC) $(LDFLAGS) $(CFLAGS) -o $@ $^

$(ROUT): CPPFLAGS += -DNDEBUG=1
$(ROUT): $(ROBJECTS)
    $(CC) $(LDFLAGS) $(CFLAGS) -o $@ $^

%.o: %.cc $(HEADERS)
    $(CC) -c $(CPPFLAGS) $(CFLAGS) -o $@ $<

%.do: %.cc $(HEADERS)
    $(CC) -c $(CPPFLAGS) $(CFLAGS) -o $@ $<

debug: $(DOUT)
    ./$(DOUT)
run: release
    ./$(ROUT)
clean:
    rm -f *.o
    rm -f *.do
    rm -f $(DOUT)
    rm -f $(ROUT)




By default 'all' will build a debug version. In order to allow the possibility of debug and release versions in the same directory we specify the variable DOBJECTS as a list of filenames ending with .do to differentiate them from release object files. We also added a rule and a line to clean to handle these files. The '+=' is an append operator for variables. Here it is used in target-specific variable values. When we compile the debug output (DOUT), the debug option -g is appended to CFLAGS and a command-line definition DEBUG for the Preprocessor. Similarly for the release output (ROUT) we define NDEBUG (no debug) (a variable that is looked for by system headers such as ).





$ cat mymain.cc
#include 
#include 
#include "mycode.hh"

int main(){
 using namespace std;
 assert(0);
 cout << "Hello World\n";
 cout << "mycode foo(): " << foo() << endl;
 return 0;
}
$ make clean
rm -f *.o
rm -f *.do
rm -f runme_debug
rm -f runme
$ make
g++ -c -DDEBUG=1 -Wall -g -o mycode.do mycode.cc
g++ -c -DDEBUG=1 -Wall -g -o mymain.do mymain.cc
g++  -Wall -g -o runme_debug mycode.do mymain.do
$ make debug
./runme_debug
runme_debug: mymain.cc:7: int main(): Assertion `0' failed.
make: *** [debug] Aborted
$ make release
g++ -c -DNDEBUG=1 -Wall -o mycode.o mycode.cc
g++ -c -DNDEBUG=1 -Wall -o mymain.o mymain.cc
g++  -Wall -o runme mycode.o mymain.o
$ make run
./runme
Hello World
mycode foo(): -11
$ ls
Makefile  mycode.cc  mycode.do  mycode.hh  mycode.o  mymain.cc  mymain.do  mymain.o  README  runme  runme_debug




Because of the failed assertion on line 7 of mymain.cc, the debug version fails but the release version does not trigger the assertion (because NDEBUG was defined on the command-line which instructs the assert header to 'do nothing').



Directories & Order-only Prerequisites



Working from a single directory may not be feasible for a more complex build. Here we add a directory for source files (src/), and instruct the makefile to create separate directories for builds (build/debug/ and build/release/):







$ cat Makefile
# define the variables

SHELL = /bin/sh

CC = g++
CFLAGS = -Wall
CPPFLAGS =
LDFLAGS =

SRC_DIR = src/
BUILD_DIR = build/
DEBUG_DIR = $(BUILD_DIR)debug/
RELEASE_DIR = $(BUILD_DIR)release/
DOUT = $(DEBUG_DIR)runme_debug
ROUT = $(RELEASE_DIR)runme
HEADERS = $(wildcard $(SRC_DIR)*.hh )
SOURCES = $(wildcard $(SRC_DIR)*.cc )
DOBJECTS = $(SOURCES:$(SRC_DIR)%.cc=$(DEBUG_DIR)%.o)
OBJECTS = $(SOURCES:$(SRC_DIR)%.cc=$(RELEASE_DIR)%.o)

# define the rules

all: $(DOUT)

release: $(ROUT)

$(DEBUG_DIR):
 mkdir -p $@
$(RELEASE_DIR):
 mkdir -p $@

$(DOUT): CFLAGS += -g
$(DOUT): CPPFLAGS += -DDEBUG=1
$(DOUT): $(DOBJECTS)
 $(CC) $(LDFLAGS) $(CFLAGS) -o $@ $^

$(ROUT): CPPFLAGS += -DNDEBUG=1
$(ROUT): $(OBJECTS)
 $(CC) $(LDFLAGS) $(CFLAGS) -o $@ $^

$(DEBUG_DIR)%.o: $(SRC_DIR)%.cc $(HEADERS) | $(DEBUG_DIR)
 $(CC) -c $(CPPFLAGS) $(CFLAGS) -o $@ $<

$(RELEASE_DIR)%.o: $(SRC_DIR)%.cc $(HEADERS) | $(RELEASE_DIR)
 $(CC) -c $(CPPFLAGS) $(CFLAGS) -o $@ $<

debug: $(DOUT)
 ./$(DOUT)
run: release
 ./$(ROUT)
test: all
 valgrind $(DOUT)
clean:
 rm -rf $(BUILD_DIR)
spearman@darkstar ~/Documents/myproject $ ls
Makefile  README  Sconstruct  src
spearman@darkstar ~/Documents/myproject $ make
mkdir -p build/debug
g++ -c -DDEBUG=1 -Wall -g -o build/debug/mycode.o src/mycode.cc
g++ -c -DDEBUG=1 -Wall -g -o build/debug/mymain.o src/mymain.cc
g++  -Wall -g -o build/debug/runme_debug build/debug/mycode.o build/debug/mymain.o
spearman@darkstar ~/Documents/myproject $ ls
Makefile  README  build  src




Four new variables are used throughout the makefile: SRC_DIR, BUILD_DIR, DEBUG_DIR, and RELEASE_DIR. 


First the proper directories are assigned to the debug and release targets (build/debug/runme_debug and build/release/runme). Next, in the wildcard function for HEADERS and SOURCES we specify the source directory. When creating the DOBJECTS and OBJECTS variables, new directory variables are used to specify source and object directories in the substitution reference formula. This gives us our lists of objects; one list for each directory. Note it is no longer necessary to use the non-standard '.do' extension to differentiate from release object files since they will be built in separate directories.


No changes are required to be made to the DOUT and ROUT rules. For the object file rules, we now have to include the directories in the pattern substitution formulae for object files and source files to prevent errors. One last addition to the object file rules is the addition of order-only prerequisites. These rules for directory creation (added above in the makefile) will be executed if the directories don't exist, but using the syntax '|' to specify them as 'order-only' prerequisites means that if the directories time-stamp is updated it will not trigger a re-build of the targets like a normal prerequisite would (in this case the targets are the object files).


The only other change has been to change the clean recipe to remove the build directory recursively.



Further Changes



At this point the makefile should work fairly well for small or medium sized C++ projects. Larger projects will benefit from defining headers explicitly for each source file to avoid re-building the entire project every time you edit a header file. In the next section an alternative utility called SCons will be introduced which can track these kinds of dependencies automatically.


Part 4: SCons





































No comments:















Post a Comment


















        

      









Subscribe to:
Post Comments (Atom)