User:Quaid/Textbook/Building the Code

From The_Open_Source_Way

< User:Quaid
Revision as of 19:24, 26 March 2010 by Quaid (Talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

By Greg DeKoenigsberg


From Source to Executable

Now you know how source control works, and you've got a gigantic pile of source code sitting in a directory.

What do you do with it?

The process of turning source code into executable binary code can be extremely complicated. The more source code you have, the more complicated that process is. Almost every serious piece of software has its own build process, that every developer must follow -- and woe be unto the developer who repeatedly breaks the build for everyone else.

In this chapter, you learn about how software is built. You learn about how the build process works in general, about some tools that you are likely to see, and you walk through a build for a FOSS project to see how it works in practice.

What is Building, Exactly?

There are many steps in the process of turning source code into a binary executable. Some examples of tasks that you might encounter during a typical build process:

  • Compiling the code. Source code must somehow become machine code, ultimately. Sometimes this is handled in real-time by an interpreter, as in the case of scripting languages such as Perl or Javascript. For more complex applications, though, this work is usually handled by a compiler. Therefore, you must ensure that you have a proper compiler installed, and that you are calling the compiler properly with the correct compiler options.
  • Linking object files and libraries. In the modern world, it's crazy to write all of the code yourself. When you want to write output to the screen, you don't write code that talks directly to the monitor; you use a library that handles input and output. When you want to play audio, you don't handcode the waveforms yourself; you use audio codecs. When you compile the code, you almost always need to include libraries of one kind or another -- which means you must know which libraries you need, and you must ensure that the libraries are where the compiler expects them to be, and that the libraries are all of the right version.
  • Determining build order and dependencies. In complex software projects, it's vital to keep track of dependencies. A change to code in a single library can have effects across your entire project, and might require some or all of your code to be recompiled -- and often in a particular order. Keeping track of dozens of libraries, and references to those libraries by hundreds of source files, can be an ugly business.
  • Testing the build results. It's essential to know when you've introduced bugs sooner rather than later; new bugs are often easy to fix, and old bugs are often not so easy to fix. Also, it frequently happens that bugs, once fixed, creep back into code. Running basic tests on a project every time it's built can be a good way to ensure that bugs get fixed and stay fixed.
  • Packaging and/or Deploying. Sometimes you want to install the program you just compiled so that it can be run from anyhere on the system, and other programs or users can find it. Or sometimes you want to bundle it up into a format that allows anyone to take your executable and install it easily for themselves. You don't want to do this for every build, but when you know that your build is good, one of the important final steps is to put the executable, and all documentation, in a central location.

Performing all of these tasks by hand would be time-consuming and difficult. Build automation tools allow the developer to handle all of these tasks automatically -- and thus, to manage projects with a much higher degree of complexity.

Living With Complexity

Fair warning: sometimes code doesn't compile. Sometimes you follow all the instructions, and it still doesn't work. What then?

If this is your first experience dealing with a large codebase written by someone else, then welcome to the real world. As you run your first build, you may have very little idea of what's actually going on. Don't get discouraged. Have patience; you'll soon begin to figure it all out.

You are about to walk through a software build process. The typical build command might return hundreds, or even thousands, of log entries, all of which scroll across the screen at lightning speed. You may understand all, some, or none of those entries.

That's okay. Everyone starts somewhere.

Here are some points to keep in mind.

  • Read instructions carefully. Almost every sizable FOSS project has a README or an INSTALL file that provides instructions for how to build and install the software. Read those instructions, and do your best to follow them to the letter. Understand that even the best instructions may leave out a step or two -- and these are opportunities to improve the project.
  • Don't expect to understand every word. Very few developers understand every single word of every build log they encounter. Learning to distinguish between the important messages and the spurious messages takes time. Don't be intimidated.
  • Read logs carefully and thoughtfully. When you see an error, read back and think about what it could mean. Does the error say "couldn't find something"? That probably means you didn't install a library dependency properly. Was the error you found the only error in the log? In a 1000-line build log, the error at the end could be the result of another error dozens, or hundreds, of lines earlier. If your build doesn't end happily, don't panic. Relax and work your way through the problem.
  • Google is your friend. If you don't understand an error message, just Google it! Googling an error message can be a surprisingly effective method for determining what's gone wrong. There's a decent chance that someone before you has run into the same error, and someone has posted the solution to your problem on a message board or mailing list. Even if the answer isn't obvious, there will frequently be clues. The Internet is a gigantic resource. Use it.
  • Ask for help. If you've done your homework and still can't figure out why your program isn't building, get on the project's mailing list, message board, or IRC channel, and ask for help. The more you've dug into the problem, and the more information you provide, the more likely it is that developers will help you figure out the problem.

Now it's time to get on with it.

Building Freeciv: Watching GNU Autotools at Work

If you are working on a project that is written in C or C++, and that project is designed to run on Linux or UNIX, then it's incredibly likely that you will be seeing GNU Autotools at work.

Developers use GNU Autotools to make sure that their users (that's you) can compile and run a piece of software across a wide variety of hardware and software configurations.

You are now going to walk through the building of Freeciv. You checked out a local working repository of Freeciv in the last chapter, right? Now it's time to turn all that code into a playable binary executable of the latest and awesomest version of Freeciv.

NOTE: Follow along with the sample build process, below. As you proceed through the build process, you may see many of the same errors; you may see completely different errors. No matter what happens, keep calm and carry on. Read the instructions. Don't expect to understand every word. Read logs carefully and thoughtfully. Google is your friend. Ask for help.

Finding the Installation Instructions

Look for an INSTALL file in the top-level directory of the local repository. If you don't find an INSTALL file, look for a README file. If you don't find a README file, look for a script called configure and run it. And if you don't find that, send a nice email to the maintainers of the project, and ask them if they could use some help with their installation instructions.

Oh, look, there's an INSTALL file right there in the top level directory of the freeciv folder.

NOTE: the version of the INSTALL file referred to here is dated 22-Oct-2009. If it's way out of date, then you'll just have to buy the next edition of the textbook. Just kidding! Send in a patch, and we'll fix it.

Installing Prerequisites

Every good INSTALL file tells you what the prerequisites are. If you find an INSTALL file that doesn't, offer to fix it.

Freeciv does, though. Right there in the table of contents:

     0. Prerequisites:
     1. Prerequisites for the clients:
          1a. Prerequisites for the Gtk+ client:
          1b. Prerequisites for the SDL client:
          1c. Prerequisites for the Xaw client:

There are two sets of requirements. One set of requirements is listed for general building with the following list:

  • Unix (or similar).
  • An ANSI C compiler
  • A "make" program
  • The programs from GNU gettext version 0.10.36 or better
  • GNU autoconf, version 2.58 or better
  • GNU automake, version 1.6 or better

Then another set of requirements is required for building the Freeciv clients, which can actually be compiled in different flavors, and each flavor has a different set of dependencies. You are just building the Gtk+ client, which means:

  • pkg-config
  • "Glib" greater or equal to 2.4.0
  • The "Atk" accessibility library
  • The "Pango" text layout and rendering library
  • The "Gtk+" widget library greater or equal to 2.4.0

That's a lot of stuff. How do you get all that stuff?

This is where Linux distributions shine. All modern Linux distributions have package management programs that allow for easy location and installation of FOSS software. In this example case, presume the Fedora distribution, but Ubuntu uses similar commands and works in largely the same way.

First, make sure that you have a C compiler. In fact, use the compiler that Freeciv recommends in the INSTALL file: gcc.

[gregdek@ip-10-242-118-147 freeciv]$ rpm -q gcc
package gcc is not installed

Hm! You have your first problem.

RPM is a program that maintains a list of software packages installed on the system. With this command, you asked, "tell me if you have gcc installed". And RPM said "nope, sorry." Which means you turn to RPM's big brother, yum. (In the Ubuntu/Debian world, these commands would be dpkg and apt-get, respectively.)

Next you ask yum to install gcc:

[gregdek@ip-10-242-118-147 freeciv]$ yum install gcc
Loaded plugins: fastestmirror
You need to be root to perform this command.

Oh, right. If you're going to work with Linux, you need to know when you need to be regular user, and when you need to be root. When you're adding new software to the system, to be accessed by other programs and potentially other users, you need to be root. (You can also use the su -c command, which allows you to masquerade as root.)

[root@ip-10-242-118-147 ~]# yum install gcc
Loaded plugins: fastestmirror
Determining fastest mirrors
 * updates-newkey:
 * fedora:
 * updates:
updates-newkey                                           | 2.3 kB     00:00     
fedora                                                   | 2.1 kB     00:00     
updates                                                  | 2.6 kB     00:00     
Setting up Install Process
Parsing package install arguments
Resolving Dependencies
--> Running transaction check
---> Package gcc.i386 0:4.1.2-33 set to be updated
--> Processing Dependency: glibc-devel >= 2.2.90-12 for package: gcc
--> Running transaction check
---> Package glibc-devel.i386 0:2.7-2 set to be updated
--> Finished Dependency Resolution

Dependencies Resolved

 Package               Arch           Version            Repository        Size
 gcc                   i386           4.1.2-33           fedora           5.2 M
Installing for dependencies:
 glibc-devel           i386           2.7-2              fedora           2.0 M

Transaction Summary
Install      2 Package(s)         
Update       0 Package(s)         
Remove       0 Package(s)         

Total download size: 7.2 M
Is this ok [y/N]:

Lots of interesting information here! And you already start to see how software fits together. The programs yum and rpm work together to make sure that when you choose to install gcc, you also get everything that gcc needs to be useful -- in this case, the header and object files necessary for developing programs that use the standard C libraries. You asked for one software package, but now you get two.

Answer yes:

Is this ok [y/N]: y
Downloading Packages:
(1/2): glibc-devel-2.7-2.i386.rpm                        | 2.0 MB     00:00     
(2/2): gcc-4.1.2-33.i386.rpm                             | 5.2 MB     00:00     
Total                                           6.4 MB/s | 7.2 MB     00:01     
============================== Entering rpm code ===============================
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing     : glibc-devel                                              1/2 
  Installing     : gcc                                                      2/2 
=============================== Leaving rpm code ===============================

  gcc.i386 0:4.1.2-33                                                           

Dependency Installed:
  glibc-devel.i386 0:2.7-2                                                      


All right, that's one dependency down. Which also means that you can build anything that needs GCC in the future, so that's useful.

Now you need a make program. They recommend gmake, so see if it's installed.

[root@ip-10-242-118-147 ~]# rpm -q gmake
package gmake is not installed

All right, install it.

[root@ip-10-242-118-147 ~]# yum install gmake
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * updates-newkey:
 * fedora:
 * updates:
Setting up Install Process
Parsing package install arguments
No package gmake available.
Nothing to do

Wait, what's this? It appears that gmake isn't installed, and isn't available? What's going on?

Well, upon reading the INSTALL instructions more closely, there is this nugget:

   You can check if you have GNU make installed on your system by

    % make -v                   [and if this doesn't work, try "gmake -v"]

   The output should include "GNU Make" somewhere.

Get in the habit of reading instructions.

[root@ip-10-242-118-147 ~]# make -v
GNU Make 3.81
Copyright (C) 2006  Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A

This program built for i386-redhat-linux-gnu

All right, you've got make. How about GNU gettext?

[root@ip-10-242-118-147 ~]# rpm -q gettext

Very good. Autoconf?

[root@ip-10-242-118-147 ~]# rpm -q autoconf
package autoconf is not installed
[root@ip-10-242-118-147 ~]# yum install -y autoconf

(snip lots of yum output)

  autoconf.noarch 0:2.61-9.fc8                                                  

Dependency Installed:
  imake.i386 0:1.0.2-5.fc8


All set. Automake?

[root@ip-10-242-118-147 ~]# yum install -y automake

(snip lots of yum output)

  automake.noarch 0:1.10-6                                                      


Note that this time you didn't even bother to see if the RPM was installed first; you just installed it, because if automake had already been installed, yum would have let us know:

[root@ip-10-242-118-147 ~]# yum install -y automake
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * updates-newkey:
 * fedora:
 * updates:
Setting up Install Process
Parsing package install arguments
Package automake-1.10-6.noarch already installed and latest version
Nothing to do

That's half of the prerequisites. Time for the other half.

[root@ip-10-242-118-147 ~]# yum install pkg-config
No package pkg-config available.
Nothing to do

[root@ip-10-242-118-147 ~]# yum install pkgconfig
Package 1:pkgconfig-0.22-4.fc8.i386 already installed and latest version
Nothing to do

Okay, so you've got pkg-config installed, even though they seem to be calling it pkgconfig for some reason, which was discovered through a lucky guess -- noting that in the INSTALL file, even though they call it pkg-config, the file in question is called pkgconfig-0.14.0.tar.gz. These kinds of little inconsistencies are maddening, and you will find them everywhere in your software life, so get used to them.

Now for Glib:

[root@ip-10-242-118-147 ~]# yum install Glib
No package Glib available.
  * Maybe you meant: glib
Nothing to do

Oh, maybe you did mean glib. Thanks!

[root@ip-10-242-118-147 ~]# yum install glib
  glib.i386 1:1.2.10-28.fc8  


This is kind of slow, isn't it? Can you specify multiple packages to be installed at once? Indeed you can.

[root@ip-10-242-118-147 ~]# yum install atk pango gtk+
Package atk-1.20.0-1.fc8.i386 already installed and latest version
  gtk+.i386 1:1.2.10-59.fc8              pango.i386 0:1.18.4-1.fc8             

Dependency Installed:
  cairo.i386 0:1.4.14-1.fc8                   libX11.i386 0:1.1.3-4.fc8         
  libXext.i386 0:1.0.1-4.fc8                  libXft.i386 0:2.1.12-3.fc8        
  libXi.i386 0:1.1.3-1.fc8                    libXrender.i386 0:0.9.4-1.fc8     
  xorg-x11-filesystem.noarch 0:7.1-2.fc6     


Whew. At long last, you're done.

Or so it would appear -- but appearances can be deceiving.


Once you have all of the prerequisites installed, the next step is to run the configure script.

The configure script, in this case, is generated by the GNU Autotools, a set of tools that examine lots and lots (and lots and lots) of variables about your system. It checks your compiler, it checks your hardware, it checks all kinds of stuff, and as a result of all of these checks (literally hundreds), it builds a makefile that the compiler uses to build the binary executable.

Simple, right? Give it a try. But first, read the instructions:

2. Generating Makefiles
This section contains two parts, one for generating makefiles from svn
versions and one for generating makefiles from release versions.

2a. Generating the Makefile for svn versions:

This step is only needed for svn versions.

To create the makefile just type

 % ./

This will create the configure script and will run it. All parameters
of are passed to configure. Read the next section about the
parameters which can be passed to configure.

All right, seems simple enough. Run it and see what happens:

[gregdek@ip-10-242-118-147 freeciv]$ ./ 
+ checking for autoconf >= 2.58 ... found 2.61, ok.
+ checking for autoheader >= 2.58 ... found 2.61, ok.
+ checking for automake >= 1.6 ... found 1.10, ok.
+ checking for aclocal >= 1.6 ... found 1.10, ok.
+ checking for libtoolize >= 1.4.3 ... 
You must have libtoolize installed to compile freeciv.
Download the appropriate package for your distribution,
or get the source tarball at
+ checking for xgettext >= 0.10.36 ... found 0.16.1, ok.
+ checking for msgfmt >= 0.10.36 ... found 0.16.1, ok.

Oops. Looks like the script found a missing dependency that's not documented! Fortunately, GNU Autotools found it. Install libtoolize -- or libtool, which is it? Anyway, it's probably one of them:

[root@ip-10-242-118-147 FREECIV]# yum install libtoolize
No package libtoolize available.
Nothing to do
[root@ip-10-242-118-147 FREECIV]# yum install libtool
  libtool.i386 0:1.5.24-3.fc8                                                   


All right, try that again.

[gregdek@ip-10-242-118-147 freeciv]$ ./ 
+ checking for autoconf >= 2.58 ... found 2.61, ok.
+ checking for autoheader >= 2.58 ... found 2.61, ok.
+ checking for automake >= 1.6 ... found 1.10, ok.
+ checking for aclocal >= 1.6 ... found 1.10, ok.
+ checking for libtoolize >= 1.4.3 ... found 1.5.24, ok.
+ checking for xgettext >= 0.10.36 ... found 0.16.1, ok.
+ checking for msgfmt >= 0.10.36 ... found 0.16.1, ok.
+ running aclocal ...
+ running autoheader ... 
+ running autoconf ... 
+ running libtoolize ... 
Putting files in AC_CONFIG_AUX_DIR, `bootstrap'.
+ running automake ... installing `bootstrap/missing' installing `bootstrap/install-sh'
ai/ installing `bootstrap/depcomp'
common/ `%'-style pattern rules are a GNU make extension
utility/ `%'-style pattern rules are a GNU make extension
+ removing config.cache ... 
+ running configure ... 

I am going to run ./configure with no arguments - if you wish 
to pass any to it, please specify them on the ./ command line.

OK, so far so good! It's successfully created a configure script, and now it's running that script. Fingers crossed...

checking build system type... i686-pc-linux-gnu
checking host system type... i686-pc-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking for style of include used by make... GNU
checking for gcc... gcc
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables... 
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking dependency style of gcc... gcc3
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for gethostbyname2... yes
checking for inet_pton... yes
checking for inet_ntop... yes
checking for getnameinfo... yes
checking for AF_INET6... yes
checking for a sed that does not truncate output... /bin/sed
checking for gawk... (cached) gawk
checking for gcc... (cached) gcc
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ISO C89... (cached) none needed
checking dependency style of gcc... (cached) gcc3
checking how to run the C preprocessor... gcc -E
checking for g++... no
checking for c++... no
checking for gpp... no
checking for aCC... no
checking for CC... no
checking for cxx... no
checking for cc++... no
checking for cl.exe... no
checking for FCC... no
checking for KCC... no
checking for RCC... no
checking for xlC_r... no
checking for xlC... no
checking whether we are using the GNU C++ compiler... no
checking whether g++ accepts -g... no
checking dependency style of g++... none
checking whether the C++ compiler works... no
configure: error: no acceptable C++ compiler found in $PATH
See `config.log' for more details.

Wait... what? But... but... it's written in C! Why do you need a C++ compiler for C?

Annoyed, Google "freeciv c++", looking for clues. (Remember: Google is your friend.) And on the first page of search results, there is this gem of wisdom:

"Freeciv is a free turn-based multiplayer strategy game, in which each player becomes the leader ... Mostly written in C."

Mostly written in C. Following a few links, discover that at least part of it is written in C++. Sigh. So, install the GNU C++ compiler -- Googling "gnu c++ fedora" tells us that the package name is likely called gcc-c++.

[root@ip-10-242-118-147 ~]# yum install gcc-c++
  gcc-c++.i386 0:4.1.2-33                                                       

Dependency Installed:
  libstdc++-devel.i386 0:4.1.2-33                                               


Try all that again.

[gregdek@ip-10-242-118-147 freeciv]$ ./
+ checking for autoconf >= 2.58 ... found 2.61, ok.
+ checking for autoheader >= 2.58 ... found 2.61, ok.
+ checking for automake >= 1.6 ... found 1.10, ok.
(snip to where we broke the last time...)
checking for g++... g++
checking whether we are using the GNU C++ compiler... yes
(snip to where we fail again...)
checking for gzgets in -lz... no
configure: error: Could not find zlib library.

configure failed

Another undocumented dependency.

So take stock at this point of where you are.

Freeciv is a good program. It runs on many platforms. There are dozens of active developers working on pieces of the codebase at any given time. And yet, in the space of 15 minutes, you have already found a handful of places where the documentation could be improved -- and you haven't even made it to a first successful build!

This is what complex software is like. Getting it 100% right is incredibly hard, and there are always little places to improve.

Anyway, back to business. Missing zlib? Install zlib.

[root@ip-10-242-118-147 ~]# yum install zlib
Package zlib-1.2.3-14.fc8.i386 already installed and latest version
Nothing to do

That's odd. But Google is your friend, and Googling "freeciv zlib" leads to this exact problem -- and it's a common problem. You need development libraries for zlib.

Many FOSS projects split their libraries in two. They provide runtime libraries, and they provide development libraries. In fact, if you read the INSTALL document all the way down to section 10:

To compile freeciv on a debian system you need the following packages:

 Common requirements:

Well, this isn't a Debian system, it's a Fedora system -- but the principle is the same. You need development libraries. Googling "fedora zlib" returns, in the top link, a reference to zlib-devel. So try that:

[root@ip-10-242-118-147 ~]# yum install zlib-devel
  zlib-devel.i386 0:1.2.3-14.fc8                                                


Try, try again.

[gregdek@ip-10-242-118-147 freeciv]$ ./ 
(snip to the next error...)
configure: error: could not guess which client to compile

configure failed

It clearly says in the INSTALL file (you did read the INSTALL file very closely, didn't you?) that "Gtk+" is the default client. So what guess is required?

Going back a little farther up in the log, though, you see the problem:

configure: checking for which client to compile:...
checking for pkg-config... /usr/bin/pkg-config
checking for GTK+ - version >= 2.4.0... no
*** Could not run GTK+ test program, checking why...
*** The test program failed to compile or link. See the file config.log for the
*** exact error that occured. This usually means GTK+ is incorrectly installed.
checking for sdl-config... no
checking for SDL - version >= 1.1.4... no
*** The sdl-config script installed by SDL could not be found
*** If SDL was installed in PREFIX, make sure PREFIX/bin is in
*** your path, or set the SDL_CONFIG environment variable to the
*** full path to sdl-config.
checking for X... no
checking whether Xfuncproto was supplied... no, found:  FUNCPROTO=15 NARROWPROTO
checking for Xfuncproto control definition FUNCPROTO... yes: 15
checking for Xfuncproto control definition NARROWPROTO... yes: 1
checking pkg-config is at least version 0.9.0... yes
checking for PNG... no
checking for png_read_image in -lpng12... no
checking for png_read_image in -lpng... no
checking png.h usability... no
checking png.h presence... no
checking for png.h... no
checking extra paths for Xpm... library no, include no
checking for XOpenDisplay in X library -lX11... no
checking will compile gui-ftwl... no
checking will compile gui-beos... no
configure: error: could not guess which client to compile

configure failed

It seems to want a GTK+ version of greater than 2.4.0. Isn't that what you installed?

[gregdek@ip-10-242-118-147 freeciv]$ rpm -q gtk+

Hm! Stuck? Once again, Google is your friend. Google for "Fedora gtk+" and the very first link is to a discussion on a mailing list, where someone asks, "is there a gtk+2?" And a helpful Fedora community member says, "yes, but it's called gtk2".

So try that.

[root@ip-10-242-118-147 ~]# yum install gtk2
  gtk2.i386 0:2.12.8-2.fc8                                                      

Dependency Installed:
  cups-libs.i386 1:1.3.9-2.fc8          libXcomposite.i386 0:0.4.0-3.fc8       
  libXcursor.i386 0:1.1.9-1.fc8         libXfixes.i386 0:4.0.3-2.fc8           
  libXinerama.i386 0:1.0.2-3.fc8        libXrandr.i386 0:1.2.2-1.fc8           


OK, that's a lot of extra stuff, but if gtk2 needs it, then gtk2 needs it.

Try, try again.

checking for GTK+ - version >= 2.4.0... no
*** Could not run GTK+ test program, checking why...
*** The test program failed to compile or link. See the file config.log for the
*** exact error that occured. This usually means GTK+ is incorrectly installed.
configure: error: could not guess which client to compile

configure failed

Same exact error. Maybe there's a gtk2-devel?

[root@ip-10-242-118-147 ~]# yum install gtk2-devel
  gtk2-devel.i386 0:2.12.8-2.fc8                                                

Dependency Installed:
  atk-devel.i386 0:1.20.0-1.fc8                                                 
  cairo-devel.i386 0:1.4.14-1.fc8                                               
  docbook-dtds.noarch 0:1.0-33.fc8                                              
  docbook-style-dsssl.noarch 0:1.79-4.1                                         
  docbook-style-xsl.noarch 0:1.73.2-5.fc8                                       
  docbook-utils.noarch 0:0.6.14-11.fc8                                          
  fontconfig-devel.i386 0:2.4.2-5.fc8                                           
  freetype-devel.i386 0:2.3.5-5.fc8                                             
  gc.i386 0:7.0-6.fc8                                                           
  glib2-devel.i386 0:2.14.6-2.fc8                                               
  gtk-doc.noarch 0:1.9-4.fc8                                                    
  libX11-devel.i386 0:1.1.3-4.fc8                                               
  libXau-devel.i386 0:1.0.3-3.fc8                                               
  libXcursor-devel.i386 0:1.1.9-1.fc8                                           
  libXdamage.i386 0:1.1.1-3.fc8                                                 
  libXdmcp-devel.i386 0:1.0.2-4.fc8                                             
  libXext-devel.i386 0:1.0.1-4.fc8                                              
  libXfixes-devel.i386 0:4.0.3-2.fc8                                            
  libXft-devel.i386 0:2.1.12-3.fc8                                              
  libXi-devel.i386 0:1.1.3-1.fc8                                                
  libXinerama-devel.i386 0:1.0.2-3.fc8                                          
  libXrandr-devel.i386 0:1.2.2-1.fc8                                            
  libXrender-devel.i386 0:0.9.4-1.fc8                                           
  libXxf86vm.i386 0:1.0.1-4.fc8                                                 
  libpng-devel.i386 2:1.2.33-1.fc8                                              
  libxcb-devel.i386 0:1.1-1.1.fc8                                               
  mesa-libGL.i386 0:7.0.2-3.fc8                                                 
  mesa-libGL-devel.i386 0:7.0.2-3.fc8                                           
  openjade.i386 0:1.3.2-30.fc8                                                  
  opensp.i386 0:1.5.2-6.fc8                                                     
  pango-devel.i386 0:1.18.4-1.fc8                                               
  perl-SGMLSpm.noarch 0:1.03ii-16.2.1                                           
  rarian.i386 0:0.6.0-4.fc8                                                     
  rarian-compat.i386 0:0.6.0-4.fc8                                              
  sgml-common.noarch 0:0.6.3-21.fc8                                             
  w3m.i386 0:0.5.2-5.fc8                                                        
  xml-common.noarch 0:0.6.3-21.fc8                                              
  xorg-x11-proto-devel.noarch 0:7.3-3.fc8                                       


There is, indeed -- and it brings in a ton of dependencies! Including -devel versions of other dependencies you thought you'd satisfied, such as atk-devel and cairo-devel.

Imagine if you had to sort through all of these dependencies yourself, by hand. With all of the effort you've already gone through to identify and satisfy a relatively small number of dependencies, imagine the work that would be required. As imperfect as this process clearly is, it could be orders of magnitude worse.

Try, try again.

****************** Configuration Summary ******************

  Build freeciv client: yes
    Debugging support:  some

  Client frontends:
    Gtk-2.0: yes
    SDL:     no
    Xaw:     no
    Win32:   no
    FTWL:    no
    Stub:    no

  Build freeciv server:    yes
    Debugging support:     some
    Auth database support: no

Now type 'make' to compile freeciv.


It worked! It worked it worked it worked it worked!

You are, at long last, ready to make.


Look, once again, at the excellent instructions:

If all has gone well previous to this point, then compiling Freeciv
should be as easy as typing "make" (or preferably, "gmake").

If you have problems, read the file BUGS, and follow the advice 
carefully.  If the problem is with "gettext", please read the Native
Language Support section, below, for possible work-arounds.

After compilation, the important results are:

  - The "client/freeciv-<GUI>" and "server/freeciv-server" binaries.
  - The "data/" directory, which contains the graphics and scenarios.
  - The "po/" directory, which contains the localization files.
  - The "civ" and "ser" scripts.

It's perfectly feasible to play Freeciv in this directory, without
installing it.  If you do this, the "civ" and "ser" scripts may be
useful, although they are not as necessary as they used to be.

See the README file for more information.

Do as it says. In fact, try a new trick:

[gregdek@ip-10-242-118-147 freeciv]$ make 1>/tmp/freeciv-make.out 2>/tmp/freeciv-make.err &
[1] 1517
[gregdek@ip-10-242-118-147 freeciv]$

Compiling an entire software project can take quite a while, and generates a prodigious amount of data -- and watching a bazillion lines of code fly by is only fun for the first few seconds. So this old-school UNIX command lets you capture all that output to look at it later. The make command is the make command, of course. The 1>/tmp/freeciv-make.out option tells the job to put the standard output into /tmp/freeciv-make.out, and 2>/tmp/freeciv-make.err tells the job to put error messages into /tmp/freeciv-make.err, and & tells the job to run in the background, so that you can be free to do other things. When the job completes, it tells you so.

As a truly ridiculous amount of computation takes place, wander off for a cup of coffee, and take this time to engage in serious business. When you come back, hit enter, and see on the screen:

[1]+  Done                    make > /tmp/freeciv-make.out 2> /tmp/freeciv-make.err

Ah, very good. Now have a look at the log files. How long are they? Here's the clever wc (word count) command, with the -l parameter to show the number of lines instead of number of words:

[gregdek@ip-10-242-118-147 freeciv]$ wc -l /tmp/freeciv-make.out
1148 /tmp/freeciv-make.out
[gregdek@ip-10-242-118-147 freeciv]$ wc -l /tmp/freeciv-make.err
551 /tmp/freeciv-make.err
[gregdek@ip-10-242-118-147 freeciv]$

Whoa! 551 errors? Take a look at that.

[gregdek@ip-10-242-118-147 freeciv]$ less /tmp/freeciv-make.err 

The less command is a tool that allows you to scroll through a text file, search the file for text, and so on. The command man less gives you the manual page for the less command. For now, the page-up and page-down keys can take you through the file. Here are some sample lines from that error file:

packets_gen.c: In function 'receive_packet_spaceship_info_100':
packets_gen.c:23371: warning: dereferencing type-punned pointer will break strict-aliasing rules
packets_gen.c: In function 'send_packet_spaceship_info_100':
packets_gen.c:23560: warning: dereferencing type-punned pointer will break strict-aliasing rules
packets_gen.c: In function 'receive_packet_ruleset_unit_100':
packets_gen.c:23830: warning: dereferencing type-punned pointer will break strict-aliasing rules
packets_gen.c: In function 'send_packet_ruleset_unit_100':
packets_gen.c:24178: warning: dereferencing type-punned pointer will break strict-aliasing rules
packets_gen.c: In function 'receive_packet_ruleset_game_100':
packets_gen.c:24743: warning: dereferencing type-punned pointer will break strict-aliasing rules
packets_gen.c: In function 'send_packet_ruleset_game_100':
packets_gen.c:24824: warning: dereferencing type-punned pointer will break strict-aliasing rules

Hmm, lots and lots of warnings. In fact, paging up and down through the whole file quickly reveals that the error file is full of nothing but warnings. Not ideal, and maybe something that someone should fix (patches welcome), but generally, warnings don't prevent a program from compiling.

Next look at the regular output of the make. Actually, use the tail command, which just shows the end of the file (the last 10 lines by default):

[gregdek@ip-10-242-118-147 freeciv]$ tail /tmp/freeciv-make.out 
gcc -DHAVE_CONFIG_H -I. -I..  -I../server -I../utility -I../common -I../ai -I../common/aicore -I../server/generator -I../client -I../client/include -DLOCALEDIR="\"/usr/local/share/locale\"" -DDEFAULT_DATA_PATH="\".:data:~/.freeciv/2.3:/usr/local/share/freeciv\"" -DDEFAULT_SAVES_PATH="\"\"" -DDEFAULT_SCENARIO_PATH="\".:data/scenario:~/.freeciv/scenarios:/usr/local/share/freeciv/scenario\""  -Wall -Wpointer-arith -Wcast-align -Wmissing-prototypes -Wmissing-declarations -g -O2 -MT civmanual.o -MD -MP -MF .deps/civmanual.Tpo -c -o civmanual.o civmanual.c
mv -f .deps/civmanual.Tpo .deps/civmanual.Po
/bin/sh ../libtool --preserve-dup-deps --tag=CC   --mode=link gcc  -Wall -Wpointer-arith -Wcast-align -Wmissing-prototypes -Wmissing-declarations -g -O2   -o civmanual civmanual.o ../server/ ../client/helpdata.lo ../server/scripting/ ../dependencies/lua-5.1/src/liblua.a ../dependencies/toluaxx/src/lib/libtolua.a ../server/generator/ ../common/  -lm   -lz  
mkdir .libs
gcc -Wall -Wpointer-arith -Wcast-align -Wmissing-prototypes -Wmissing-declarations -g -O2 -o civmanual civmanual.o ../client/helpdata.o  ../server/.libs/libfreeciv-srv.a ../server/scripting/.libs/libscripting.a ../dependencies/lua-5.1/src/liblua.a ../dependencies/toluaxx/src/lib/libtolua.a ../server/generator/.libs/libgenerator.a ../common/.libs/libfreeciv.a -lm -lz  
make[2]: Leaving directory `/home/gregdek/FREECIV/freeciv/manual'
make[2]: Entering directory `/home/gregdek/FREECIV/freeciv'
make[2]: Nothing to be done for `all-am'.
make[2]: Leaving directory `/home/gregdek/FREECIV/freeciv'
make[1]: Leaving directory `/home/gregdek/FREECIV/freeciv'
[gregdek@ip-10-242-118-147 freeciv]$ 

That's what make looks like when it succeeds. Be sure that you successfully generated the important bits, as you recall from the INSTALL guide:

After compilation, the important results are:

  - The "client/freeciv-<GUI>" and "server/freeciv-server" binaries.
  - The "data/" directory, which contains the graphics and scenarios.
  - The "po/" directory, which contains the localization files.
  - The "civ" and "ser" scripts.

See if you find these by using the ls command.

[gregdek@ip-10-242-118-147 freeciv]$ ls client/freeciv-*
[gregdek@ip-10-242-118-147 freeciv]$ ls server/freeciv-server
[gregdek@ip-10-242-118-147 freeciv]$ ls data/
Freeciv          civclient.dsc        freeciv-server.icns  isotrident     freeciv-server.png   isotrident.tilespec
Makefile         civserver.dsc        freeciv.rc           misc     freeciv.rc-2.0       nation       graphics             scenario
amplio     gtk_menus.xml        stdsounds
amplio.tilespec  default              helpdata.txt         stdsounds.soundspec
buildings        default.serv         hex2t                themes
civ1             flags                hex2t.tilespec       trident
civ1.serv        fonts                icons                trident.tilespec
civ2             freeciv-client.icns  isophex              wonders
civ2.serv        freeciv-client.png   isophex.tilespec
[gregdek@ip-10-242-118-147 freeciv]$ ls po/
ChangeLog     he.po   nb.po      ro.po
Makefile        cs.po      es.po     hu.po   nl.po      ru.po  da.po      et.po     statistics.rb
POTFILES     et.po.sig  it.po   no.po     de.po     sv.po
POTFILES.skip     fa.po      ja.po   pl.po          el.po     tr.po
ar.po   fi.po      ko.po   pt.po          en_GB.po  uk.po
ca.po      fr.po      lt.po   pt_BR.po     eo.po     zh_CN.po
[gregdek@ip-10-242-118-147 freeciv]$ ls civ
[gregdek@ip-10-242-118-147 freeciv]$ ls ser
[gregdek@ip-10-242-118-147 freeciv]$ 

That certainly looks like everything was generated -- but the proof is in the pudding. Does the code run? Run the ser script, which starts the freeciv server, just to make sure:

[gregdek@ip-10-242-118-147 freeciv]$ ./ser
This is the server for Freeciv version 2.2.99-dev
You can learn a lot about Freeciv at
2: Loading rulesets
2: AI*1 has been added as Easy level AI-controlled player.
2: AI*2 has been added as Easy level AI-controlled player.
2: AI*3 has been added as Easy level AI-controlled player.
2: AI*4 has been added as Easy level AI-controlled player.
2: AI*5 has been added as Easy level AI-controlled player.
2: Now accepting new client connections.

For introductory help, type 'help'.
> quit
[gregdek@ip-10-242-118-147 freeciv]$ 

It works! Of course, the only real way to tell if it works is to play a very long game of Freeciv, but that is an exercise left to the reader.

Review: What Just Happened?

Now that you've successfully transformed code into gold, step back and take a look at what's going on. The process you just walked through is virtually identical to a large number of FOSS projects, so you're likely to see it often.

Where do all of these complicated configuration files come from in the first place, and why?

The goal of good build tools is to allow the developer to describe the project as simply as possible. There's a lot of complexity, but it's important to isolate that complexity.

Developers of an Autotools-based project seek to describe the build process of the entire project in two files:

  •, which (basically) describes what needs to be built;
  • (or, which (basically) describes how it needs to be built.

These descriptions start out simple, but as the project grows in complexity, the descriptions of the project naturally grow as well.

Given these configuration files, you ran through the Autotools suite. The purpose of the Autotools suite is to build a good configure script, which in turn builds a good Makefile, which in turn builds good executable binaries.

| |
|  |
       | Run 
       | (a wrapper for the Autogen tools)
|  configure   |
       | ./configure
       | (repeat as necessary)
|   Makefile   |
       | make
|   Lots of    |
| successfully |
|  compiled    |
|    code      |

Note: one of the things you did not do was to run the make install command, which would install Freeciv as an "official" binary that could be run by anyone on the system. Since this is a development version, keep it in your own working directory, where you can happily break it as much as you want.

Again: very few people know every detail about Autotools, or about any build tools. Experienced developers know as much as they need to know to get the job done, and they know where to look for help when they're stuck.

Exercise - Building Your Developer Workstation

By now, you have probably chosen a FOSS project that interests you. Maybe you've been running that code from a pre-built binary that you downloaded, but now it's time to build the executables from scratch.

Go check out the project's codebase from its SCM. Walk through the entire build process. In some cases, this might be pretty simple; it some cases, it might be quite complicated. Use all of the tools available to you: install instructions, mailing lists, Google, IRC, etc. Build the code, and then run the code; that's your goal.

As you go through this build process, blog your process in a manner similar to how you walked through your build of Freeciv in this chapter.

If the build was easy, proceed to the next exercise: create two separate builds in two separate directories -- the latest stable release, and the release from HEAD. FOSS developers frequently do this to compare behaviors between versions.

Supplemental Reading

John Calcote wrote an excellent guide to the GNU Autotools. If you find yourself working with a project that uses these tools, Calcote's work is a great online reference for beginners.

The much longer, but more definitive, work on GNU Autotools is GNU Autoconf, Automake, and Libtool (also known as "The Goat Book") by Gary V. Vaughan, Ben Elliston, Tom Tromey and Ian Lance Taylor. Interestingly, it was a collaborative work written online by people who had never met in person.

The Mozilla project uses GNU Autotools for all of their projects, and their build documentation is available online. It's a great exercise for those who want experience with building complex software designed to run on multiple platforms.

For those who want to start their own project and want it to be Autotools-friendly in the first place, here's a great tutorial.

GNU Autotools is very common, and this book had to start somewhere with an exploration of build automation -- but there are lots of other excellent build tools. In the world of Java software, Ant and Maven are extremely popular choices. Here's a fairly comprehensive list of popular build automation tools.

Building software can be frustrating, and a lot of FOSS projects do a lot of things wrong. To be clear: most proprietary software projects probably do a lot of things wrong too, but the failures of FOSS projects are visible for all to see -- and thus, to learn from. Tom 'spot' Callaway has compiled a list of things that projects do wrong, in a piece entitled How to tell if a FLOSS project is doomed to FAIL.

Personal tools