![]()
|
NAMECons - A Software Construction System
DESCRIPTIONA guide and reference for version 2.2.0 Copyright (c) 1996-2000 Free Software Foundation, Inc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; see the file COPYING. If not, write to the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
IntroductionCons is a system for constructing, primarily, software, but is quite different from previous software construction systems. Cons was designed from the ground up to deal easily with the construction of software spread over multiple source directories. Cons makes it easy to create build scripts that are simple, understandable and maintainable. Cons ensures that complex software is easily and accurately reproducible. Cons uses a number of techniques to accomplish all of this. Construction scripts are just Perl scripts, making them both easy to comprehend and very flexible. Global scoping of variables is replaced with an import/export mechanism for sharing information between scripts, significantly improving the readability and maintainability of each script. Construction environments are introduced: these are Perl objects that capture the information required for controlling the build process. Multiple environments are used when different semantics are required for generating products in the build tree. Cons implements automatic dependency analysis and uses this to globally sequence the entire build. Variant builds are easily produced from a single source tree. Intelligent build subsetting is possible, when working on localized changes. Overrides can be setup to easily override build instructions without modifying any scripts. MD5 cryptographic signatures are associated with derived files, and are used to accurately determine whether a given file needs to be rebuilt. While offering all of the above, and more, Cons remains simple and easy to use. This will, hopefully, become clear as you read the remainder of this document.
Why Cons? Why not Make?Cons is a make replacement. In the following paragraphs, we look at a few of the undesirable characteristics of make--and typical build environments based on make--that motivated the development of Cons.
Build complexityTraditional make-based systems of any size tend to become quite complex. The original make utility and its derivatives have contributed to this tendency in a number of ways. Make is not good at dealing with systems that are spread over multiple directories. Various work-arounds are used to overcome this difficulty; the usual choice is for make to invoke itself recursively for each sub-directory of a build. This leads to complicated code, in which it is often unclear how a variable is set, or what effect the setting of a variable will have on the build as a whole. The make scripting language has gradually been extended to provide more possibilities, but these have largely served to clutter an already overextended language. Often, builds are done in multiple passes in order to provide appropriate products from one directory to another directory. This represents a further increase in build complexity.
Build reproducibilityThe bane of all makes has always been the correct handling of dependencies. Most often, an attempt is made to do a reasonable job of dependencies within a single directory, but no serious attempt is made to do the job between directories. Even when dependencies are working correctly, make's reliance on a simple time stamp comparison to determine whether a file is out of date with respect to its dependents is not, in general, adequate for determining when a file should be rederived. If an external library, for example, is rebuilt and then ``snapped'' into place, the timestamps on its newly created files may well be earlier than the last local build, since it was built before it became visible.
Variant buildsMake provides only limited facilities for handling variant builds. With the proliferation of hardware platforms and the need for debuggable vs. optimized code, the ability to easily create these variants is essential. More importantly, if variants are created, it is important to either be able to separate the variants or to be able to reproduce the original or variant at will. With make it is very difficult to separate the builds into multiple build directories, separate from the source. And if this technique isn't used, it's also virtually impossible to guarantee at any given time which variant is present in the tree, without resorting to a complete rebuild.
RepositoriesMake provides only limited support for building software from code that exists in a central repository directory structure. The VPATH feature of GNU make (and some other make implementations) is intended to provide this, but doesn't work as expected: it changes the path of target file to the VPATH name too early in its analysis, and therefore searches for all dependencies in the VPATH directory. To ensure correct development builds, it is important to be able to create a file in a local build directory and have any files in a code repository (a VPATH directory, in make terms) that depend on the local file get rebuilt properly. This isn't possible with VPATH, without coding a lot of complex repository knowledge directly into the makefiles.
Keeping it simpleA few of the difficulties with make have been cited above. In this and subsequent sections, we shall introduce Cons and show how these issues are addressed.
Perl scriptsCons is Perl-based. That is, Cons scripts--Conscript and Construct files, the equivalent to Makefile or makefile--are all written in Perl. This provides an immediate benefit: the language for writing scripts is a familiar one. Even if you don't happen to be a Perl programmer, it helps to know that Perl is basically just a simple declarative language, with a well-defined flow of control, and familiar semantics. It has variables that behave basically the way you would expect them to, subroutines, flow of control, and so on. There is no special syntax introduced for Cons. The use of Perl as a scripting language simplifies the task of expressing the appropriate solution to the often complex requirements of a build.
Hello, World!To ground the following discussion, here's how you could build the Hello, World! C application with Cons: $env = new cons(); Program $env 'hello', 'hello.c'; If you install this script in a directory, naming the script Construct,
and create the hello.c source file in the same directory, then you can
type % cons hello cc -c hello.c -o hello.o cc -o hello hello.o
Construction environmentsA key simplification of Cons is the idea of a construction environment. A construction environment is an object characterized by a set of key/value pairs and a set of methods. In order to tell Cons how to build something, you invoke the appropriate method via an appropriate construction environment. Consider the following example: $env = new cons( CC => 'gcc', LIBS => 'libworld.a' ); Program $env 'hello', 'hello.c'; In this case, rather than using the default construction environment, as is,
we have overridden the value of Library $env 'libworld', 'world.c'; Now if you type % cons hello gcc -c hello.c -o hello.o gcc -c world.c -o world.o ar r libworld.a world.o ar: creating libworld.a ranlib libworld.a gcc -o hello hello.o libworld.a
Automatic and complete dependency analysisWith Cons, dependencies are handled automatically. Continuing the previous example, note that when we modify world.c, world.o is recompiled, libworld.a recreated, and hello relinked: % vi world.c [EDIT] % cons hello gcc -c world.c -o world.o ar r libworld.a world.o ar: creating libworld.a ranlib libworld.a gcc -o hello hello.o libworld.a This is a relatively simple example: Cons ``knows'' world.o depends upon
world.c, because the dependency is explicitly set up by the Now it turns out that hello.c also includes the interface definition file, world.h: % emacs world.h [EDIT] % cons hello gcc -c hello.c -o hello.o gcc -o hello hello.o libworld.a How does Cons know that hello.c includes world.h, and that hello.o must therefore be recompiled? For now, suffice it to say that when considering whether or not hello.o is up-to-date, Cons invokes a scanner for its dependency, hello.c. This scanner enumerates the files included by hello.c to come up with a list of further dependencies, beyond those made explicit by the Cons script. This process is recursive: any files included by included files will also be scanned. Isn't this expensive? The answer is--it depends. If you do a full build of a large system, the scanning time is insignificant. If you do a rebuild of a large system, then Cons will spend a fair amount of time thinking about it before it decides that nothing has to be done (although not necessarily more time than make!). The good news is that Cons makes it very easy to intelligently subset your build, when you are working on localized changes.
Automatic global build sequencingBecause Cons does full and accurate dependency analysis, and does this globally, for the entire build, Cons is able to use this information to take full control of the sequencing of the build. This sequencing is evident in the above examples, and is equivalent to what you would expect for make, given a full set of dependencies. With Cons, this extends trivially to larger, multi-directory builds. As a result, all of the complexity involved in making sure that a build is organized correctly--including multi-pass hierarchical builds--is eliminated. We'll discuss this further in the next sections.
Building large trees--still just as simple
A hierarchy of build scriptsA larger build, in Cons, is organized by creating a hierarchy of build
scripts. At the top of the tree is a script called Construct. The rest
of the scripts, by convention, are each called Conscript. These scripts
are connected together, very simply, by the
The Build commandThe Build qw( drivers/display/Conscript drivers/mouse/Conscript parser/Conscript utilities/Conscript ); This is a simple two-level hierarchy of build scripts: all the subsidiary Conscript files are mentioned in the top-level Construct file. Notice that not all directories in the tree necessarily have build scripts associated with them. This could also be written as a multi-level script. For example, the Construct file might contain this command: Build qw( parser/Conscript drivers/Conscript utilities/Conscript ); and the Conscript file in the drivers directory might contain this: Build qw( display/Conscript mouse/Conscript ); Experience has shown that the former model is a little easier to understand, since the whole construction tree is laid out in front of you, at the top-level. Hybrid schemes are also possible. A separately maintained component that needs to be incorporated into a build tree, for example, might hook into the build tree in one place, but define its own construction hierarchy. By default, Cons does not change its working directory to the directory containing a subsidiary Conscript file it is including. This behavior can be enabled for a build by specifying, in the top-level Construct file: Conscript_chdir 1; When enabled, Cons will change to the subsidiary Conscript file's containing directory while reading in that file, and then change back to the top-level directory once the file has been processed. It is expected that this behavior will become the default in some future version of Cons. To prepare for this transition, builds that expect Cons to remain at the top of the build while it reads in a subsidiary Conscript file should explicitly disable this feature as follows: Conscript_chdir 0;
Relative, top-relative, and absolute file namesYou may have noticed that the file names specified to the Build command are relative to the location of the script it is invoked from. This is generally true for other filename arguments to other commands, too, although we might as well mention here that if you begin a file name with a hash mark, ``#'', then that file is interpreted relative to the top-level directory (where the Construct file resides). And, not surprisingly, if you begin it with ``/'', then it is considered to be an absolute pathname. This is true even on systems which use a back slash rather than a forward slash to name absolute paths.
Using modules in build scriptsYou may pull modules into each Conscript file using the normal Perl
use English; require My::Module; Each
Scope of variablesThe top-level Construct file and all Conscript files begin life in a common, separate Perl package. Cons controls the symbol table for the package so that, the symbol table for each script is empty, except for the Construct file, which gets some of the command line arguments. All of the variables that are set or used, therefore, are set by the script itself--not by some external script. Variables can be explicitly imported by a script from its parent script. To import a variable, it must have been exported by the parent and initialized (otherwise an error will occur).
The Export commandThe $env = new cons(); $INCLUDE = "#export/include"; $LIB = "#export/lib"; Export qw( env INCLUDE LIB ); Build qw( util/Conscript ); The values of the simple variables mentioned in the $env = new cons(); Export qw( env INCLUDE LIB ); Build qw( util/Conscript ); $env = new cons(CFLAGS => '-O'); Build qw( other/Conscript ); It doesn't matter whether the variable is set before or after the
The Import commandVariables exported by the Import qw( env INCLUDE ); This is only legal if the parent script exported both All the imported variables are automatically re-exported, so the sequence: Import qw ( env INCLUDE ); Build qw ( beneath-me/Conscript ); will supply both Import qw ( env INCLUDE ); Export qw ( env ); Build qw ( beneath-me/Conscript ); Needless to say, the variables may be modified locally before invoking
Build script evaluation orderThe only constraint on the ordering of build scripts is that superior
scripts are evaluated before their inferior scripts. The top-level
Construct file, for instance, is evaluated first, followed by any
inferior scripts. This is all you really need to know about the evaluation
order, since order is generally irrelevant. Consider the following Build qw( drivers/display/Conscript drivers/mouse/Conscript parser/Conscript utilities/Conscript ); We've chosen to put the script names in alphabetical order, simply because that's the most convenient for maintenance purposes. Changing the order will make no difference to the build.
A Model for sharing files
Some simple conventionsIn any complex software system, a method for sharing build products needs to be established. We propose a simple set of conventions which are trivial to implement with Cons, but very effective. The basic rule is to require that all build products which need to be shared between directories are shared via an intermediate directory. We have typically called this export, and, in a C environment, provided conventional sub-directories of this directory, such as include, lib, bin, etc. These directories are defined by the top-level Construct file. A simple Construct file for a Hello, World! application, organized using multiple directories, might look like this: # Construct file for Hello, World! # Where to put all our shared products. $EXPORT = '#export'; Export qw( CONS INCLUDE LIB BIN ); # Standard directories for sharing products. $INCLUDE = "$EXPORT/include"; $LIB = "$EXPORT/lib"; $BIN = "$EXPORT/bin"; # A standard construction environment. $CONS = new cons ( CPPPATH => $INCLUDE, # Include path for C Compilations LIBPATH => $LIB, # Library path for linking programs LIBS => '-lworld', # List of standard libraries ); Build qw( hello/Conscript world/Conscript ); The world directory's Conscript file looks like this: # Conscript file for directory world Import qw( CONS INCLUDE LIB ); # Install the products of this directory Install $CONS $LIB, 'libworld.a'; Install $CONS $INCLUDE, 'world.h'; # Internal products Library $CONS 'libworld.a', 'world.c'; and the hello directory's Conscript file looks like this: # Conscript file for directory hello Import qw( CONS BIN ); # Exported products Install $CONS $BIN, 'hello'; # Internal products Program $CONS 'hello', 'hello.c'; To construct a Hello, World! program with this directory structure, go to
the top-level directory, and invoke % cons export Install world/world.h as export/include/world.h cc -Iexport/include -c hello/hello.c -o hello/hello.o cc -Iexport/include -c world/world.c -o world/world.o ar r world/libworld.a world/world.o ar: creating world/libworld.a ranlib world/libworld.a Install world/libworld.a as export/lib/libworld.a cc -o hello/hello hello/hello.o -Lexport/lib -lworld Install hello/hello as export/bin/hello
Clean, understandable, location-independent scriptsYou'll note that the two Conscript files are very clean and to-the-point. They simply specify products of the directory and how to build those products. The build instructions are minimal: they specify which construction environment to use, the name of the product, and the name of the inputs. Note also that the scripts are location-independent: if you wish to reorganize your source tree, you are free to do so: you only have to change the Construct file (in this example), to specify the new locations of the Conscript files. The use of an export tree makes this goal easy. Note, too, how Cons takes care of little details for you. All the export directories, for example, were made automatically. And the installed files were really hard-linked into the respective export directories, to save space and time. This attention to detail saves considerable work, and makes it even easier to produce simple, maintainable scripts.
Separating source and build treesIt's often desirable to keep any derived files from the build completely separate from the source files. This makes it much easier to keep track of just what is a source file, and also makes it simpler to handle variant builds, especially if you want the variant builds to co-exist.
Separating build and source directories using the Link commandCons provides a simple mechanism that handles all of these requirements. The
Link 'build' => 'src'; The specified directories are ``linked'' to the specified source directory. Let's suppose that you setup a source directory, src, with the sub-directories world and hello below it, as in the previous example. You could then substitute for the original build lines the following: Build qw( build/world/Conscript build/hello/Conscript ); Notice that you treat the Conscript file as if it existed in the build directory. Now if you type the same command as before, you will get the following results: % cons export Install build/world/world.h as export/include/world.h cc -Iexport/include -c build/hello/hello.c -o build/hello/hello.o cc -Iexport/include -c build/world/world.c -o build/world/world.o ar r build/world/libworld.a build/world/world.o ar: creating build/world/libworld.a ranlib build/world/libworld.a Install build/world/libworld.a as export/lib/libworld.a cc -o build/hello/hello build/hello/hello.o -Lexport/lib -lworld Install build/hello/hello as export/bin/hello Again, Cons has taken care of the details for you. In particular, you will notice that all the builds are done using source files and object files from the build directory. For example, build/world/world.o is compiled from build/world/world.c, and export/include/world.h is installed from build/world/world.h. This is accomplished on most systems by the simple expedient of ``hard'' linking the required files from each source directory into the appropriate build directory. The links are maintained correctly by Cons, no matter what you do to the source directory. If you modify a source file, your editor may do this ``in place'' or it may rename it first and create a new file. In the latter case, any hard link will be lost. Cons will detect this condition the next time the source file is needed, and will relink it appropriately. You'll also notice, by the way, that no changes were required to the underlying Conscript files. And we can go further, as we shall see in the next section.
Variant builds
Hello, World! for baNaNa and peAcH OS'sVariant builds require just another simple extension. Let's take as an
example a requirement to allow builds for both the baNaNa and peAcH
operating systems. In this case, we are using a distributed file system,
such as NFS to access the particular system, and only one or the other of
the systems has to be compiled for any given invocation of # Construct file for Hello, World! die qq(OS must be specified) unless $OS = $ARG{OS}; die qq(OS must be "peach" or "banana") if $OS ne "peach" && $OS ne "banana"; # Where to put all our shared products. $EXPORT = "#export/$OS"; Export qw( CONS INCLUDE LIB BIN ); # Standard directories for sharing products. $INCLUDE = "$EXPORT/include"; $LIB = "$EXPORT/lib"; $BIN = "$EXPORT/bin"; # A standard construction environment. $CONS = new cons ( CPPPATH => $INCLUDE, # Include path for C Compilations LIBPATH => $LIB, # Library path for linking programs LIBS => '-lworld', # List of standard libraries ); # $BUILD is where we will derive everything. $BUILD = "#build/$OS"; # Tell cons where the source files for $BUILD are. Link $BUILD => 'src'; Build ( "$BUILD/hello/Conscript", "$BUILD/world/Conscript", ); Now if we login to a peAcH system, we can build our Hello, World! application for that platform: % cons export OS=peach Install build/peach/world/world.h as export/peach/include/world.h cc -Iexport/peach/include -c build/peach/hello/hello.c -o build/peach/hello/hello.o cc -Iexport/peach/include -c build/peach/world/world.c -o build/peach/world/world.o ar r build/peach/world/libworld.a build/peach/world/world.o ar: creating build/peach/world/libworld.a ranlib build/peach/world/libworld.a Install build/peach/world/libworld.a as export/peach/lib/libworld.a cc -o build/peach/hello/hello build/peach/hello/hello.o -Lexport/peach/lib -lworld Install build/peach/hello/hello as export/peach/bin/hello
Variations on a themeOther variations of this model are possible. For example, you might decide
that you want to separate out your include files into platform dependent and
platform independent files. In this case, you'd have to define an
alternative to You might also want to be able to compile your whole system with debugging
or profiling, for example, enabled. You could do this with appropriate
command line options, such as
Signatures
MD5 cryptographic signaturesWhenever Cons creates a derived file, it stores a signature for that file. The signature is stored in a separate file, one per directory. After the previous example was compiled, the .consign file in the build/peach/world directory looked like this: world.o:834179303 23844c0b102ecdc0b4548d1cd1cbd8c6 libworld.a:834179304 9bf6587fa06ec49d864811a105222c00 The first number is a timestamp--for a UNIX systems, this is typically the number of seconds since January 1st, 1970. The second value is an MD5 checksum. The Message Digest Algorithm is an algorithm that, given an input string, computes a strong cryptographic signature for that string. The MD5 checksum stored in the .consign file is, in effect, a digest of all the dependency information for the specified file. So, for example, for the world.o file, this includes at least the world.c file, and also any header files that Cons knows about that are included, directly or indirectly by world.c. Not only that, but the actual command line that was used to generate world.o is also fed into the computation of the signature. Similarly, libworld.a gets a signature which ``includes'' all the signatures of its constituents (and hence, transitively, the signatures of their constituents), as well as the command line that created the file. The signature of a non-derived file is computed, by default, by taking the current modification time of the file and the file's entry name (unless there happens to be a current .consign entry for that file, in which case that signature is used). Notice that there is no need for a derived file to depend upon any particular Construct or Conscript file--if changes to these files affect the file in question, then this will be automatically reflected in its signature, since relevant parts of the command line are included in the signature. Unrelated changes will have no effect. When Cons considers whether to derive a particular file, then, it first computes the expected signature of the file. It then compares the file's last modification time with the time recorded in the .consign entry, if one exists. If these times match, then the signature stored in the .consign file is considered to be accurate. If the file's previous signature does not match the new, expected signature, then the file must be rederived. Notice that a file will be rederived whenever anything about a dependent file changes. In particular, notice that any change to the modification time of a dependent (forward or backwards in time) will force recompilation of the derived file. The use of these signatures is an extremely simple, efficient, and effective method of improving--dramatically--the reproducibility of a system. We'll demonstrate this with a simple example: # Simple "Hello, World!" Construct file $CFLAGS = '-g' if $ARG{DEBUG} eq 'on'; $CONS = new cons(CFLAGS => $CFLAGS); Program $CONS 'hello', 'hello.c'; Notice how Cons recompiles at the appropriate times: % cons hello cc -c hello.c -o hello.o cc -o hello hello.o % cons hello cons: "hello" is up-to-date. % cons DEBUG=on hello cc -g -c hello.c -o hello.o cc -o hello hello.o % cons DEBUG=on hello cons: "hello" is up-to-date. % cons hello cc -c hello.c -o hello.o cc -o hello hello.o
Code RepositoriesMany software development organizations will have one or more central repository directory trees containing the current source code for one or more projects, as well as the derived object files, libraries, and executables. In order to reduce unnecessary recompilation, it is useful to use files from the repository to build development software--assuming, of course, that no newer dependency file exists in the local build tree.
RepositoryCons provides a mechanism to specify a list of code repositories that will be searched, in-order, for source files and derived files not found in the local build directory tree. The following lines in a Construct file will instruct Cons to look first under the /usr/experiment/repository directory and then under the /usr/product/repository directory: Repository qw ( /usr/experiment/repository /usr/product/repository ); The repository directories specified may contain source files, derived files (objects, libraries and executables), or both. If there is no local file (source or derived) under the directory in which Cons is executed, then the first copy of a same-named file found under a repository directory will be used to build any local derived files. Cons maintains one global list of repositories directories. Cons will eliminate the current directory, and any non-existent directories, from the list.
Finding the Construct file in a RepositoryCons will also search for Construct and Conscript files in the
repository tree or trees. This leads to a chicken-and-egg situation,
though: how do you look in a repository tree for a Construct file if the
Construct file tells you where the repository is? To get around this,
repositories may be specified via % cons -R /usr/experiment/repository -R /usr/product/repository . Any repository directories specified in the Construct or Conscript
files will be appended to the repository directories specified by
command-line
Repository source filesIf the source code (include the Conscript file) for the library version of the Hello, World! C application is in a repository (with no derived files), Cons will use the repository source files to create the local object files and executable file: % cons -R /usr/src_only/repository hello gcc -c /usr/src_only/repository/hello.c -o hello.o gcc -c /usr/src_only/repository/world.c -o world.o ar r libworld.a world.o ar: creating libworld.a ranlib libworld.a gcc -o hello hello.o libworld.a Creating a local source file will cause Cons to rebuild the appropriate derived file or files: % pico world.c [EDIT] % cons -R /usr/src_only/repository hello gcc -c world.c -o world.o ar r libworld.a world.o ar: creating libworld.a ranlib libworld.a gcc -o hello hello.o libworld.a And removing the local source file will cause Cons to revert back to building the derived files from the repository source: % rm world.c % cons -R /usr/src_only/repository hello gcc -c /usr/src_only/repository/world.c -o world.o ar r libworld.a world.o ar: creating libworld.a ranlib libworld.a gcc -o hello hello.o libworld.a
Repository derived filesIf a repository tree contains derived files (usually object files, libraries, or executables), Cons will perform its normal signature calculation to decide whether the repository file is up-to-date or a derived file must be built locally. This means that, in order to ensure correct signature calculation, a repository tree must also contain the .consign files that were created by Cons when generating the derived files. This would usually be accomplished by building the software in the repository (or, alternatively, in a build directory, and then copying the result to the repository): % cd /usr/all/repository % cons hello gcc -c hello.c -o hello.o gcc -c world.c -o world.o ar r libworld.a world.o ar: creating libworld.a ranlib libworld.a gcc -o hello hello.o libworld.a (This is safe even if the Construct file lists the /usr/all/repository
directory in a Now if we want to build a copy of the application with our own hello.c
file, we only need to create the one necessary source file, and use the
% mkdir $HOME/build1 % cd $HOME/build1 % ed hello.c [EDIT] % cons -R /usr/all/repository hello gcc -c hello.c -o hello.o gcc -o hello hello.o /usr/all/repository/libworld.a Notice that Cons has not bothered to recreate a local libworld.a library (or recompile the world.o module), but instead uses the already-compiled version from the repository. Because the MD5 signatures that Cons puts in the .consign file contain timestamps for the derived files, the signature timestamps must match the file timestamps for a signature to be considered valid. Some software systems may alter the timestamps on repository files (by copying them, e.g.), in which case Cons will, by default, assume the repository signatures are invalid and rebuild files unnecessarily. This behavior may be altered by specifying: Repository_Sig_Times_OK 0; This tells Cons to ignore timestamps when deciding whether a signature is valid. (Note that avoiding this sanity check means there must be proper control over the repository tree to ensure that the derived files cannot be modified without updating the .consign signature.)
Local copies of filesIf the repository tree contains the complete results of a build, and we try to build from the repository without any files in our local tree, something moderately surprising happens: % mkdir $HOME/build2 % cd $HOME/build2 % cons -R /usr/all/repository hello cons: "hello" is up-to-date. Why does Cons say that the hello program is up-to-date when there is no hello program in the local build directory? Because the repository (not the local directory) contains the up-to-date hello program, and Cons correctly determines that nothing needs to be done to rebuild this up-to-date copy of the file. There are, however, many times in which it is appropriate to ensure that a
local copy of a file always exists. A packaging or testing script, for
example, may assume that certain generated files exist locally. Instead of
making these subsidiary scripts aware of the repository directory, the
Local qw( hello ); Then, if we re-run the same command, Cons will make a local copy of the program from the repository copy (telling you that it is doing so): % cons -R /usr/all/repository hello Local copy of hello from /usr/all/repository/hello cons: "hello" is up-to-date. Notice that, because the act of making the local copy is not considered a ``build'' of the hello file, Cons still reports that it is up-to-date. Creating local copies is most useful for files that are being installed into
an intermediate directory (for sharing with other directories) via the
Install_Local $env, '#export', 'hello'; is exactly equivalent to: Install $env '#export', 'hello'; Local '#export/hello'; Both the
Repository dependency analysisDue to its built-in scanning, Cons will search the specified repository trees for included .h files. Unless the compiler also knows about the repository trees, though, it will be unable to find .h files that only exist in a repository. If, for example, the hello.c file includes the hello.h file in its current directory: % cons -R /usr/all/repository hello gcc -c /usr/all/repository/hello.c -o hello.o /usr/all/repository/hello.c:1: hello.h: No such file or directory Solving this problem forces some requirements onto the way construction
environments are defined and onto the way the C In order to inform the compiler about the repository trees, Cons will add
appropriate $env = new cons( CC => 'gcc', CPPPATH => '.', LIBS => 'libworld.a', ); Due to the definition of the % cons -R /usr/all/repository hello gcc -c -I. -I/usr/all/repository /usr/all/repository/hello.c -o hello.o gcc -o hello hello.o /usr/all/repository/libworld.a The order of the Repository qw( /u1 /u2 ); $env = new cons( CPPPATH => 'a:b:c', ); Would yield a compilation command of: cc -Ia -I/u1/a -I/u2/a -Ib -I/u1/b -I/u2/b -Ic -I/u1/c -I/u2/c -c hello.c -o hello.o Because Cons relies on the compiler's #include "file.h" /* DON'T USE DOUBLE-QUOTES LIKE THIS */ This is because most C preprocessors, when faced with such a directive, will
always first search the directory containing the source file. This
undermines the elaborate Consequently, when using repository trees in Cons, always use angle-brackets for included files: #include <file.h> /* USE ANGLE-BRACKETS INSTEAD */
Repository_ListCons provides a @list = Repository_List; print join(' ', @list), "\n";
Repository interaction with other Cons featuresCons' handling of repository trees interacts correctly with other Cons features--which is to say, it generally does what you would expect. Most notably, repository trees interact correctly, and rather powerfully,
with the 'Link' command. A repository tree may contain one or more
subdirectories for version builds established via
Default targetsUntil now, we've demonstrated invoking Cons with an explicit target to build: % cons hello Normally, Cons does not build anything unless a target is specified, but specifying '.' (the current directory) will build everything: % cons # does not build anything % cons . # builds everything under the top-level directory Adding the Default '.'; The following would add the hello and goodbye commands (in the same directory as the Construct or Conscript file) to the default list: Default qw( hello goodbye ); The
Selective buildsCons provides two methods for reducing the size of given build. The first is by specifying targets on the command line, and the second is a method for pruning the build tree. We'll consider target specification first.
Selective targetingLike make, Cons allows the specification of ``targets'' on the command line. Cons targets may be either files or directories. When a directory is specified, this is simply a short-hand notation for every derivable product--that Cons knows about--in the specified directory and below. For example: % cons build/hello/hello.o means build hello.o and everything that hello.o might need. This is from a previous version of the Hello, World! program in which hello.o depended upon export/include/world.h. If that file is not up-to-date (because someone modified src/world/world.h), then it will be rebuilt, even though it is in a directory remote from build/hello. In this example: % cons build Everything in the build directory is built, if necessary. Again, this may cause more files to be built. In particular, both export/include/world.h and export/lib/libworld.a are required by the build/hello directory, and so they will be built if they are out-of-date. If we do, instead: % cons export then only the files that should be installed in the export directory will be
rebuilt, if necessary, and then installed there. Note that
No ``special'' targetsWith Cons, make-style ``special'' targets are not required. The simplest analog with Cons is to use special export directories, instead. Let's suppose, for example, that you have a whole series of unit tests that are associated with your code. The tests live in the source directory near the code. Normally, however, you don't want to build these tests. One solution is to provide all the build instructions for creating the tests, and then to install the tests into a separate part of the tree. If we install the tests in a top-level directory called tests, then: % cons tests will build all the tests. % cons export will build the production version of the system (but not the tests), and: % cons build should probably be avoided (since it will compile tests unecessarily). If you want to build just a single test, then you could explicitly name the test (in either the tests directory or the build directory). You could also aggregate the tests into a convenient hierarchy within the tests directory. This hierarchy need not necessarily match the source hierarchy, in much the same manner that the include hierarchy probably doesn't match the source hierarchy (the include hierarchy is unlikely to be more than two levels deep, for C programs). If you want to build absolutely everything in the tree (subject to whatever options you select), you can use: % cons . This is not particularly efficient, since it will redundantly walk all the trees, including the source tree. The source tree, of course, may have buildable objects in it--nothing stops you from doing this, even if you normally build in a separate build tree.
Build PruningIn conjunction with target selection, build pruning can be used to reduce
the scope of the build. In the previous peAcH and baNaNa example, we have
already seen how script-driven build pruning can be used to make only half
of the potential build available for any given invocation of % cons build +world The In the example, above, the hello program will not be built, since Cons will have no knowledge of the script hello/Conscript. The libworld.a archive will be built, however, if need be. There are a couple of uses for build pruning via the command line. Perhaps the most useful is the ability to make local changes, and then, with sufficient knowledge of the consequences of those changes, restrict the size of the build tree in order to speed up the rebuild time. A second use for build pruning is to actively prevent the recompilation of certain files that you know will recompile due to, for example, a modified header file. You may know that either the changes to the header file are immaterial, or that the changes may be safely ignored for most of the tree, for testing purposes.With Cons, the view is that it is pragmatic to admit this type of behavior, with the understanding that on the next full build everything that needs to be rebuilt will be. There is no equivalent to a ``make touch'' command, to mark files as permanently up-to-date. So any risk that is incurred by build pruning is mitigated. For release quality work, obviously, we recommend that you do not use build pruning (it's perfectly OK to use during integration, however, for checking compilation, etc. Just be sure to do an unconstrained build before committing the integration).
Temporary overridesCons provides a very simple mechanism for overriding aspects of a build. The
essence is that you write an override file containing one or more
% cons -o over export will build the export directory, with all derived files subject to the
overrides present in the over file. If you leave out the
Overriding environment variablesThe override file can contain two types of overrides. The first is incoming
environment variables. These are normally accessible by the Construct
file from the
The Override commandThe second type of override is accomplished with the Override <regexp>, <var1> => <value1>, <var2> => <value2>, ...; The regular expression regexp is matched against every derived file that is a candidate for the build. If the derived file matches, then the variable/value pairs are used to override the values in the construction environment associated with the derived file. Let's suppose that we have a construction environment like this: $CONS = new cons( COPT => '', CDBG => '-g', CFLAGS => '%COPT %CDBG', ); Then if we have an override file over containing this command: Override '\.o$', COPT => '-O', CDBG => ''; then any Here's the original version of the Hello, World! program, built with this environment. Note that Cons rebuilds the appropriate pieces when the override is applied or removed: % cons hello cc -g -c hello.c -o hello.o cc -o hello hello.o % cons -o over hello cc -O -c hello.c -o hello.o cc -o hello hello.o % cons -o over hello cons: "hello" is up-to-date. % cons hello cc -g -c hello.c -o hello.o cc -o hello hello.o It's important that the Note that it is still useful to provide, say, the ability to create a fully optimized version of a system for production use--from the Construct and Conscript files. This way you can tailor the optimized system to the platform. Where optimizer trade-offs need to be made (particular files may not be compiled with full optimization, for example), then these can be recorded for posterity (and reproducibility) directly in the scripts.
More on construction environments
Default construction variablesWe have mentioned, and used, the concept of a construction environment, many times in the preceding pages. Now it's time to make this a little more concrete. With the following statement: $env = new cons(); a reference to a new, default construction environment is created. This contains a number of construction variables and some methods. At the present writing, the default list of construction variables is defined as follows: CC => 'cc', CFLAGS => '', CCCOM => '%CC %CFLAGS %_IFLAGS -c %< -o %>', INCDIRPREFIX => '-I', CXX => '%CC', CXXFLAGS => '%CFLAGS', CXXCOM => '%CXX %CXXFLAGS %_IFLAGS -c %< -o %>', LINK => '%CXX', LINKCOM => '%LINK %LDFLAGS -o %> %< %_LDIRS %LIBS', LINKMODULECOM => '%LD -r -o %> %<', LIBDIRPREFIX => '-L', AR => 'ar', ARFLAGS => 'r', ARCOM => "%AR %ARFLAGS %> %<\n%RANLIB %>", RANLIB => 'ranlib', AS => 'as', ASFLAGS => '', ASCOM => '%AS %ASFLAGS %< -o %>', LD => 'ld', LDFLAGS => '', PREFLIB => 'lib', SUFLIB => '.a', SUFLIBS => '.so:.a', SUFOBJ => '.o', ENV => { 'PATH' => '/bin:/usr/bin' }, On Win32 systems (Windows NT), the following construction variables are overridden in the default: CC => 'cl', CFLAGS => '/nologo', CCCOM => '%CC %CFLAGS %_IFLAGS /c %< /Fo%>', CXXCOM => '%CXX %CXXFLAGS %_IFLAGS /c %< /Fo%>', INCDIRPREFIX => '/I', LINK => 'link', LINKCOM => '%LINK %LDFLAGS /out:%> %< %_LDIRS %LIBS', LINKMODULECOM => '%LD /r /o %> %<', LIBDIRPREFIX => '/LIBPATH:', AR => 'lib', ARFLAGS => '/nologo ', ARCOM => "%AR %ARFLAGS /out:%> %<", RANLIB => '', LD => 'link', LDFLAGS => '/nologo ', PREFLIB => '', SUFEXE => '.exe', SUFLIB => '.lib', SUFLIBS => '.dll:.lib', SUFOBJ => '.obj', These variables are used by the various methods associated with the
environment, in particular any method that ultimately invokes an external
command will substitute these variables into the final command, as
appropriate. For example, the Objects $env 'foo.c', 'bar.c'; This will arrange to produce, if necessary, foo.o and bar.o. The
command invoked is simply The construction variables are also used for other purposes. For example,
Note that, for any particular environment, the value of a variable is set
once, and then never reset (to change a variable, you must create a new
environment. Methods are provided for copying existing environments for this
purpose). Some internal variables, such as The Another variable,
Interpolating construction variablesConstruction environment variables may be interpolated in the source and
target file names by prefixing the construction variable name with $env = new cons( DESTDIR => 'programs', SRCDIR => 'src', ); Program $env '%DESTDIR/hello', '%SRCDIR/hello.c'; Expansion of construction variables is recursive--that is, the file
Default construction methodsThe list of default construction methods includes the following:
The
|