pbuilder User's Manual

pbuilder stands for Personal Builder, and it is an automatic Debian Package Building system for personal development workstation environments. pbuilder aims to be an easy-to-setup system for auto-building Debian packages inside a clean-room environment, so that it is possible to verify that a package can be built on most Debian installations. The clean-room environment is achieved through the use of a base chroot image, so that only minimal packages will be installed inside the chroot.

The Debian distribution consists of free software accompanied with source. The source code within Debian's "main" section must build within Debian "main", with only the explicitly specified build-dependencies installed.

The primary aim of pbuilder is different from other auto-building systems in Debian in that its aim is not to try to build as many packages as possible. It does not try to guess what a package needs, and in most cases it tries the worst choice of all if there is a choice to be made.

In this way, pbuilder tries to ensure that packages tested against pbuilder will build properly in most Debian installations, hopefully resulting in a good overall Debian source-buildability.

The goal of making Debian buildable from source is somewhat accomplished, and has seen good progress. In the past age of Debian 3.0, there were many problems when building from source. More recent versions of Debian is much better.

pbuilder create will create a base chroot image tar-ball (base.tgz). All other commands will operate on the resulting base.tgz If the Debian release to be created within chroot is not going to be "sid" (which is the default), the distribution code-name needs to be specified with the --distribution command-line option.

debootstrap ^[1] is used to create the bare minimum Debian installation, and then build-essential packages are installed on top of the minimum installation using apt-get inside the chroot.

For fuller documentation of command-line options, see the pbuilder.8 manual page. Some configuration will be required for /etc/pbuilderrc for the mirror site ^[2] to use, and proxy configuration may be required to allow access through HTTP. See the pbuilderrc.5 manual page for details.

pbuilder update will update the base.tgz. It will extract the chroot, invoke apt-get update and apt-get dist-upgrade inside the chroot, and then recreate the base.tgz (the base tar-ball.)

It is possible to switch the distribution which the base.tgz is targeted at at this point. Specify --distribution sid --override-config to change the distribution to sid. ^[3]

For fuller documentation of command-line options, see the pbuilder.8 manual page

To build a package inside the chroot, invoke pbuilder build whatever.dsc. pbuilder will extract the base.tgz to a temporary working directory, enter the directory with chroot, satisfy the build-dependencies inside chroot, and build the package. The built packages will be moved to a directory specified with the --buildresult command-line option.

The --basetgz option can be used to specify which base.tgz to use.

pbuilder will extract a fresh base chroot image from base.tgz. (base.tgz is created with pbuilder create, and updated with pbuilder update). The chroot is populated with build-dependencies by parsing debian/control and invoking apt-get.

For fuller documentation of command-line options, see the pbuilder.8 manual page

pdebuild is a little wrapper script that does the most frequent of all tasks. A Debian Developer may try to do debuild, and build a package, inside a Debian source directory. pdebuild will allow similar control, and allow package to be built inside the chroot, to check that the current source tree will build happily inside the chroot.

pdebuild calls dpkg-source to build the source packages, and then invokes pbuilder on the resulting source package. However, unlike debuild, the resulting deb files will be found in the --buildresult directory.

See the pdebuild.1 manual page for more details.

There is a slightly different mode of operation available in pdebuild since version 0.97. pdebuild usually runs debian/rules clean outside of the chroot; however, it is possible to change the behavior to run it inside the chroot with the --use-pdebuild-internal. It will try to bind mount the working directory inside chroot, and run dpkg-buildpackage inside. It has the following characteristics, and is not yet the default mode of operation.

It is possible to specify all settings by command-line options. However, for typing convenience, it is possible to use a configuration file.

/etc/pbuilderrc and ${HOME}/.pbuilderrc are read in when pbuilder is invoked. The possible options are documented in the pbuilderrc.5 manual page.

It is useful to use --configfile option to load up a preset configuration file when switching between configuration files for different distributions.

Please note ${HOME}/.pbuilderrc supersede system settings. For example, if you are upgrading from sarge to etch, you may need to adjust some part of your local setting just like new /usr/share/pbuilder/pbuilderrc, e.g., "unset DEBOOTSTRAPOPTS", to cope with the use of cdebootstrap. The same care is needed if you edited old system setting in /etc/pbuilderrc.

pbuilder requires full root privilege when it is satisfying the build-dependencies, but most packages do not need root privilege to build, or even refused to build when they are built as root. pbuilder can create a user which is only used inside pbuilder and use that user id when building, and use the fakeroot command when root privilege is required.

BUILDUSERID configuration option should be set to a value for a user id that does not already exist on the system, so that it is more difficult for packages that are being built with pbuilder to affect the environment outside the chroot. When BUILDUSERNAME configuration option is also set, pbuilder will use the specified user name and fakeroot for building packages, instead of running as root inside chroot.

Even when using the fakerooting method, pbuilder will run with root privilege when it is required. For example, when installing packages to the chroot, pbuilder will run under root privilege.

To be able to invoke pbuilder without being root, you need to use user-mode-linux, as explained in Chapter 3, Using User-mode-linux with pbuilder.

pbuilder can be used for back-porting software from the latest Debian distribution to the older stable distribution, by using a chroot that contains an image of the older distribution, and building packages inside the chroot. There are several points to consider, and due to the following reasons, automatic back-porting is usually not possible, and manual interaction is required:

pbuilder can be automated, because its operations are non-interactive. It is possible to run pbuilder through multiple packages non-interactively. Several such scripts are known to exist. Junichi Uekawa has been running such a script since 2001, and has been filing bugs on packages that fail the test of pbuilder. There were several problems with auto-building:

Most of the initial bugs have been resolved in the pbuilder sweep done around 2002, but these transitional problems which affect a large portion of Debian Archive do arise from time to time. Regression tests have their values.

A script that was used by Junichi Uekawa in the initial run is now included in the pbuilder distribution, as pbuildd.sh. It is available in /usr/share/doc/pbuilder/examples/pbuildd/ and its configuration is in /etc/pbuilder/pbuildd-config.sh. It should be easy enough to set up for people who are used to pbuilder. It has been running for quite a while, and it should be possible to set the application up on your system also. This version of the code is not the most tested, but should function as a starter.

To set up pbuildd, there are some points to be aware of.

There are some people who use pbuilder to automatically back-port a subset of packages to the stable distribution.

I would like some information on how people are doing it, I would appreciate any feedback or information on how you are doing, or any examples.

pbuilder can be used for automated testing of packages. It has the feature of allowing hooks to be placed, and these hooks can try to install packages inside the chroot, or run them, or whatever else that can be done. Some known tests and ideas:

To use B92test-pkg script, first, add it to your hook directory. ^[4]. The test files are shell scripts placed in debian/pbuilder-test/NN_name (where NN is number) following run-parts standard^[5] for file names. After a successful build, packages are first tested for installation and removal, and then each test is ran inside the chroot. The current directory is the top directory of the source-code. This means you can expect to be able to use ./debian/ directory from inside your scripts.

Example scripts for use with pbuilder-test can be found in /usr/share/doc/pbuilder/examples/pbuilder-test

Most packages are compiled with gcc or g++ and using the default compiler version, which was gcc 2.95 for Debian GNU/Linux 3.0 (i386). However, Debian 3.0 was distributed with other compilers, under package names such as gcc-3.2 for gcc compiler version 3.2. It was therefore possible to try compiling packages against different compiler versions. pentium-builder provides an infrastructure for using a different compiler for building packages than the default gcc, by providing a wrapper script called gcc which calls the real gcc. To use pentium-builder in pbuilder, it is possible to set up the following in the configuration:

EXTRAPACKAGES="pentium-builder gcc-3.2 g++-3.2"
export DEBIAN_BUILDARCH=athlon
export DEBIAN_BUILDGCCVER=3.2

It will instruct pbuilder to install the pentium-builder package and also the GCC 3.2 compiler packages inside the chroot, and set the environment variables required for pentium-builder to function.

user-mode-linux isn't completely trivial to set up. It would probably be useful to acquaint yourself with it a bit before attempting to use rootstrap or pbuilder-user-mode-linux. For details, read /usr/share/doc/uml-utilities/README.Debian and the user-mode-linux documentation. (It's in a separate package, user-mode-linux-doc.)

user-mode-linux requires the user to be in the uml-net group in order to configure the network unless you are using slirp.

If you compile your own kernel, you may want to verify that you enable TUN/TAP support, and you might want to consider the SKAS patch.

rootstrap is a wrapper around debootstrap. It creates a Debian disk image for use with UML. To configure rootstrap, there are several requirements.

The following needs to happen:

pbuilder-user-mode-linux emulates most of pbuilder, but there are some differences.

To run pbuilder-user-mode-linux in parallel on a system, there are a few things to bear in mind.

It is possible to use pbuilder-user-mode-linux for other uses than just building Debian packages. pbuilder-user-mode-linux login will let a user use a shell inside the user-mode-linux pbuilder base image, and pbuilder-user-mode-linux execute will allow the user to execute a script inside the image.

You can use the script to install ssh and add a new user, so that it is possible to access inside the user-mode-linux through ssh.

Note that it is not possible to use a script from /tmp due to the way pbuilder-user-mode-linux mounts a tmpfs at /tmp.

The following example script may be useful in starting a sshd inside user-mode-linux.

#!/bin/bash

apt-get install -y ssh xbase-clients xterm
echo "enter root password"
passwd
cp /etc/ssh/sshd_config{,-}
sed 's/X11Forwarding.*/X11Forwarding yes/' /etc/ssh/sshd_config- > /etc/ssh/sshd_config

/etc/init.d/ssh restart
ifconfig
echo "Hit enter to finish"
read

It often happens that pbuilder cannot create the latest chroot. Try upgrading pbuilder and debootstrap. It is currently only possible to create software that handles the past. Future prediction is a feature which may be added later after we have become comfortable with the past.

There are people who occasionally back port debootstrap to stable versions; hunt for them.

When there are errors with the debootstrap phase, the debootstrap script needs to be fixed. pbuilder does not provide a way to work around debootstrap.

Because of the way pbuilder works, there are several directories which cannot be bind-mounted when running pbuilder. The directories include /tmp, /var/cache/pbuilder, and system directories such as /etc and /usr. The recommendation is to use directories under the user's home directory for bind-mounts.

It is possible to invoke a shell session after a build failure. Example hook script are provided as C10shell and C11screen scripts. C10shell script will start bash inside chroot, and C11screen script will start GNU screen inside the chroot.

It is sometimes necessary to modify the chroot environment. login will remove the contents of the chroot after logout. It is possible to invoke a shell using hook scripts. pbuilder update executes 'E' scripts, and a sample for invoking a shell is provided as C10shell.

$ mkdir ~/loginhooks
$ cp C10shell ~/loginhooks/E10shell
$ sudo pbuilder update --hookdir ~/loginhooks/E10shell

It is also possible to add --save-after-exec and/or --save-after-login options to the pbuilder login session to accomplish the goal. It is possible to add the --uml-login-nocow option to pbuilder-user-mode-linux login session as well.

It is possible to set

BUILDRESULTUID=$SUDO_UID

in pbuilderrc to set the proper BUILDRESULTUID when using sudo.

If you are setting $TMPDIR to an unusual value, of other than /tmp, you will find that some errors may occur inside the chroot, such as dpkg-source failing.

There are two options, you may install a hook to create that directory, or set

export TMPDIR=/tmp

in pbuilderrc. Take your pick.

An example script is provided as examples/D10tmp with pbuilder.

When working with multiple chroots, it would be nice to work with scripts that reduce the amount of typing. An example script pbuilder-distribution.sh is provided as an example. Invoking the script as pbuilder-etch will invoke pbuilder with a etch chroot.

This section^[6] describes briefly a way to setup and use multiple pbuilder setups by creating a pbuilderrc configuration in your home path ($HOME/.pbuilderrc) and using the variable "DIST" when running pbuilder or pdebuild.

First, setup $HOME/.pbuilderrc to look like:

if [ -n "${DIST}" ]; then
        BASETGZ="`dirname $BASETGZ`/$DIST-base.tgz"
        DISTRIBUTION="$DIST"
        BUILDRESULT="/var/cache/pbuilder/$DIST/result/"
        APTCACHE="/var/cache/pbuilder/$DIST/aptcache/"
fi

Then, whenever you wish to use pbuilder for a particular distro, assign a value to "DIST" that is one of the distros available for Debian or any Debian based distro you happen to be running (i.e. whatever is found under /usr/lib/debootstrap/scripts).

Here's some examples on running pbuilder or pdebuild:

DIST=gutsy sudo pbuilder create

DIST=sid sudo pbuilder create --mirror http://http.us.debian.org/debian

DIST=gutsy sudo pbuilder create \
        --othermirror "deb http://archive.ubuntu.com/ubuntu gutsy universe \
        multiverse"

DIST=gutsy sudo pbuilder update

DIST=sid sudo pbuilder update --override-config --mirror \
http://http.us.debian.org/debian \
--othermirror "deb http://http.us.debian.org/debian sid contrib non-free"

DIST=gutsy pdebuild

If you have some very specialized requirements on your apt setup inside pbuilder, it is possible to specify that through the --othermirror option. Try something like: --othermirror "deb http://local/mirror stable main|deb-src http://local/source/repository ./"

To use the local file system instead of HTTP, it is necessary to do bind-mounting. --bindmounts is a command-line option useful for such cases.

It might be convenient to use your built packages from inside the chroot. It is possible to automate the task with the following configuration. First, set up pbuilderrc to bindmount your build results directory.

BINDMOUNTS="/var/cache/pbuilder/result"

Then, add the following hook

# cat /var/cache/pbuilder/hooks/D70results
#!/bin/sh
cd /var/cache/pbuilder/result/
/usr/bin/dpkg-scanpackages . /dev/null > /var/cache/pbuilder/result/Packages
/usr/bin/apt-get update

This way, you can use deb file:/var/cache/pbuilder/result

You can use hook scripts for this. D scripts are run before satisfying build-dependency.

This snippet comes from Ondrej Sury.

To make distinguishing bash prompts inside pbuilder easier, it is possible to set environment variables such as PS1 inside pbuilderrc

With versions of bash more recent than 2.05b-2-15, the value of the debian_chroot variable, if set, is included in the value of PS1 (the Bash prompt) inside the chroot. In prior versions of bash,^[7] setting PS1 in pbuilderrc worked.

example of debian_chroot

	export debian_chroot="pbuild$$"

example of PS1

	export PS1="pbuild chroot 32165 # "

Bash prompts will help you remember that you are inside a chroot. There are other cases where you may want other signs of being inside a chroot. Check out the examples/F90chrootmemo hook script. It will create a file called /CHROOT inside your chroot.

For the help of low-bandwidth systems, it is possible to use /var/cache/apt/archives as the package cache. Just specify it instead of the default /var/cache/pbuilder/aptcache.

It is however not possible to do so currently with the user-mode-linux version of pbuilder, because /var/cache/apt/archives is usually only writable by root.

Use of dedicated tools such as apt-proxy is recommended, since caching of packages would benefit the system outside the scope of pbuilder.

Currently stable back port of pbuilder is available at backports.org.

You might see a lot of warning messages when running pbuilder.

	dpkg-genchanges: warning: no utmp entry available and LOGNAME not defined; using uid of process (1234)

It is currently safe to ignore this warning message. Please report back if you find any problem with having LOGNAME unset. Setting LOGNAME caused a few problems when invoking chroot. For example, dpkg requires getpwnam to succeed inside chroot, which means LOGNAME and the related user information have to be set up inside chroot.

pbuilder does not currently allow Build-Conflicts against essential packages. It should be obvious that essential packages should not be removed from a working Debian system, and a source package should not try to force removal of such packages on people building the package.

By default, pbuilder uses hard links to manage the pbuilder package cache. It is not possible to make hard links across different devices; and thus this error will occur, depending on your set up. If this happens, set

APTCACHEHARDLINK=no

in your pbuilderrc file. Note that packages in APTCACHE will be copied into chroot local cache, so plan for enough space on BUILDPLACE device.

It is possible to use fakechroot instead of being root to run pbuilder; however, several things make this impractical. fakechroot overrides library loads and tries to override default libc functions when providing the functionality of virtual chroot. However, some binaries do no use libc to function, or override the overriding provided by fakechroot. One example is ldd. Inside fakechroot, ldd will check the library dependency outside of the chroot, which is not the expected behavior.

To work around the problem, debootstrap has a --variant fakechroot option. Use that, so that ldd and ldconfig are overridden.

Make sure you have set your LD_PRELOAD path correctly, as described in the fakechroot manpage.

To use debconf inside pbuilder, setting DEBIAN_FRONTEND to “readline” in pbuilderrc should work. Setting it to “dialog” should also work, but make sure whiptail or dialog is installed inside the chroot.

If you see messages such as this when building a chroot, you are mounting the file system with the nodev option.

	/var/lib/dpkg/info/base-files.postinst: /dev/null: Permission denied

You will also have problems if you mount the file system with the noexec option, or nosuid. Make sure you do not have these flags set when mounting the file system for /var/cache/pbuilder or $BUILDPLACE.

This is not a problem when using user-mode-linux.

See 316135 for example.

pbuilder is often slow. The slowest part of pbuilder is extracting the tar.gz every time pbuilder is invoked. That can be avoided by using pbuilder-user-mode-linux. pbuilder-user-mode-linux uses COW file system, and thus does not need to clean up and recreate the root file system.

pbuilder-user-mode-linux is slower in executing the actual build system, due to the usual user-mode-linux overhead for system calls. It is more friendly to the hard drive.

pbuilder with cowdancer is also an alternative that improves speed of pbuilder startup.

To sign a package marking for sponsorship, it is possible to use --auto-debsign and --debsign-k options of pdebuild.

	pdebuild  --auto-debsign  --debsign-k XXXXXXXX

When running pdebuild, pbuilder will run dpkg-buildpackage to create a Debian source package to pass it on to pbuilder. File named XXXX_YYY_source.changes is what remains from that process. It is harmless unless you try to upload it to the Debian archive.

This behavior is different when running through --use-pdebuild-internal

amd64 architectures are capable of running binaries in i386 mode. It is possible to use pbuilder to run packages, using linux32 and debootstrap --arch option. Specifically, a command-line option like the following will work.

pbuilder create --distribution sid --debootstrapopts --arch --debootstrapopts i386 \
  --basetgz /var/cache/pbuilder/base-i386.tgz --mirror http://ftp.jp.debian.org/debian
linux32 pbuilder build --basetgz /var/cache/pbuilder/base-i386.tgz

To use ccache with pbuilder, use the following configuration. Note that the directory used for CCACHE_DIR needs to exist, and be writable by user within chroot. The default user within chroot is

uid=1234

.

Example is provided as pbuilderrc.ccache.

export CCACHE_DIR="/var/cache/pbuilder/ccache"
export PATH="/usr/lib/ccache:${PATH}"

EXTRAPACKAGES=ccache
BINDMOUNTS="${CCACHE_DIR}"

This entry created thanks to a blog posting.

To improve speed of operation, it is possible to use tmpfs for pbuilder build location. Mount tmpfs to /var/cache/pbuilder/build, and set

APTCACHEHARDLINK=no

.

pdebuild command can be used with svn-buildpackage --svn-builder command-line option. ^[8]

alias svn-cowbuilder="svn-buildpackage --svn-builder='pdebuild --pbuilder cowbuilder"

To report bugs, it would be important to have a log of what's going wrong. Most of the time, adding a --debug option and re-running the session should do the trick. Please send the log of such session along with your problem to ease the debugging process.

There is a mailing list for pbuilder on alioth (pbuilder-maint@lists.alioth.debian.org). You can subscribe through the alioth web interface. http://alioth.debian.org/mail/?group_id=30778.

For coordination and communication, IRC channel #pbuilder on irc.oftc.net is used. Please log your intent there when you are going to start doing some changes and committing some change.

This section tries to document current development practices and how things generally operate in development.

pbuilder is co-maintained with resources provided by Alioth. There is an Alioth project page at http://alioth.debian.org/projects/pbuilder. Home page is also available, at http://alioth.debian.org/projects/pbuilder which shows this text. git repository is available through http, git, or (if you have an account on alioth, ) ssh.

git-clone git://git.debian.org/git/pbuilder/pbuilder.git
git-clone http://git.debian.org/git/pbuilder/pbuilder.git
git-clone ssh://git.debian.org/git/pbuilder/pbuilder.git

Test-suites are available in ./testsuite/ directory. Changes are expected not to break the test-suites. ./run-test.sh is a basic test-suite, which puts a summary in run-test.log, and run-test-cdebootstrap.log. ./run-test-regression.sh is a regression test-suite, which puts the result in run-test-regression.log. Currently, run-test.sh is ran automatically daily to ensure that pbuilder is working.

When making changes, changes should be documented in the Git commit log. git-dch will generate debian/changelog from the commit log. Make the first line of your commit log meaningful, and add any bug-closing information available.

A TODO file is available in debian/TODO. It's mostly not well-maintained, but hopefully it will be more up-to-date when people start using it. emacs todoo-mode is used in editing the file.

When releasing a new version of pbuilder, the version is tagged with the git tag X.XXX (version number). This is done with ./git-tag.sh script available in the source tree.

There are cases when some small experimenting is required, and you do not want to damage the main system, like when installing experimental library packages, or compiling with experimental compilers. For such cases, the pbuilder login command is available.

pbuilder login is a debugging feature for pbuilder itself, but it also allows users to have a temporary chroot.

Note that the chroot is cleaned after logging out of the shell, and mounting file systems inside it is considered harmful.

To facilitate using pbuilder for other uses, pbuilder execute is available. pbuilder execute will take a script specified in the command-line argument, and invoke the script inside the chroot.

The script can be useful for sequences of operations such as installing ssh and adding a new user inside the chroot.

LVM2 has a useful snapshot function that features Copy-on-write images. That could be used for pbuilder just as it can be used for the user-mode-linux pbuilder port. lvmpbuilder script in the examples directory implements such port. The scripts and documentation can be found under /usr/share/doc/pbuilder/examples/lvmpbuilder/

cowdancer allows copy-on-write semantics on file system using hard links and hard-link-breaking-on-write tricks. pbuilder using cowdancer seems to be much faster and it is one ideal point for improvement. cowbuilder, a wrapper for pbuilder for using cowdancer is available from cowdancer package since 0.14

Example command-lines for cowbuilder look like the following.

# cowbuilder --create --distribution sid
# cowbuilder --update --distribution sid
# cowbuilder --build XXX.dsc

It is also possible to use cowdancer with pdebuild command. Specify with command-line option --pbuilder or set it in PDEBUILD_PBUILDER configuration option.

$ pdebuild --pbuilder cowbuilder

$ cat sources.list
# sarge-backports
deb     http://www.backports.org/debian sarge-backports main
deb-src http://www.backports.org/debian sarge-backports main

# sarge
deb     http://ftp.de.debian.org/debian sarge main
deb-src http://ftp.de.debian.org/debian sarge main

$ cat preferences
Explanation: use cowdancer from backports
Package: cowdancer
Pin: release a=sarge-backports
Pin-Priority: 950

The --no-targz option of pbuilder will allow usage of pbuilder in a different way from conventional usage. It will try to use an existing chroot, and will not try to clean up after working on it. It is an operation mode more like sbuild.

It should be possible to create base chroot images for dchroot with the following commands:

# pbuilder create --distribution etch --no-targz --basetgz /chroot/etch
# pbuilder create --distribution lenny --no-targz --basetgz /chroot/lenny
# pbuilder create --distribution sid --no-targz --basetgz /chroot/sid

It is possible to use pbuilder in a vserver environment. This requires either vserver-patches in version 2.1.1-rc14 or higher, or a Linux kernel version 2.6.16 or higher.

To use pbuilder in a vserver, you need to set the secure_mount CAPS in the ccapabilities of this vserver.

This document was started on 28 Dec 2002 by Junichi Uekawa, trying to document what is known about pbuilder.

This documentation is available from the pbuilder source tar-ball, and from the git repository of pbuilder (web-based access is possible). A copy of this documentation can be found on the Alioth project page for pbuilder. There is also a PDF version. The homepage for pbuilder is http://pbuilder.alioth.debian.org/ hosted by alioth project.

Documentation is written using DocBook XML, with emacs PSGML mode, and using wysidocbookxml for live previewing.

The following is a most possibly inaccurate account of how pbuilder came to happen, and other attempts to make something like pbuilder happen. This part of the document was originally in the AUTHORS file, to give credit to those who existed before pbuilder.