Setting up and running Launchpad

Note

Ask for help right away if you run into problems.

This page explains how to set up and run Launchpad (for development) on your own machine, using a LXD-managed container to isolate it from the rest of your system.

Supported operating systems

For the host system, any reasonably modern Ubuntu release should work. Other Linux distributions that have LXD should work too, though we don’t test on them.

For the containers, these instructions should work with Ubuntu 16.04 LTS or later. We currently test on 16.04 and 18.04, with the aim of upgrading production to 18.04 soon. 20.04 is known not to work yet.

Launchpad requires at least Python 3.5 (i.e. newer than Ubuntu 14.04 LTS).

We’d like Launchpad to run on other operating systems, especially Debian GNU/Linux, so that more people can contribute to Launchpad development. If you’re interested in working on Launchpad portability, please let us know. Note that our focus is on getting Launchpad to build easily so more people can participate in Launchpad development. Running a stable production instance would be much harder than running a single-developer test instance, and we don’t recommend it. Unlike many open source projects, we’re not seeking to maximize the number of installations; our goal is to improve the instance we’re already running.

Why use LXD containers?

Launchpad development setup makes significant changes to your machine; it’s nice to be unaffected by those when you’re not doing such development. Using containers means that the version of Ubuntu on your host system doesn’t need to match Launchpad’s requirements. Multiple containers can be used to work around Launchpad’s limitations regarding concurrent test runs on a single machine.

LXD also has some nice snapshotting and ZFS capabilities, and is what other Launchpad developers use, so other people will be able to help you debug your setup if needed.

The installation script is written with the assumption that it will be running inside such a container. If you run it on your host system, it may well break your host system.

Create a LXD container

This assumes you already have LXD set up. If not, follow the instructions for getting it installed and configured on your network.

  1. If you haven’t done so already, run this script to set up LXD to let you use your home directory inside the container. Some developers prefer to only mount a subdirectory of their home directory in the container: to do that, replace $HOME with $HOME/src or similar.

#! /bin/sh

id=400000  # some large uid outside of typical range, and outside of already mapped ranges in /etc/sub{u,g}id
uid=$(id -u)
gid=$(id -g)
user=$(id -un)
group=$(id -gn)

# give lxc permission to map your user/group id through
sudo usermod --add-subuids ${uid}-${uid} --add-subgids ${gid}-${gid} root

# create a profile to control this
lxc profile create $user >/dev/null 2>&1

# configure profile
cat << EOF | lxc profile edit $user
name: $user
description: allow home dir mounting for $user
config:
  raw.idmap: |
    uid $uid $id
    gid $gid $id
  user.user-data: |
    #cloud-config
    runcmd:
      - "groupadd $group --gid $id"
      - "useradd $user --uid $id --gid $group --groups adm,sudo --shell /bin/bash"
      - "echo '$user ALL=(ALL) NOPASSWD:ALL' >/etc/sudoers.d/90-cloud-init-users"
      - "chmod 0440 /etc/sudoers.d/90-cloud-init-users"
devices:
  home:
    type: disk
    source: $HOME
    path: $HOME
EOF
  1. Create a container. This command creates a Ubuntu 16.04 unprivileged container using the profile created in the previous step.

lxc launch ubuntu:16.04 lpdev -p default -p $USER
  1. Find the container IP, either from lxc list or lxc info lpdev.

  2. In order to be able to ssh into the container, you need to add your public key to your local .ssh/authorized_keys configuration. Also make sure that both .ssh (700) and authorized_keys (600) have the correct permissions.

  3. Connect as follows. (The -A permits you to access Launchpad code hosting from within the container without needing to reenter passphrases.)

ssh -A $USER@IP_ADDRESS_FROM_LXC_LS

Getting Launchpad

Do all this inside the container you set up previously. Be aware that changes in your home directory inside the container will also be seen outside the container and vice versa.

If your Launchpad username differs from your local one, then put this in ~/.ssh/config in the container before doing anything else, replacing LPUSERNAME with your Launchpad username:

Host bazaar.launchpad.net
        User LPUSERNAME
Host git.launchpad.net
        User LPUSERNAME

Then:

$ mkdir ~/launchpad
$ cd ~/launchpad
$ curl https://git.launchpad.net/launchpad/plain/utilities/rocketfuel-setup >rocketfuel-setup

Read through the rocketfuel-setup script at this point and make sure you’re OK with what it’s going to do. (See Launchpad installation details if you want to know more.)

$ chmod a+x rocketfuel-setup
$ ./rocketfuel-setup

This will take a while – maybe a few hours to get everything, depending on your Internet connection.

Note that you will be prompted for your sudo password, and for a Launchpad login ID (that is, your username on launchpad.net). The sudo access is necessary to get Launchpad running on your box; the Launchpad login is not strictly necessary, and you can just hit Return there if you want. See below for an explanation.

Note that this will make changes to your Apache configuration if you already have an Apache server in your container. It will also add entries to /etc/hosts, and it will setup a PostgreSQL server in your container.

If you are running rocketfuel-setup to bring up a new container but your home directory already has a usable Launchpad tree, you can pass --no-workspace to only perform the system-wide setup.

Note that if rocketfuel-setup bails out with instructions to fix something, you just need to run it again and it should pick up where it left off.

$ sudo apt-get dist-upgrade

This is just to make doubly-sure everything from the Launchpad PPA gets installed.

$ ls
launchpad/    lp-sourcedeps/
$ cd launchpad

You are now in a newly-cloned Git repository, with one branch (‘master’), into whose working tree the other source dependencies have been symlinked. The source dependencies actually live in ../lp-sourcedeps.

Installing the pre-commit hook

If you intend to make any changes to Launchpad, you should also set up pre-commit now:

  1. Install pre-commit itself. If your host system is Ubuntu 20.10 or newer, then sudo apt install pre-commit is enough; otherwise, you can install it in your user account (pipx works well to keep it isolated; whatever you do, don’t run pip system-wide as root!). We require this to be installed separately rather than including it in Launchpad’s virtual environment because developers commonly run git commit outside the container used for running Launchpad.

  2. Install the pre-commit git hook by running pre-commit install in your newly-cloned launchpad repository.

Building

Before you can run Launchpad for the first time, you need to set up PostgreSQL.

Note

DO NOT run the database setup script below if you use PostgreSQL for anything other than Launchpad! Running the script will destroy any PostgreSQL databases on your system. See https://dev.launchpad.net/DatabaseSetup for details.

$ ./utilities/launchpad-database-setup $USER

(Please have read the previous comment before you run the above command!)

Finally, build the database schema (this may take several minutes):

$ make schema

Running

Now you should be able to start up Launchpad:

$ make run

This only runs the basic web application. Codehosting and Soyuz require additional steps.

For subsequent builds, you can just do make run right away. You don’t need to do make schema every time, and you should avoid it because it’s expensive and because it will clean out any data you might have put into your test instance (through the web UI or by running other scripts).

Setting up remote access

Unless the only thing you’re doing is running parts of the test suite, you probably want to make your new Launchpad instance accessible from other machines on the same local network, or in particular from the host system.

Amending the Apache configuration

Launchpad’s default development Apache config (/etc/apache2/sites-available/local-launchpad.conf) only listens on 127.0.0.88. This can be overridden with the LISTEN_ADDRESS environment variable when running make install. You probably want to make it listen on everything:

$ sudo make LISTEN_ADDRESS='*' install

Amending the hosts file

Launchpad makes extensive use of virtual hosts, so you’ll need to add entries to /etc/hosts on any machine from which you want to access the Launchpad instance. You’ll see the relevant hostnames in /etc/hosts on the machine running the instance - they need to be added to the remote machine, mapped to the server machine or container’s external IP address.

If some of those other machines run Windows, it may be helpful to know that the Windows equivalent of /etc/hosts is located at C:\WINDOWS\system32\drivers\etc\hosts. Note that Windows’ version has a line length limit, so you might have to split it across multiple lines or only include the hostnames that you need.

You should now be able to access https://launchpad.test/ in a web browser on a suitably configured remote computer. Accept the local self-signed certificate. You can log in as admin@canonical.com without a password. (This is only for development convenience, and assumes that you trust machines that can route to your LXD containers; of course a production deployment would need real authentication.). If you want to create more user accounts, see Creating additional user accounts in the development environment.

Accessing launchpad.test from a single host over SSH

As an alternative to the above, SSH provides a SOCKS proxy. By running that proxy on the target machine, you can view its Launchpad web site as if you were on that machine, without having to open non-SSH ports to a wider network. To do so:

$ ssh -D8110 target-machine

Then set your browser’s SOCKS proxy settings to use target-machine:8110.

Stopping

You can stop Launchpad by hitting Control-C in the terminal where you started it:

^C
[...shutting down Launchpad...]
$

Or you can be at a prompt in the same directory and run this:

$ make stop

Troubleshooting

“The LXC container is not getting an IPv4 address assigned and the network connectivity inside the container doesn’t work.”

On Ubuntu 21.10, ufw uses nftables by default, so if you are using Ubuntu 21.10 on the host and ufw is enabled with the default policy of blocking incoming and routed traffic, the rules added by LXD will not take effect, and hence LXD’s traffic will be dropped.

The fix is to add ufw allow rules to allow incoming and routed traffic on the bridge interface, like this (replacing lxdbr0 with the name of the bridge interface on your computer):

sudo ufw allow in on lxdbr0
sudo ufw route allow in on lxdbr0