Installation

The VarFish installation for developers should be set up differently from the installation for production use.

The reason being is that the installation for production use runs completely in a Docker environment. All containers are assigned to a Docker network that the host by default has no access to, except for the reverse proxy that gives access to the VarFish webinterface.

The developers installation is intended not to carry the full VarFish database such that it is light-weight and fits on a laptop. We advise to install the services not running in a Docker container.

Please find the instructions for the Windows installation at the end of the page.

Install Postgres

Follow the instructions for your operating system to install Postgres. Make sure that the version is 12 (11, 13 and 14 would also work). Ubuntu 20 already includes postgresql 12. In case of older Ubuntu versions, this would be:

$ sudo apt install postgresql-12

Adapt the postgres configuration file, for postgres 14 this would be:

sudo sed -i -e ‘s/.*max_locks_per_transaction.*/max_locks_per_transaction = 1024 # min 10/’ /etc/postgresql/14/main/postgresql.conf

Install Redis

Redis is the broker that celery uses to manage the queues. Follow the instructions for your operating system to install Redis. For Ubuntu, this would be:

$ sudo apt install redis-server

Install Python Pipenv

We use pipenv for managing dependencies. The advantage over pip is that also the versions of “dependencies of dependencies” will be tracked in a Pipfile.lock file. This allows for better reprocubility. Earlier developer’s instructions used conda but these instructions were updated.

Also, note that VarFish is developed using Python 3.10 only. To install Python 3.10, you can use pyenv. If you already have Python 3.10 (check with python --version then you can skip this step).

$ git clone https://github.com/pyenv/pyenv.git ~/.pyenv
$ echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.bashrc
$ echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bashrc
$ echo -e 'if command -v pyenv 1>/dev/null 2>&1; then\n  eval "$(pyenv init -)"\nfi' >> ~/.bashrc
$ exec $SHELL
$ pyenv install 3.10
$ pyenv global 3.10

Now, install the latest version of pip and pipenv:

$ pip install --upgrade pip pipenv

Clone git repository

Clone the VarFish Server repository and switch into the checkout.

$ git clone https://github.com/varfish-org/varfish-server
$ cd varfish-server

Install Python Requirements

Some required packages have dependencies that are usually not preinstalled. Therefore, run

$ sudo bash utils/install_os_dependencies.sh

Now, you can install the Python dependencies as follows:

$ pipenv install --dev

Afterwards, you can activate the virtual environment:

$ pipenv shell
# e.g.,
$ make black

Alternatively, you can also run commands directly in the virtual environment:

$ pipenv run make black

For greater verbosity, we will use pyenv run COMMAND below, but you can skip the pyenv run if you are in the pyenv shell.

Setup Database

Use the tool provided in utils/ to set up the database. The name for the database should be varfish (create new user: yes, name: varfish, password: varfish).

$ bash utils/setup_database.sh

Setup vue.js

Use the tool provided in utils/ to set up vue.js.

$ sudo bash utils/install_vue_dev.sh

Open an additional terminal and switch into the vue directory. Then install the Varfish vue app.

$ cd varfish/vueapp/
$ npm install

When finished, keep this terminal open to run the vue app.

$ npm run serve

Setup VarFish

First, create a .env file with the following content.

export DATABASE_URL="postgres://varfish:varfish@127.0.0.1/varfish"
export CELERY_BROKER_URL=redis://localhost:6379/0
export PROJECTROLES_ADMIN_OWNER=root
export DJANGO_SETTINGS_MODULE=config.settings.local

If you wish to enable structural variants, add the following line.

export VARFISH_ENABLE_SVS=1

To create the tables in the VarFish database, run the migrate command. This step can take a few minutes.

$ pipenv run python manage.py migrate

Once done, create a superuser for your VarFish instance. By default, the VarFish root user is named root (the setting can be changed in the .env file with the PROJECTROLES_ADMIN_OWNER variable).

$ pipenv run python manage.py createsuperuser

Last, download the icon sets for VarFish and make scripts, stylesheets and icons available.

$ pipenv run python manage.py geticons -c bi cil fa-regular fa-solid gridicons octicon
$ pipenv run python manage.py collectstatic

When done, open two terminals and start the VarFish server and the celery server.

terminal1$ pipenv run make serve
terminal2$ pipenv run make celery

Installation (Windows)

The setup was done on a recent version of Windows 10 with Windows Subsystem for Linux Version 2 (WSL2).

Installation WSL2

Following [this tutorial](https://www.omgubuntu.co.uk/how-to-install-wsl2-on-windows-10) to install WSL2.

  • Note that the whole thing appears to be a bit convoluted, you start out with wsl.exe –install

  • Then you can install latest LTS Ubuntu 22.04 with the Microsoft Store

  • Once complete, you probably end up with a WSL 1 (one!) that you can conver to version 2 (two!) with wsl –set-version Ubuntu-22.04 2 or similar.

  • WSL2 has some advantages including running a full Linux kernel but is even slower in I/O to the NTFS Windows mount.

  • Everything that you do will be inside the WSL image.

Install Dependencies

$ sudo apt install libsasl2-dev python3-dev libldap2-dev libssl-dev gcc make rsync
$ sudo apt install postgresql postgresql-server-dev-14 postgresql-client redis
$ sudo service postgresql start
$ sudo service postgresql status
$ sudo service redis-server start
$ sudo service redis-server status
$ sudo sed -i -e 's/.*max_locks_per_transaction.*/max_locks_per_transaction = 1024 # min 10/' /etc/postgresql/14/main/postgresql.conf
$ sudo service postgresql restart

Create a postgres user varfish with password varfish and a database.

$ sudo -u postgres createuser -s -r -d varfish -P
$ [enter varfish as password]
$ sudo -u postgres createdb --owner=varfish varfish

From here on, you can follow the instructions for the Linux installation, starting at ref:dev_install_python_pipenv.

Open WSL image in PyCharm

This has been tested with PyCharm Professional only.

  • You can simply open projects in the WSL, e.g., \wsl$Ubuntu-22.04home….

  • You can add the interpreter in the varfish-server miniconda3 environment to PyCharm which gives you access to.