Installation
The VarFish installation for developers should be set up differently from the installation for production use.
The reason being is that the installation for production use runs completely in a Docker environment. All containers are assigned to a Docker network that the host by default has no access to, except for the reverse proxy that gives access to the VarFish webinterface.
The developers installation is intended not to carry the full VarFish database such that it is light-weight and fits on a laptop. We advise to install the services not running in a Docker container.
Please find the instructions for the Windows installation at the end of the page.
Install Postgres
Follow the instructions for your operating system to install Postgres. Make sure that the version is 12 (11, 13 and 14 would also work). Ubuntu 20 already includes postgresql 12. In case of older Ubuntu versions, this would be:
$ sudo apt install postgresql-12
Adapt the postgres configuration file, for postgres 14 this would be:
sudo sed -i -e ‘s/.*max_locks_per_transaction.*/max_locks_per_transaction = 1024 # min 10/’ /etc/postgresql/14/main/postgresql.conf
Install Redis
Redis is the broker that celery uses to manage the queues. Follow the instructions for your operating system to install Redis. For Ubuntu, this would be:
$ sudo apt install redis-server
Install miniconda
miniconda helps to set up encapsulated Python environments. This step is optional. You can also use pipenv, but to our experience, resolving the dependencies in pipenv is terribly slow.
$ wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
$ bash Miniconda3-latest-Linux-x86_64.sh -b -p ~/miniconda3
$ source ~/miniconda3/bin/activate
$ conda init
$ conda create -n varfish python=3.8 pip
$ conda activate varfish
Clone git repository
Clone the VarFish Server repository and switch into the checkout.
$ git clone https://github.com/bihealth/varfish-server
$ cd varfish-server
Install Python Requirements
Some required packages have dependencies that are usually not preinstalled. Therefore, run
$ sudo bash utility/install_os_dependencies.sh
Now, with the conda/Python environment activated, install all the requirements.
$ for i in requirements/*; do pip install -r $i; done
Setup Database
Use the tool provided in utility/ to set up the database. The name for the
database should be varfish (create new user: yes, name: varfish, password: varfish).
$ bash utility/setup_database.sh
Setup vue.js
Use the tool provided in utility/ to set up vue.js.
$ sudo bash utility/install_vue_dev.sh
Open an additional terminal and switch into the vue directory. Then install the Clinvar Exprot vue app.
$ cd clinvar_export/vueapp
$ npm install
When finished, keep this terminal open to run the vue app.
$ npm run serve
Setup VarFish
First, create a .env file with the following content.
export DATABASE_URL="postgres://varfish:varfish@127.0.0.1/varfish"
export CELERY_BROKER_URL=redis://localhost:6379/0
export PROJECTROLES_ADMIN_OWNER=root
export DJANGO_SETTINGS_MODULE=config.settings.local
If you wish to enable structural variants, add the following line.
export VARFISH_ENABLE_SVS=1
To create the tables in the VarFish database, run the migrate command.
This step can take a few minutes.
$ python manage.py migrate
Once done, create a superuser for your VarFish instance. By default, the VarFish root user is named root (the
setting can be changed in the .env file with the PROJECTROLES_ADMIN_OWNER variable).
$ python manage.py createsuperuser
Last, download the icon sets for VarFish and make scripts, stylesheets and icons available.
$ python manage.py geticons -c bi cil fa-regular fa-solid gridicons octicon
$ python manage.py collectstatic
When done, open two terminals and start the VarFish server and the celery server.
terminal1$ make serve
terminal2$ make celery
Installation (Windows)
The setup was done on a recent version of Windows 10 with Windows Subsystem for Linux Version 2 (WSL2).
Installation WSL2
Following [this tutorial](https://www.omgubuntu.co.uk/how-to-install-wsl2-on-windows-10) to install WSL2.
Note that the whole thing appears to be a bit convoluted, you start out with wsl.exe –install
Then you can install latest LTS Ubuntu 22.04 with the Microsoft Store
Once complete, you probably end up with a WSL 1 (one!) that you can conver to version 2 (two!) with wsl –set-version Ubuntu-22.04 2 or similar.
WSL2 has some advantages including running a full Linux kernel but is even slower in I/O to the NTFS Windows mount.
Everything that you do will be inside the WSL image.
Install Dependencies
$ sudo apt install libsasl2-dev python3-dev libldap2-dev libssl-dev gcc make rsync
$ sudo apt install postgresql postgresql-server-dev-14 postgresql-client redis
$ sudo service postgresql start
$ sudo service postgresql status
$ sudo service redis-server start
$ sudo service redis-server status
$ sudo sed -i -e 's/.*max_locks_per_transaction.*/max_locks_per_transaction = 1024 # min 10/' /etc/postgresql/14/main/postgresql.conf
$ sudo service postgresql restart
Create a postgres user varfish with password varfish and a database.
$ sudo -u postgres createuser -s -r -d varfish -P
$ [enter varfish as password]
$ sudo -u postgres createdb --owner=varfish varfish
Create a miniconda3 installation with an environment.
$ mkdir -p Development Downloads
$ cd Downloads
$ wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
$ bash Miniconda3-latest-Linux-x86_64.sh -b -p ~/miniconda3
$ source ~/miniconda3/bin/activate
$ conda install -y mamba
$ mamba create -y -n varfish-server python==3.9 nodejs=12
$ conda activate varfish-server
Finally, checkout varfish-server and create a development .env file:
$ cd ~/Development
$ git clone git@github.com:bihealth/varfish-server.git
$ cat <<"EOF" >.env
export VARFISH_ENABLE_SPANR_SUBMISSION=1
export VARFISH_ENABLE_CADD_SUBMISSION=1
export VARFISH_ENABLE_SPANR_SUBMISSION=1
export VARFISH_ENABLE_SVS=1
export DJANGO_SETTINGS_MODULE=config.settings.local
export DJANGO_SECURE_SSL_REDIRECT=0
export DJANGO_SECRET_KEY="0Vabi8RKYcSgVTGhr23AlIFA5D1aXh25ZBvxXi9Tgu9UrrFdiolaQchS9k7CfqIMev7KoLV2RH84XxQDcDCmIoeyVmMmNUh7jE8N"
export DATABASE_URL="postgres://varfish:varfish@127.0.0.1/varfish"
export VARFISH_ENABLE_BEACON_SITE=1
export FIELD_ENCRYPTION_KEY=_XRAzgLd6NHj8G4q9FNV0p3Um9g4hy8BPBN-AL0JWO0=
EOF
$ make test-noselenium
Open WSL image in PyCharm
This has been tested with PyCharm Professional only.
You can simply open projects in the WSL, e.g., \wsl$Ubuntu-22.04home….
You can add the interpreter in the varfish-server miniconda3 environment to PyCharm which gives you access to.