Sync calendars and address books between the Linux console and Android
Posted by Eric Scheibler at October 14, 2014
Calendars and address books contain very sensible personal data. So it’s not advisable to share them with companies like Google or Apple. But with more than one client system in use, it is necessary to run an own calendar server to synchronize that data between the clients.
The first part of this article describes the installation and configuration of a calendar and contacts server. After that follows the presentation of calendar and address book programs for the Linux console and the Android operating system.
Table of contents
- 1. Calendar and contacts server
- 2. Linux console
- 3. Android
Calendar and contacts server
There are quite a number of calendar and contacts servers, which implement the CalDAV and CardDAV protocol. Among others you can choose between OwnCloud, Baïkal and Darwin Calendar Server. I used the last one for a few years but switched to Baikal recently. It’s relatively easy to setup and it comes with a web interface to create calendars and address books.
You need a web server to run Baikal. I use LightTPD. You can find installation instructions here. For this tutorial I assume, that the web server is configured to accept encrypted connections. This article provides information about the creation of a valid SSL certificate.
After the web server is running, install SQLite:
sudo aptitude install sqlite php5-sqlite
Then download the latest release from the Baikal website and extract it into the content folder of the web server:
cd /tmp wget http://baikal-server.com/get/baikal-flat-0.2.7.zip unzip baikal-flat-0.2.7.zip sudo mv baikal-flat /var/www/baikal sudo chown -R www-data:www-data /var/www/baikal
Before you can start the setup process, you have to create the following file:
sudo touch /var/www/baikal/Specific/ENABLE_INSTALL sudo chown www-data:www-data /var/www/baikal/Specific/ENABLE_INSTALL
Then visit https://example.org/baikal/admin/ and follow the setup instructions. After the setup is complete, log in as Admin and create a new user and as many address books and calendars as you want.
If the server runs at your home, you should register a DynDNS domain. So you can access it from outside too. To achieve that, I use the free service from no-ip.com.
If you can’t run an own server, then choose a more privacy-friendly provider, for example: Fastmail or Posteo.
This section contains the instructions for the Linux console. I’ve used Debian but the applications should run under other Linux distributions too. First come some information about system prerequisites. The second subsection describes the synchronization of the address books and calendars between the server and the local machine. Afterwards follow the presentation of the address book and calendar applications. All programs are intended for the console. Therefore no graphical user interface is required.
Before we can start, we have to fulfill the following requirements:
sudo aptitude install git python-setuptools sudo easy_install pip sudo pip install virtualenv
Pip is a package manager for python. For its installation the Setuptools program is required and must be installed first. Virtualenv is used to create a virtual Python environment for every installed program. So it’s possible to use the same Python module in different versions without conflicts. Furthermore it helps to keep your local Python installation clean. To uninstall a previously installed program, you only have to delete it’s virtual environment folder.
First we need a directory for the virtual environments:
Then create a sub folder for the binaries:
Finally add the created binary folder to the shell’s path variable. Here it’s shown for the ZSH:
vim ~/.zshrc [...] # add ~/.virtualenvs/bin to the path variable if [ -d "$HOME/.virtualenvs/bin" ] ; then PATH="$HOME/.virtualenvs/bin:$PATH" fi [...]
In the following sections we create a virtual environment for every installed program and set a symbolic link from the program binary to the created bin folder. If you want to get additional information about virtualenv, you can find them for example here.
Vdirsyncer is used to synchronize calendar and contact data between the server and the local machine. Install it with the following commands:
# change to the virtualenv folder cd ~/.virtualenvs # create a new virtual environment with the name vdirsyncer virtualenv vdirsyncer # install vdirsyncer with the local pip version ./vdirsyncer/bin/pip install vdirsyncer # create symlink ln -s ~/.virtualenvs/vdirsyncer/bin/vdirsyncer ~/.virtualenvs/bin
Then download the example configuration file:
mkdir ~/.config/vdirsyncer/ wget -O ~/.config/vdirsyncer/config https://github.com/untitaker/vdirsyncer/raw/master/example.cfg
And configure the program:
The config file contents should look like this:
# An example configuration for vdirsyncer. [general] # A folder where vdirsyncer can store some metadata about each pair. status_path = ~/.config/vdirsyncer/status/ # CARDDAV [pair contacts] a = contacts_local b = contacts_remote collections = ["family", "friends"] conflict_resolution = a wins [storage contacts_local] type = filesystem path = ~/.contacts/ fileext = .vcf [storage contacts_remote] type = carddav url = https://example.org/baikal/card.php/addressbooks/YOURUSERNAME/ auth = digest verify = /etc/ssl/local/example.org.pem username = YOURUSERNAME password = YOURPASSWORD # CALDAV [pair calendar] a = calendar_local b = calendar_remote collections = ["calendar", "holidays", "birthdays"] conflict_resolution = a wins [storage calendar_local] type = filesystem path = ~/.calendars/ fileext = .ics [storage calendar_remote] type = caldav url = https://example.org/baikal/cal.php/calendars/YOURUSERNAME/ auth = digest verify = /etc/ssl/local/example.org.pem username = YOURUSERNAME password = YOURPASSWORD
The CalDAV section of the vdirsyncer config contains the default calendar and two additional calendars (holidays and birthdays). The CardDAV section contains two address books. Change that to your needs. Furthermore you have to replace the host name and user name in the URL’s. A different calendar server could also require a slightly changed URL schema.
If you use a self signed SSL certificate to secure the connection to your calendar server, you have to import it’s public key into the client system. Therefore create the folder /etc/ssl/local at your Linux client and copy the public key file into that directory.
After configuration, run the synchronization process by typing:
Lastly create a cron job to sync your contacts and calendars every hour:
crontab -e [...] 0 * * * * $HOME/.virtualenvs/bin/vdirsyncer sync
You can find further information about the program at vdirsyncer’s project page.
Khard is an address book client for the Linux console. It lists, creates, modifies and removes vCard-formated contacts data at your local machine. Furthermore it serves as an external address book for the mail clients Mutt and Alot and the SIP client twinkle.
Installation of khard:
cd ~/.virtualenvs virtualenv khard ./khard/bin/pip install khard ln -s ~/.virtualenvs/khard/bin/khard ~/.virtualenvs/bin
Next follows the download and modification of khard’s example configuration file:
mkdir ~/.config/khard/ wget -O ~/.config/khard/khard.conf https://github.com/scheibler/khard/raw/master/khard.conf.example
The “addressbooks” section contains the paths to all address books and calendars, which are specified in the vdirsyncer configuration. Furthermore you must set the path to your favorite text editor. It’s required to create new contacts and modify existing ones.
After you are finished you can list all existing contacts with the following command:
Without any parameters, the contacts of all address books are listed. You can filter only certain address books like this:
khard list -a family,friends
You also can search for someone:
khard list -s max
The overview table only shows the first phone number and email address. To get all contact’s details, type:
The next command creates a new contact:
khard new -a friends
The form for the new contact opens in your favorite text editor. After you have filled the fields and saved the document, the new contact is created. To cancel, leave the first name, last name and organization fields blank. If you want to modify the created contact later, type:
Or remove it with:
After you’re finished, you must run the synchronization process:
It’s possible to use khard as an external address book for Mutt and Twinkle too. For further information about that, please have a look into khard’s Readme file.
Khal is a calendar application for the Linux console. It lists, creates, modifies and removes CalDAV-formated calendar entries at your local machine.
cd ~/.virtualenvs virtualenv khal ./khal/bin/pip install khal ln -s ~/.virtualenvs/khal/bin/khal ~/.virtualenvs/bin
Next copy and adapt its example configuration file (should be similar to the khard config):
mkdir ~/.config/khal wget -O ~/.config/khal/khal.conf https://github.com/geier/khal/raw/master/khal.conf.sample
After the successful configuration you can list all events for the next week like this:
khal agenda --days=7
Or only show appointments of one calendar:
khal agenda --days=7 -a holidays
Create a new single event at November 9th, which lasts from 14:00 till 16:00:
khal new -a calendar -l "optional location" 09.11. 14:00 16:00 Title of appointment
Or create an all-day appointment, which recurs every year:
khal new -a birthdays -r yearly 19.02. Max birthday
Further information about khal can be found here.
At some web sites you can subscribe to CalDAV formated calendars or download them. For example you can add these subscriptions to your Android phone. But then, they aren’t synchronized with your own calendar server and so they are not available at your Linux machine. This section describes, how to download a whole calendar and add it to your own calendar server.
At www.ifeiertage.de you can download a calendar with all German holidays. After downloading you get a single text file with all appointments. But the calendar server only supports .ics files with a single one inside. Therefore you have to split the text file into multiple appointment files. This can be done with the python script split_vcalendar.py:
cd ~/.local/bin wget http://eric-scheibler.de/downloads/split_vcalendar.py chmod +x split_vcalendar.py
Create a new calendar at your calendar server and enter it into your vdirsyncer config file. Then download and split the calendar file and move the .ics files into the calendar folder. In this example I’ve downloaded and processed the holidays for the German state Saxony:
cd /tmp wget -O feiertage.ics http://www.ifeiertage.de/calendar.php\?bl\=sn\&o\=o1\&o2\=sonder\&o4\=zeitum\&t\=dnl split_vcalendar.py feiertage.ics feiertage mv feiertage/*.ics ~/.calendars/feiertage/ vdirsyncer sync
Android has no build in support for CalDAV and CardDAV. Therefore you have to install the apps CalDAV-Sync and CardDav-Sync from the play store. Each application costs about 2 Euros. CardDav-Sync is also available for free but supports slightly less features. Both apps are implemented as sync adapters and therefore integrate seamlessly into the default contacts and calendar app of the Android os. For Android 4.1 and 4.4 the two additional apps JB Workaround CalDAV-Sync and JB Workaround CardDAV-Sync are required too. They only fix an Android bug, which otherwise would cause the deletion of the newly created CalDAV and CardDAV accounts after a software update.
Now we have to import the SSL certificate of the calendar server. To accomplish this, log into the server and convert the certificate to the binary format:
openssl x509 -in /etc/ssl/local/example.org.pem -outform DER -out /tmp/example.org.crt
Then copy it to the sd card of your smartphone, for example with adb:
adb push /tmp/example.org.crt /sdcard/
Lastly import the certificate by opening the android settings and navigate to “security”. Then choose “Install from SD card” and follow the import dialog.
Now you can create a new CalDAV account under Settings -> Add account -> CalDAV -> CalDAV. In the next dialog you must specify the server address in the format https://example.org/baikal/cal.php, the user name and the corresponding password. SSL is already activated. After the connection to the server is astablished, choose the calendars to synchronize and enter your email address. You also can deactivate the two-way synchronization, so that data is only synchronized from the server to the client. This avoids conflicts during synchronization but is not recommended if you also want to create new events on your smartphone.
Later on you can open the synchronization settings via Settings -> CalDAV -> Edit account settings or by starting the CalDAV-Sync App. Among others you can adapt the list of calendars to synchronize and the synchronization interval.
Now the configuration process is complete and you can view the synchronized apointments in the default calendar app. If you use the screen reader Talkback, I recommend to switch the layout of the calendar app to “Agenda”. To do so, open the combo box, which is located in the top-left corner of the calendar app and choose its last entry.
The configuration of the CardDAV synchronization is a similar process. You can create a new CardDAV account under Settings -> Add account -> CardDAV -> CardDAV. This time the server address is https://example.org/baikal/card.php. Beside that, use the same data like bfore. Then choose an address book and enter the email address again. The two-way synchronization for CardDAV entries is still marked as beta and therefore not activated by default. But if you also want to create new contacts on your smartphone, you should give it a try and activate it. I’ve encountered no problems so far.
If you want to add another address book, you have to perform the whole CardDAV account creation process again. Afterwards you can find your contacts data in the default contacts app. There you also can add new contacts and edit the existing ones.
Before you restart your phone, you have to launch the two Workaround-apps. This only must be done once.