Anaconda destroys Plasma

I just started my computer. Plasma did not start. I only saw the message “Could not start D-bus. Can you call qdbus-qt5?”. No. I cannot call this! I don’t have a working desktop!

What I managed to do was start a session with a different desktop environment and then search the internet for a solution. Will the error message, you will quickly find that “Anaconda update breaks KDE if it’s added to PATH”. Yes! I installed Anaconda the last time I used this computer! So I removed the lines it had added to my .bashrc and everything worked again.

Note to self: Do not add Anaconda to your PATH. Do not let it edit your .bashrc (which it does when you use conda init). Stupid snake!

Redirect http to https with nginx

Create two server environments, one for port 80 (http) and one for port 443 (https). Have the http environment do nothing but redirect to the other one:

server {
        listen 80;
        listen [::]:80;
        server_name yesterdayscoffee.de www.yesterdayscoffee.de;
        return 301 https://$server_name$request_uri;
}
server {
        listen 443 ssl;
        listen [::]:443 ssl;

... all the rest of your configuration

}

HTTPS with LetsEncrypt and nginx

First install Certbot on your computer using the instructions at https://certbot.eff.org/

Then you can create a certificate for the page www.yesterdayscoffee.de (with and without www) like this:

certbot -d yesterdayscoffee.de -d www.yesterdayscoffee.de --manual --preferred-challenges http certonly

During the proces, you will be asked to create a file with a specific content at a specific location on your web server (this is for the option http, there are other ways of proving that you control the domain). Once you have done this and everything is fine, the certificate will be created.

The certificate consists of a bunch of files in a location the certbot tells you. You will probably need to put two files on the server: The private key in privkey.pem and the certificate file fullchain.pem. As a location, the folder /etc/letsencrypt/live/ is suggested.

Now you have the certificate, the next step is to tell the web server to use them for your web page. We are using nginx with the configuration file for our web page yesterdayscoffee.de at the default location /etc/nginx/sites-available/. The only thing to do is to add the port 443 for the https protocol and specify the location of the certificate files. These are the lines:

listen 443 ssl;
listen [::]:443 ssl;

ssl_certificate /etc/letsencrypt/live/www.yesterdayscoffee.de/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/www.yesterdayscoffee.de/privkey.pem;

Restart nginx for the changes to take effect with:

sudo /etc/init.d/nginx restart

You may need to tell your firewall to open the https port for nginx:

sudo ufw allow 'Nginx HTTPS'

That’s it! Now it should work. Coming up: How to redirect http to https and how to renew certificates (which with letsencrypt you have to do every 90 days).

Fixing the locale on Kubuntu

After installing my brand new Kubuntu, I got the following error:

locale: Cannot set LC_ALL to default locale: No such file or directory
perl: warning: Setting locale failed.

Here is how to fix it.

The first step is to see the current settings with locale. This should give an output similar to the following:

> locale
LANG=en_IE.UTF-8
LANGUAGE=en_US
LC_CTYPE="en_DE.UTF-8"
LC_NUMERIC=en_DE.UTF-8
LC_TIME=en_DE.UTF-8
LC_COLLATE="en_DE.UTF-8"
LC_MONETARY=en_DE.UTF-8
LC_MESSAGES="en_DE.UTF-8"
LC_PAPER=en_DE.UTF-8
LC_NAME=en_DE.UTF-8
LC_ADDRESS=en_DE.UTF-8
LC_TELEPHONE=en_DE.UTF-8
LC_MEASUREMENT=en_DE.UTF-8
LC_IDENTIFICATION=en_DE.UTF-8
LC_ALL=

This output already might give you a hint about what is wrong. In my case, I have the entry en_DE which looks fishy. The next step is, to see what locales are installed on your machine. This is done with the parameter -a for locale:

> locale -a
C
C.UTF-8
en_GB.utf8
en_IE.utf8
en_US.utf8
en_ZA.utf8
POSIX

Here we already found the problem. The output does not contain the locale “en_DE”. In this case, it is because “en_DE” does not actually exist. I have no idea where it comes from. But somehow the combination of being in Germany and installing an English operating system caused it. So what I want to do is to set everything that has the wrong locale to the correct locale “de_DE” instead.

As the German locale is not installed on our system yet, we first have to create it. This is done with locale-gen:

> sudo locale-gen de_DE.utf8
Generating locales (this might take a while)...
  de_DE.UTF-8... done   
Generation complete.

Now we can set it as the default with the following:

sudo dpkg-reconfigure locales

You should restart the computer for the changes to take effect (just to be safe).

Find file names with invalid encoding on Linux

I have files copied from Windows computers in ancient times. The filenames contain special characters and they have been messed up somewhere along the way. For example I got a file named 9.5.2 Modelo de aceptaci??n (espa??ol).doc in the folder 9 Garant??a del Estado.

First, I want to find and list these files. Stackexchange tells us how to do that:

LC_ALL=C find . -name '*[! -~]*'

This will find all names that have non-ASCII letters, not only those that are broken. But in my case I have folders where ALL of the names are broken, so I don’t mind.

Second, I want to fix the names. I did it manually, but for future reference, if I ever were to do anything like that again, I might use one of the solutions proposed in this thread on serverfault.com.

Unison preferences for syncing

Unison is a tool to compare and synchronize two folders. You can configure it by GUI, but at least for me (Kubuntu 18.04) not all settings work. Specifically, I cannot set the value “0”. But there is an easy way around the problem. Unison puts a file called Profilename.prf (where “Profilename” should be replaced with the actual name of your profile) into the folder .unison in your home directory. This is simply a text file with key-value pairs, that you can edit at your leisure.

Here are standard settings for comparing two directories without comparing the file permissions:

label = My first comparison
root = /home/test/FolderOne/
root = /home/test/FolderTwo/
perms = 0
dontchmod = true

Now for the coolest feature of Unison: It is written in OCaml!! OCaml was used in my third semester to teach functional programming. I remember clearly the teacher telling us about the “usefulness” of the language. She had one slide with examples of programs written in OCaml. And she must have looked very hard to find any. There were a grand total of three programs on the slide. Two formal logic resolvers or something to that effect (we were like “yeah, really useful”). And MLDonkey (peer-to-peer filesharing was BIG in those days before Netflix, Spotify and fast internet) which she clearly didn’t know what it was for. So now, if she still has that slide, I can add another program! And a really useful one at that!

Distribute updates from server to pool computers (hacky way)

This is the crowning achievement of my days at the MINT computer pool. Sadly, by the time I write this post, it will all be deleted. So here it is, archived for posterity.

The setting is a typical computer pool setting. 24 computers and one server. All computers load their home directories from the server (see here). Otherwise, each computer is totally independent. The idea is now, that updates to the pool computers can also be distributed from the server. So that I don’t have to sit down at each computer and execute a script which is a real pain. Again, a really common problem and many solutions exist – but I made my own, hacky, one.

Setting up the server side of things is easy. Basically, we create a few folders and put one shell script in the home of a special user called admin. There is a folder /home/admin/poolsetup/script into which we put a script poolupdate.sh. You can get the script from my wkutils github repository. This script goes through all files in another folder, /home/admin/poolsetup/updates, and executes any scripts it finds in there. It redirects the outputs of the execution into a log file in the folder /home/admin/poolsetup/logs. We can use this file to check what happened. And the update script uses the log file to avoid executing scripts twice. If a log file for a given script exists, we don’t execute the script again.

Because the home directories are loaded from the server, each pool computer will have access to files in the home directory of admin. We don’t have to do any copying to distribute the update script to the pool computers and any changes to the script will take effect right away.

So here are the commands for the server setup – basically just create the necessary folder structure with the correct permissions and put the update script there:

mkdir /home/admin/poolsetup/script
mkdir /home/admin/poolsetup/updates
mkdir /home/admin/poolsetup/logs
chmod o+rx /home/admin/poolsetup/script
chmod o+rx /home/admin/poolsetup/updates
chmod o+rwx /home/admin/poolsetup/logs
wget https://github.com/Kaffeedrache/wkutils/blob/master/admin/poolupdate.sh
mv poolupdate.sh /home/admin/poolsetup/script
chmod o+rx /home/admin/poolsetup/script/poolupdate.sh

On client side, we only have to make the computer execute the update script on a regular basis. We use cron and therefor call crontab which manages the cron jobs:

crontab -e

In the editor, we add these two lines:

00 17 * * 5 bash /home/admin/poolsetup/script/poolupdate.sh
@reboot bash -c "while [[ ! -d /home/admin/ ]] ; do sleep 5; done" ; bash /home/admin/poolsetup/script/poolupdate.sh

The first line will execute the update script every Friday at 5 PM. The second line will execute the script at every system start. We need the ugly loop with the sleep to ensure that the home directories have been mounted, before trying to access them.

So how does making an update work now? Write a script that does what you want to do. Put this script into the folder /home/admin/poolsetup/updates. When a pool computer starts up, it will execute the script. After that, look into the log folder and read the corresponding log to see what has happened. When all computers execute the update, usually you need to read only one log file and then check if all other log files have the same size. Done!

Access a server with an SSH key

Install ssh on the server:

apt-get install openssh-server

Generate a key pair (files id_rsa and id_rsa.pub) with a passphrase:

ssh-keygen

Edit the ssh configuration file:

vi /etc/ssh/sshd_config

In the file, make the following settings:

   PasswordAuthentication no
   PermitRootLogin without-password
   RSAAuthentication yes
   PubkeyAuthentication yes

Add the public key as an authorized key for root that can be used for login:

cat id_rsa.pub >> /root/.ssh/authorized_keys

That’s all for the server!

Now for the client. Copy the key pair into the folder ~/.ssh. Now you should be able to connect with:

ssh -i ~/.ssh/id_rsa root@server

Regular backup with rsync and cron

This is nothing new, just for me to remember. We use rsync and cron to make a backup of all home directories regularly. There will be a weekly backup that is readily accessible and a zipped monthly backup.

First, we need to install rsync (this is for Ubuntu, replace with the package manager of your choice):

apt-get install rsync

Then we need to create the directories where we want to save the backups. Here I am putting it into /media/backup for no particular reason, use any directory you like.

mkdir /media/backup/weekly
mkdir /media/backup/monthly

Next the command that actually copies the files:

rsync -a --exclude=".*/" --delete /home/ /media/backup/weekly

The command uses rsync, which is the tool for the job. We want to backup the complete folder home with all the directories contained in there. We exclude hidden files (starting with .) to avoid copying .cache and other temporary files. You may want to refine this setting for your case. The option --delete overwrites the files in the target directory /media/backup/weekly. So last week’s data will be overwritten with this week’s data when the backup runs. I’d suggest running this command directly to see what happens before proceeding with the automation.

Now that we know how to copy the data, we just need to execute this command regularly. This is done with cron via this command:

crontab -e

This opens an editor with all cron jobs that are currently set up. Add two lines for the backup:

00 18 * * 5 rsync -a --exclude=".*/" --delete /home/ /media/backup/weekly
00 6 1 * * tar -cjf /media/backup/monthly/monthly_$(date +%Y%m%d).tar.bz2 /media/backup/weekly/

The numbers in the beginning of each line give the minutes, hours, day, month and day of the week. The rest of each line is the command. The first line copies the data to /media/backup/weekly on every Friday afternoon (at 18:00). So we will always have a backup that is at most a week old. The second line is executed on every first day of the month at 6:00 and copies the data to /media/backup/monthly in a zipped form.