Migrate GitLab Instance to new Host

Here I migrate a GitLab installation from one server to another. The hostname and IP addresses will be different. Both were installed from the Omnibus version. However, currently the exact version numbers differ.

In order to migrate successfully, both versions must be exactly the same type and version. For example, both installed using the Omnibus version and exactly the same version number.

If you are unsure whether you installed the Omnibus version, you can confirm with the below command. This will fail if not. If it returns output similar to the below, you have the Omnibus version. It also shows me that currently have version 10.2.3 installed.

root@repo:~# gitlab-rake gitlab:env:info

System information
System:         Ubuntu 16.04
Current User:   git
Using RVM:      no
Ruby Version:   2.3.5p376
Gem Version:    2.6.13
Bundler Version:1.13.7
Rake Version:   12.1.0
Redis Version:  3.2.5
Git Version:    2.13.6
Sidekiq Version:5.0.4
Go Version:     unknown

GitLab information
Version:        10.2.3
Revision:       3141105
Directory:      /opt/gitlab/embedded/service/gitlab-rails
DB Adapter:     postgresql
URL:            https://gitlab.openit-uk.com
HTTP Clone URL: https://gitlab.openit-uk.com/some-group/some-project.git
SSH Clone URL:  git@gitlab.openit-uk.com:some-group/some-project.git
Using LDAP:     no
Using Omniauth: no

GitLab Shell
Version:        5.9.4
Repository storage paths:
- default:      /var/opt/gitlab/git-data/repositories
Hooks:          /opt/gitlab/embedded/service/gitlab-shell/hooks
Git:            /opt/gitlab/embedded/bin/git

If you do not have the Omnibus version, consult the official documentation on how to migrate. You need to make sure the version you are going to and from are the same.

The Strategy

  1. Install GitLab on new server [10.7.3]
  2. Update old GitLab to the latest version [10.7.3]
  3. Backup old GitLab
  4. Transfer backup files to new host
  5. Restore from backup on new instance

Install GitLab

Install GitLab on the new server.

Update GitLab

This can simply be achieved by updating the system.

apt-get update
apt-get dist-upgrade

GitLab should now be updated to the latest version. It should also have created an application backup in the below location as part of the process.

root@repo:~# ls -l /var/opt/gitlab/backups/
total 204
-rw------- 1 git git  71680 May 16 19:29 1526498956_2018_05_16_10.2.3_gitlab_backup.tar

Check the version is now the latest and the same as the origin server.

root@repo:~# gitlab-rake gitlab:env:info | grep "GitLab information" -A1
GitLab information
Version:	10.7.3

Backup GitLab

To fully backup, you need to take both a configuration and a application backup. The configuration backup includes all of GitLabs configuration including the external URL. The application backup includes all of your repositories.

Configuration Backup

All GitLab configuration is inside /etc/gitlab. To backup this directory, run:

sudo sh -c 'umask 0077; tar -cf $(date "+etc-gitlab-%s.tar") -C / etc/gitlab'

This creates the following file in the current working directory.

root@repo:~# ls -la etc-gitlab-1526500078.tar
-rw------- 1 root root 102400 May 16 19:47 etc-gitlab-1526500078.tar

With the following content.

root@repo:~# tar -tvf etc-gitlab-1526500078.tar
drwxrwxr-x root/root         0 2017-12-09 23:32 etc/gitlab/
drwxr-xr-x root/root         0 2017-12-06 14:58 etc/gitlab/trusted-certs/
drwx------ root/root         0 2017-12-09 23:41 etc/gitlab/ssl/
-rw-r--r-- root/root      2025 2017-12-09 23:09 etc/gitlab/ssl/repo.pikedom.com.crt
-rw-r--r-- root/root      1708 2017-12-09 23:08 etc/gitlab/ssl/repo.pikedom.com.key
-rw-r--r-- root/root      2025 2017-12-09 23:39 etc/gitlab/ssl/gitlab.openit-uk.com.crt
-rw-r--r-- root/root      1708 2017-12-09 23:41 etc/gitlab/ssl/gitlab.openit-uk.com.key
-rw------- root/root     14906 2018-05-16 19:31 etc/gitlab/gitlab-secrets.json
-rw------- root/root     72812 2017-12-09 23:32 etc/gitlab/gitlab.rb

Application Backup

By default, backups are kept in /var/opt/gitlab/backups.

root@repo:~# grep backup_path /etc/gitlab/gitlab.rb
# gitlab_rails['manage_backup_path'] = true
# gitlab_rails['backup_path'] = "/var/opt/gitlab/backups"

If you change this location, be sure to run the below after.

sudo gitlab-ctl reconfigure

To actually take the application backup, run the following.

sudo gitlab-rake gitlab:backup:create

This creates a backup in the following location.

root@repo:~# ls -ltr /var/opt/gitlab/backups/
total 204
-rw------- 1 git git  71680 May 16 19:29 1526498956_2018_05_16_10.2.3_gitlab_backup.tar
-rw------- 1 git git 133120 May 16 20:14 1526501659_2018_05_16_10.7.3_gitlab_backup.tar

Host Backup (optional)

If rebuilding the machine and keeping the same IP, to avoid having to delete the host key entry in the ~/.ssh/know_hosts file, run the following to backup the SSH host keys.

sudo tar -cvf $(date "+hostkeys-%s.tar") $(grep HostKey /etc/ssh/sshd_config | grep -v ^'#' | awk '{print $2}')

Transfer

Lets zip and compress these files into a single archive for transfer. Although I don’t plan on restoring the host keys, here I zip and compress all three all the same.

/var/opt/gitlab/backups/1526501659_2018_05_16_10.7.3_gitlab_backup.tar
/root/etc-gitlab-1526500078.tar
/root/hostkeys-1526501148.tar

Zip and compress with the following.

sudo tar -cvzf ~/gitlab-backup.tar.gz /var/opt/gitlab/backups/1526501659_2018_05_16_10.7.3_gitlab_backup.tar /root/etc-gitlab-1526500078.tar /root/hostkeys-1526501148.tar

Use scp to copy the backup securely over the internet. Here I run the below from the new GitLab server to pull the above backup to the current working directory.

sudo scp -v root@old.dummydomains.org.uk:/root/gitlab-backup.tar.gz .

Restore

As I am mainly concerned about the Git repositories themselves, I will only be doing an application restore.

If you are moving to a new host with the same hostname etc., you should also restore the configuration backup to /etc/gitlab/. If you want everything to be the same but the hostname is different, you can manually amend anything such as external_url before running sudo gitlab-ctl reconfigure.

Application Backup Restore

Stop processes using the database.

sudo gitlab-ctl stop unicorn
sudo gitlab-ctl stop sidekiq
sudo gitlab-ctl status

Check they’re actually down.

sudo gitlab-ctl status | grep ^down

Move the application backup to the backups directory as defined in gitlab.rb and correct the owner if necessary.

sudo mv -v ~/temp/1526501659_2018_05_16_10.7.3_gitlab_backup.tar /var/opt/gitlab/backups/
sudo chown -v git:git /var/opt/gitlab/backups/1526501659_2018_05_16_10.7.3_gitlab_backup.tar

Then restore.

sudo gitlab-rake gitlab:backup:restore BACKUP=1526501659_2018_05_16_10.7.3

Now restart GitLab.

sudo gitlab-ctl start

And check for problems.

sudo gitlab-rake gitlab:check SANITIZE=true

If all was successful, you should be able to log into the new site as before with whatever users you had setup. If you had 2FA enabled, you may need to restore the gitlab-secrets.json.

Configuration Backup Restore

Exactly what you choose to restore from the /etc/gitlab/ directory is up to you. None maybe necessary. If any of your users had 2FA setup, you will need to restore the gitlab-secrects.json file or else they will lose their access. If unsure, you could always do a diff to compare the differences.

diff old-gitlab.rb new-gitlab.rb

If moving to a new host but the hostname and everything should remain the same, you can simply move the old directory out the way and replace with the old.

First, stop unicorn and sidekiq at a minimum.

sudo gitlab-ctl stop

Then restore the configuration directory.

mv -v /etc/gitlab{,.original}
mv -v ~/temp/etc/gitlab /etc/gitlab

Then start GitLab again and check.

sudo gitlab-ctl start
sudo gitlab-rake gitlab:check SANITIZE=true

Fix any issue and then re-run if necessary. Try re-configuring if you have issues.

 
sudo gitlab-ctl reconfigure

If even the IP address remains the same as before and you want to prevent your users from seeing an SSH man-in-the-middle attack warning, you will also need to restore the HostKeys.

You need to replace the HostKeys defined in your sshd_config.

andy@repo:~$ grep HostKey /etc/ssh/sshd_config 
# HostKeys for protocol version 2
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_dsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
HostKey /etc/ssh/ssh_host_ed25519_key

…with the ones from your backup.

andy@repo:~$ tar -tvf temp/hostkeys-1526501148.tar.gz 
-rw------- root/root      1679 2017-12-05 13:37 etc/ssh/ssh_host_rsa_key
-rw------- root/root       672 2017-12-05 13:37 etc/ssh/ssh_host_dsa_key
-rw------- root/root       227 2017-12-05 13:37 etc/ssh/ssh_host_ecdsa_key
-rw------- root/root       399 2017-12-05 13:37 etc/ssh/ssh_host_ed25519_key

Rename the existing keys.

for i in $(grep ^HostKey /etc/ssh/sshd_config | awk '{print $2}'); do sudo mv -v $i $i.bkp-$(date +%s);done

Then restore the old ones in-place.

sudo tar -xvzf temp/hostkeys-1526501148.tar.gz -C /

Restart GitLab and check for errors.

sudo gitlab-ctl restart
sudo gitlab-rake gitlab:check SANITIZE=true

Resources

https://www.fabian-keller.de/blog/migrating-a-gitlab-instance-to-another-server
https://www.tienle.com/2016/05-24/migrate-gitlab-to-new-server.html
https://stackoverflow.com/questions/31534293/how-to-migrate-gitlab-to-a-new-server

Install and Configure GitLab on Ubuntu 16.04

Dependencies

Run to the below command to install the GitLab dependencies.

sudo apt-get update
sudo apt-get install ca-certificates curl openssh-server postfix

When prompted, select Internet Site when configuring Postfix.

Postfix - Mail Type
Postfix – Mail Type

Enter the FQDN.

Postfix - enter FQDN
Postfix – enter FQDN

Installing GitLab

GitLab provides a bash script that add their repository for you. Just download it and run it.

cd /tmp
curl -LO https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh
sudo bash /tmp/script.deb.sh

Now you can install the GitLab package using the apt-get package manager.

sudo apt-get install gitlab-ce

Configuring GitLab

You need to first edit the configuration file.

sudo vim /etc/gitlab/gitlab.rb

I added the following line to define my primary hostname and URL.

external_url 'https://repo.dummydomains.org.uk'

I also enabled letsencrypt. If you do, make sure the DNS is setup already.

letsencrypt['enable'] = true
letsencrypt['contact_emails'] = ['postmaster@dummydomains.org.uk']
letsencrypt['auto_renew'] = true
letsencrypt['auto_renew_hour'] = "3"
letsencrypt['auto_renew_minute'] = "30"
letsencrypt['auto_renew_day_of_month'] = "*/7"

This will auto-renew at 3:30 AM every seventh day. Now run the below command to bring about your changes.

sudo gitlab-ctl reconfigure

If all was successful, you should see gitlab Reconfigured! on the last line of output.

Browse to the URL and change your password.

GitLab - Change Password
GitLab – Change Password

Then log in with the root user and the password you just created.

GitLab - Sign In
GitLab – Sign In

GitLab is now installed!

GitLab - Sign In
GitLab – Sign In

It’s time to start configuring it! You may want to look at preventing people signing up.

Resources

https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-gitlab-on-ubuntu-16-04
https://docs.gitlab.com/omnibus/
https://about.gitlab.com/installation/#ubuntu
https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https
https://docs.gitlab.com/omnibus/settings/ssl.html#let-39-s-encrypt-integration
https://www.howtoforge.com/tutorial/how-to-install-and-configure-gitlab-on-ubuntu-16-04/
https://tecadmin.net/install-gitlab-ce-on-ubuntu/
https://hostpresto.com/community/tutorials/install-and-configure-gitlab-on-ubuntu-16-04/

The Quick Start Guide to vcsh/mr Setup

I recently wrote a post about how to install configure vcsh and mr to convienently manage and use all your repositories.  However for demostration this was quite a manual and long process.  To achieve a very similar outcome in a much shorter time, you could use the provided template and customise from there.

The below will use vcsh to clone the git repository to mr.git in the ~/.config/vcsh/repo.d/ directory.

$ yaourt -Sy vcsh myrepos
$ vcsh clone https://github.com/RichiH/vcsh_mr_template mr
$ mr update

The above repository is not owned by me so I can not push changes to it. As such you will need to edit the configuration and point it to your repository. I have setup the follow:

https://github.com/crazzyfool/mr.git

Edit ~/.config/mr/available.d/mr.vcsh like so.

$ vim ~/.config/mr/available.d/mr.vcsh

Don’t forget to replace the remote repository with your own.

[$HOME/.config/vcsh/repo.d/mr.git]
checkout = vcsh clone https://github.com/crazzyfool/mr.git mr
update = vcsh mr pull
push =  vcsh mr push
status = vcsh mr status
gc = vcsh mr gc

To see the current remote repository.

[andy@home-pc ~]$ vcsh mr remote show origin
* remote origin
  Fetch URL: https://github.com/RichiH/vcsh_mr_template
  Push  URL: https://github.com/RichiH/vcsh_mr_template
  HEAD branch: master
  Remote branch:
    master tracked
  Local branch configured for 'git pull':
    master merges with remote master
  Local ref configured for 'git push':
    master pushes to master (up to date)

To remove the above origin from the repository and set your own as the upstream repository.

$ vcsh mr remote remove origin
$ vcsh mr remote add origin https://github.com/crazzyfool/mr.git
$ vcsh mr push --set-upstream origin master

Now you can add configurations as you like and push them to your remote repository simply with.

$ vcsh mr push

Or you could of course use mr to push changes to all repositories.

$ mr push

But that’s it! Take a look at my previous article or the official documentation for more information about how to add repositories.

nvPY on Gentoo

nvPY is a fast, simple to use, no fuss, cross-platform note taking application. One of the big benefits is that all your notes are stored somewhere on the cloud – so you don’t need to worry about them!

Unfortunately we can not simply use emerge to install nvPY, so there are a few prerequisites prior to installation.

Prerequisites

There are not many prerequisites to nvPY but here they are all the same.

Simplenotes Account

In order to use nvPY, you will first need to create yourself a simplenotes account at simplenote.com. Now armed with your simplenotes login credentails, you can begin to install.

Git

Because vnpy is not available through the Gentoo package manager, we first need to make sure we have the git client installed so that we can download the source code.

emerge --ask git

Python

At the time of writing, nvpy works best with python version 2.7.x and it currently does not work very well yet with python 3.x. Also, nvpy requires that the python version to be used is compiled with support for TkInter.

I already have dev-lang/python-3.3.2-r2:3.3 install as my default. To install a lower version, like 2.7.x but keep 3.x as your default, do. Note, you will need to be root.

emerge -av python:2.7

If, like me, you already have python 2.7.x installed, make sure it is compiled with TkInter support. If not, as root, do the following.

echo "=dev-lang/python-2.7.5 tk" >> /etc/portage/package.use
emerge --ask --newuse =dev-lang/python-2.7.5-r3

Installation

You should now be able to simply follow the instructions on github. To summarise (copy really I guess), see below. Make sure you do this as a normal user.

git clone git://github.com/cpbotha/nvpy.git
cd nvpy
python nvpy

Launch nvPY

In order to run nvPY, it first needs to be able to connect to your SimpleNotes account. So before you first try to run nvPY, create the following file, making sure only your normal user can read it.

nano ~/.nvpy.cfg

Populate it with only the following information. Obviously, replace the values on the right hand side of the equals sign with your SimpleNotes login credentials.


[nvpy]
sn_username = your_simplenote_username
sn_password = your_simplenote_password

Make sure only you can read it.

chmod -v 600 ~/.nvpy.cfg

Finally, try running it.

python2.7 nvpy &

Hopefully that worked!!

nvPY
nvPY installed on Gentoo

Yay!! Job done!

Resources

The Official Site – GitHub:
github.com/cpbotha/nvpy

How to install Python Tkinter in Gentoo Linux:
how-to.wikia.com/wiki/How_to_install_Python_Tkinter_in_Gentoo_Linux

How to choose python version to install in gentoo:
serverfault.com/questions/145856/how-to-choose-python-version-to-install-in-gentoo