Managing dot files with vcsh and myrepo

April 30, 2018 Arch Linux, Linux, Manjaro i3

Ever wanted to manage all those ~/.conf files with Git?  Probably not but if you know anything about Git, the advantages are obvious. Is it even possible though you may ask!?! To do so would mean separating the working tree from the actual git repository. Fortunately Git allows for just that.

By default, when you create a non-bare repository, you end up with the actual repository, a .git directory being created in the directory you want to track. The directory you want to track is actually called the working tree. The repository itself is a load of files (objects really) that maps and tracks the state of a working tree. That’s what a non-bare repository is in fact – a git repository with a checked-out working tree included. Fortunately the working tree and the git repository do not actually need to be in the same location. You can set these with the git variables, $GIT_DIR for the repository location and $GIT_WORK_TREE for the working tree.

Best of all however is that you don’t even need to get your hands dirty with this kind of thing as a nice chap called Richard Hartmann has taken all the potential pain out of it with vcsh!  All you need is a basic/working knowledge of Git.

The Plan

In this example I want to track configuration changes to two applications, the i3 Window Manager and Bash.  I’ll walk you through how to use vcsh for this, and then demonstrate how a program called mr, available with the myrepos package, can make your life even simpler.

You’ll need to setup a separate repository for each application configuration you want to track. Here I have setup one for i3wm and bash.

https://github.com/crazzyfool/config-i3.git
https://github.com/crazzyfool/config-bash.git

Later you’ll also need one for mr.

https://github.com/crazzyfool/config-mr.git

vcsh

Using vcsh allows one to have multiple working trees in the same directory by storing the actual repository data in a separate location.

Installation

I run Manjaro so to install its:

$ yaourt -Sy vcsh

Add an Application Configuration

So I want to use vcsh to manage the configuration changes to the i3 Windows Manager, located ~/.i3/config.

vcsh init config-i3
vcsh config-i3 add ~/.i3/
vcsh config-i3 commit -m "My initial i3 config"
vcsh config-i3 remote add origin https://github.com/crazzyfool/config-i3.git
vcsh config-i3 push -u origin master

You may also want to create a gitignore file too.

vcsh write-gitignore config-i3

See below for more about that.

Add a Second Application Configuration

vcsh init config-bash
vcsh config-bash add ~/.bashrc
vcsh config-bash commit -m "My initial bash config"
vcsh config-bash remote add origin https://github.com/crazzyfool/config-bash.git
vcsh config-bash push -u origin master

Maybe you want to add a second file to the bash configuration.

vcsh config-bash add ~/.bash_profile
vcsh config-bash commit -m "Added .bash_profile"
vcsh config-bash push

What Just Happened?

All the actual local repositories are stored in ~/.config/vcsh/repo.d while the working trees are located in the ~/ directory.

/home/andy/.config/vcsh/
└── repo.d
    ├── config-bash.git
    └── config-i3.git

Using vcsh to Manage your Configuration

You can get a full list of options from the help menu.

$ vcsh help
usage: vcsh<options> <command>

   options:
   -c <file>            Source file
   -d                   Enable debug mode
   -v                   Enable verbose mode

   commands:
   clone [-b <branch>] \
         <remote> \
         [<repo>]       Clone from an existing repository
   commit               Commit in all repositories
   delete <repo>        Delete an existing repository
   enter <repo>         Enter repository; spawn new instance of $SHELL
   foreach [<-g>]
     <git command>      Execute a command for every repository
   help                 Display this help text
   init <repo>          Initialize a new repository
   list                 List all repositories
   list-tracked \
        [<repo>]        List all files tracked all or one repositories
   list-untracked \
        [<-a>] [<-r>]
        [<repo>]        List all files not tracked by all or one repositories
   pull                 Pull from all vcsh remotes
   push                 Push to vcsh remotes
   rename <repo> \
          <newname>     Rename repository
   run <repo> \
       <command>        Use this repository
   status \
     [--terse] [<repo>] Show statuses of all/one vcsh repositories
   upgrade <repo>       Upgrade repository to currently recommended settings
   version              Print version information
   which <substring>    Find substring in name of any tracked file
   write-gitignore \
   <repo>               Write .gitignore.d/<repo> via git ls-files

   <repo> <git command> Shortcut to run git commands directly
   <repo>               Shortcut to enter repository

Using vcsh to List

You can list all repositories with vcsh list.

$ vcsh list
config-bash
config-i3

To list all the currently tracked files.

$ vcsh list-tracked
/home/andy/.bash_profile
/home/andy/.bashrc
/home/andy/.i3/config

If you just want to see what files one particular repository, such as config-bash is tracking, you can use the following.

$ vcsh list-tracked config-bash
/home/andy/.bash_profile
/home/andy/.bashrc

Editing Configuration

Once you’ve edited a file, say .bashrc, you will need to add and commit those changes. Running the below command will show you that file has been modified.

$ vcsh config-bash status
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

	modified:   .bashrc

no changes added to commit (use "git add" and/or "git commit -a")

If you haven’t already created a .gitignore file, your output might look more verbose.

[andy@home-pc ~]$ vcsh config-bash status
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

	modified:   .bashrc

Untracked files:
  (use "git add <file>..." to include in what will be committed)

	.Xauthority
	.Xclients
	.Xresources
	.bash_history
	.bash_logout
	.cache/
	.config/
	.dir_colors
	.dmenurc
	.dmrc
	.gimp-2.8/
	.gitconfig
	.gksu.lock
	.gtkrc-2.0
	.i3/
	.icons/
	.local/
	.moc/
	.moonchild productions/
	.profile
	.screenlayout/
	.ssh/
	.viminfo
	.xinitrc
	.xsession-errors
	.xsession-errors.old
	Projects/
	configure.sh
	test-file.txt

no changes added to commit (use "git add" and/or "git commit -a")

If this is the case, you can get vcsh to create one for you.

$ vcsh write-gitignore config-bash

Now the ~/.gitignore.d/config-bash file will exist with the following content:

$ cat ~/.gitignore.d/config-bash 
*
!/.bash_profile
!/.bashrc

To add that change commit.

$ vcsh config-bash add .bashrc
$ vcsh config-bash commit -m "Added ll alias to .bashrc"
$ vcsh config-bash push

Status

You can set the status of a repository with.

$ vcsh config-bash status

Or the status of all repositories with.

$ vcsh status

Pull Changes

To pull changes from Github.

$ vcsh pull

…or for just one repository.

$ vcsh config-i3 pull

Push Changes

To push your changes back up to Github.

$ vcsh push

Or for a given repository.

$ vcsh config-i3 push

Running Git Commands

You can also just run normal Git command appended onto vcsh repo-name. For example, git log.

$ vcsh config-bash log
commit bd775543cbaeca89c20b5dad421f15b1cb02da45 (HEAD -> master, origin/master)
Author: Andrew Pike <[email protected]>
Date:   Sat Apr 28 20:35:44 2018 +0100

    Added ll alias to .bashrc

commit 5c9f459ed14de5e677e02f9eff7d3c749ddb07c8
Author: Andrew Pike <[email protected]>
Date:   Sat Apr 28 20:09:29 2018 +0100

    Added .bash_profile

commit 08acbe7e6d7b483e9dc0a4b748dada50f195cc09
Author: Andrew Pike <[email protected]>
Date:   Sat Apr 28 19:22:36 2018 +0100

    My initial bash config

Or git diff.

$ vcsh config-bash diff
diff --git a/.bashrc b/.bashrc
index 7b8ab85..e8252f1 100644
--- a/.bashrc
+++ b/.bashrc
@@ -96,6 +96,7 @@ alias free='free -m'                      # show sizes in MB
 alias np='nano -w PKGBUILD'
 alias more=less
 alias ll='ls -la'
+alias ls='ls -l'
 
 xhost +local:root > /dev/null 2>&1

Limitations

vcsh works great but what happens when you want to migrate to or pull in the configuration to another machine? Enter mr and myrepos.

myrepos

A command called mr, included in the myrepos package is used to simultaneously managed repositories. You can also use it to quickly enable or disable any repositories too. As mr knows about all the repositories you are interested in, you can save your mr configuration to Git and then proceed to pull in all your other configurations from their repositories in one go.

Installation

On Manjaro this would be:

$ yaourt -Sy myrepos

Some useful dependencies to consider when installing myrepos. Note you can use myrepos to manage repositories using many different revision control systems including Subversion and CVS!

Optional dependencies for myrepos
    bzr: support for bzr repositories
    cvs: support for cvs repositories
    darcs: support for darcs repositories
    git-annex: support for git-annex clones
    gitk: support for visualizing git repository history
    git: support for git repositories [installed]
    mercurial: support for mercurial repositories
    perl-html-parser: support for webcheckout
    perl-libwww: support for webcheckout
    perl-uri: support for webcheckout heuristically guessing partial URLs [installed]
    repo: support for repo repositories
    svn: support for subversion repositories [installed]
    unison: support for unison as a vcs
    vcsh: support for vcsh

Configure Mr

You need to create the file structure yourself. I’m using the recommended layout.

[andy@home-pc ~]$ mkdir -pv .config/mr/{available.d,config.d}
mkdir: created directory '.config/mr'
mkdir: created directory '.config/mr/available.d'
mkdir: created directory '.config/mr/config.d

And create the configuration file.

$ vim ~/.mrconfig

The content should look like the below.

[DEFAULT]
git_gc = git gc "$@"
jobs = 5

include = cat ~/.config/mr/config.d/*

Add Repositories to mr

Now we need to add our repositories to mr. To add i3, create a config file in the available.d directory>

 
$ vim ~/.config/mr/available.d/i3.vcsh

Create a configuration file like so.

[$HOME/.config/vcsh/repo.d/config-i3.git]
checkout = vcsh clone https://github.com/crazzyfool/config-i3.git config-i3
update = vcsh config-i3 pull 
push = vcsh config-i3 push
status = vcsh config-i3 status
gc = vcsh config-i3 gc

The first line is the path to the Git directory. Now lets add one for bash.

$ vim ~/.config/mr/available.d/bash.vcsh

And this one looks very similar.

[$HOME/.config/vcsh/repo.d/config-bash.git]
checkout = vcsh clone https://github.com/crazzyfool/config-bash.git config-bash
update = vcsh config-bash pull
push = vcsh config-bash push
status = vcsh config-bash status
gc = vcsh config-bash gc

These repositories are now available to us but they are not enabled yet. To do so we need to create a symlink.

[andy@home-pc ~]$ cd ~/.config/mr/config.d/
[andy@home-pc config.d]$ ls -l
total 0
[andy@home-pc config.d]$ ln -s ../available.d/i3.vcsh . && ln -s ../available.d/bash.vcsh .
[andy@home-pc config.d]$ 
[andy@home-pc config.d]$ ls -l
total 0
lrwxrwxrwx 1 andy andy 24 Apr 29 18:58 bash.vcsh -> ../available.d/bash.vcsh
lrwxrwxrwx 1 andy andy 22 Apr 29 18:58 i3.vcsh -> ../available.d/i3.vcsh

Add a non-vcsh Repository

Now you can use vcsh to update and manage other repositories you use. For example, to configure Vim, I use The Ultimate Vim configuration.

https://github.com/amix/vimrc

To update you need to do a rebase:

cd ~/.vim_runtime
git pull --rebase

Lets try to accomplish this using mr…

$ cp -v ~/.config/mr/available.d/{bash,vim}.vcsh
$ vim ~/.config/mr/available.d/vim.vcsh

My one looks like the following.

[.vim_runtime]
checkout =
    git clone --depth=1 https://github.com/amix/vimrc.git --separate-git-dir ~/.config/vcsh/repo.d/config-vim.git ~/.vim_runtime
    sh ~/.vim_runtime/install_awesome_vimrc.sh

update =
    cd ~/.vim_runtime
    git pull --rebase

Don’t forget to enable it!

[andy@home-pc ~]$ cd ~/.config/mr/config.d/
[andy@home-pc config.d]$ ln -s ../available.d/vim.vcsh .
[andy@home-pc config.d]$ ls -la vim.vcsh 
lrwxrwxrwx 1 andy andy 23 Apr 29 22:54 vim.vcsh -> ../available.d/vim.vcsh

Now when you do a mr update for the first time, mr should automatically checkout and install the Ultimate Vim configuration.

$ mr update
mr update: /home/andy/.config/vcsh/repo.d/config-i3.git
Already up to date.

mr update: /home/andy/.config/vcsh/repo.d/config-bash.git
Already up to date.

mr checkout: /home/andy/.config/vcsh/repo.d/config-vim.git
Cloning into '/home/andy/.vim_runtime'...
Installed the Ultimate Vim configuration successfully! Enjoy :-)

mr update: finished (3 ok)

If already checked-out and installed, it should check for updates and do a rebase if any are found.

[andy@home-pc ~]$ mr update
mr update: /home/andy/.config/vcsh/repo.d/config-i3.git
Already up to date.

mr update: /home/andy/.config/vcsh/repo.d/config-bash.git
Already up to date.

mr update: /home/andy/.config/vcsh/repo.d/config-vim.git
Already up to date.
Current branch master is up to date.

mr update: finished (3 ok)

Add any other Repositories

You can use mr to manage any other repositories you administer.

$ vim ~/.config/mr/available.d/scripts.vcsh

My one looks like:

[Scripts]
checkout = 
    git clone https://github.com/crazzyfool/scripts.git ~/Scripts

update = 
    cd ~/Scripts
    git pull

push = 
    cd ~/Scripts
    git push

status = 
    cd ~/Scripts
    git status

Enable it.

$ cd ~/.config/mr/config.d/
$ ln -s ../available.d/scripts.vcsh .
$ cd
$ mr update

Add Mr to Git

For reasons that will become clear in the next and final section, it is a good idea to add your mr configuration to Github.

$ vcsh init config-mr
$ vcsh config-mr add .config/mr ~/.mrconfig
$ vcsh config-mr commit -m "Added initial mr commit"
vcsh config-mr remote add origin https://github.com/crazzyfool/config-mr.git
vcsh config-mr push -u origin master

Create mr configuration.

$ vim ~/.config/mr/available.d/mr.vcsh

Such as.

[$HOME/.config/vcsh/repo.d/config-mr.git]
checkout = vcsh clone https://github.com/crazzyfool/config-mr.git config-mr
update = vcsh config-mr pull 
push = vcsh config-mr push
status = vcsh config-mr status
gc = vcsh config-mr gc

Enable it.

$ cd ~/.config/mr/config.d/
$ ln -s ../available.d/mr.vcsh .
$ cd

You’ll also need to create a gitignores file. My one looks like this.

$ cat ~/.gitignore.d/config-mr 
*
!/.gitignore.d
!/.gitignore.d/config-mr
!/.config
!/.config/mr
!/.config/mr/available.d
!/.config/mr/available.d/*
!/.config/mr/config.d
!/.config/mr/config.d/*
!/.mrconfig

Add those files to the repository.

vcsh config-mr add ~/.config/mr/available.d/mr.vcsh ~/.config/mr/config.d/mr.vcsh ~/.gitignore.d/config-mr
vcsh config-mr commit -m "Added more files"
$ vcsh config-mr push

Pull Configuration to a New Host

This is ultimately why the vcsh/mr combination is so useful. Assuming you have installed vcsh and mr, getting setup should just be a simple case of.

$ vcsh clone https://github.com/crazzyfool/config-mr.git
$ mr update

However, as I’m not actually on a new host, I need to delete a load of files first.

$ rm -Rvf ~/.config/vcsh/repo.d/config-* ~/.config/mr/ ~/.mrconfig ~/.gitignore.d/ ~/.vim_runtime/ ~/.vimrc ~/Scripts/ ~/.bashrc ~/.bash_profile ~/.i3/config

The end result should be that all five repositories are pulled it.

$ vcsh clone https://github.com/crazzyfool/config-mr.git
Initialized empty shared Git repository in /home/andy/.config/vcsh/repo.d/config-mr.git/
Switched to a new branch 'master'
remote: Counting objects: 31, done.
remote: Compressing objects: 100% (18/18), done.
remote: Total 31 (delta 4), reused 31 (delta 4), pack-reused 0
Unpacking objects: 100% (31/31), done.
From https://github.com/crazzyfool/config-mr
 * branch            master     -> FETCH_HEAD
 * [new branch]      master     -> origin/master
[andy@home-pc ~]$ 
[andy@home-pc ~]$ vcsh list
config-mr
[andy@home-pc ~]$ mr update
mr update: /home/andy/.config/vcsh/repo.d/config-mr.git
Already up to date.

mr checkout: /home/andy/Scripts
Cloning into '/home/andy/Scripts'...

mr checkout: /home/andy/.config/vcsh/repo.d/config-bash.git
Initialized empty shared Git repository in /home/andy/.config/vcsh/repo.d/config-bash.git/
Switched to a new branch 'master'
From https://github.com/crazzyfool/config-bash
 * branch            master     -> FETCH_HEAD
 * [new branch]      master     -> origin/master

mr checkout: /home/andy/.config/vcsh/repo.d/config-i3.git
Initialized empty shared Git repository in /home/andy/.config/vcsh/repo.d/config-i3.git/
Switched to a new branch 'master'
From https://github.com/crazzyfool/config-i3
 * branch            master     -> FETCH_HEAD
 * [new branch]      master     -> origin/master

mr checkout: /home/andy/.vim_runtime
Cloning into '/home/andy/.vim_runtime'...
Installed the Ultimate Vim configuration successfully! Enjoy :-)

mr update: finished (5 ok)

Conclusion

As demonstrated, the combination of both vcsh and myrepos not only gives you a powerful way to manage and track the .conf files in your $HOME directory but also a very convenient way of managing all repositories you may use. It also means that reinstalling your OS or working on a new machine is not so much of a pain. 🙂

Related Articles

Be the first to comment

Leave a Reply