Skip to content

Repository Setup and Preliminaries

This projects relies on several components you need to have installed on your system.

  • Git
  • On Mac OS: Homebrew (The missing package manager for macOS)

The below instructions will help you to configure your local copy of this repository and eventually install software dependencies required for this project.

Note: in the following instructions, terminal commands are prefixed by a virtual prompt $>which obviously does not belong to the command.

This repository is hosted on Gitlab @ Uni.lu. Git interactions with this repository (push, pull etc.) are performed over SSH using the port 8022 To clone this repository, proceed as follows (adapt accordingly):

$> mkdir -p ~/git/gitlab.uni.lu/www/
$> cd ~/git/gitlab.uni.lu/www/
$> git clone ssh://git@gitlab.uni.lu:8022/www/ulhpc-demos.git ulhpc-demos
$> cd ulhpc-demos
# /!\ IMPORTANT: run 'make setup' only **AFTER** Pre-requisites sofftware are installed
To setup you local copy of this repository (after pre-requisites are satisfied), simply run:

$> make setup    # Under Mac OS / Linux

This will initiate the Git submodules of this repository (see .gitmodules) and setup the git flow layout for this repository. Later on, you can update your local branches by running:

 $> make up

If upon pulling the repository, you end in a state where another collaborator have upgraded the Git submodules for this repository, you'll end in a dirty state (as reported by modifications within the .submodules/ directory). In that case, just after the pull, you have to run make up to ensure consistency with regards the Git submodules:

Finally, you can upgrade the Git submodules to the latest version by running:

$> make upgrade

Pre-requisites Software

Install the following software, depending on your running platform:

Platform Software Description Usage
Mac OS Homebrew The missing package manager for macOS brew install ...
Mac OS Brew Cask Plugin Mac OS Apps install made easy brew cask install ...
Mac OS iTerm2 (optional) enhanced Terminal
Windows Git for Windows I'm sure you guessed
Windows SourceTree (optional) enhanced git GUI
Windows Virtual Box Free hypervisor provider for Vagrant
Windows Vagrant Reproducible environments made easy.

Follow the below custom instructions depending on your running platform and Operating System.

Mac OS X

Once you have Homebrew installed:

$> brew install git-core git-flow    # (newer) Git stuff
$> brew install mkdocs               # (optional) install mkdocs
$> brew install pyenv pyenv-virtualenv direnv # see https://varrette.gforge.uni.lu/tutorials/pyenv.html
$> brew tap caskroom/cask            # install brew cask  -- see https://caskroom.github.io/
$> brew cask install virtualbox      # install virtualbox -- see https://www.virtualbox.org/
$> brew cask install vagrant         # install Vagrant    -- see https://www.vagrantup.com/downloads.html
$> brew cask install vagrant-manager # see http://vagrantmanager.com/
$> brew cask install docker          # install Docker -- https://docs.docker.com/engine/installation/mac/

Note: later on, you might wish to use the following shell function to update the software installed using Homebrew.

bup () {
    echo "Updating your [Homebrew] system"
    brew update
    brew upgrade
    brew cu
    brew cleanup
    brew cask cleanup
}

Linux

# Below instructions for Debian/Ubuntu
# Adapt the package names (and package manager) in case you are using another Linux distribution.
$> sudo apt-get update
$> sudo apt-get install git git-flow build-essential
$> sudo apt-get install rubygems virtualbox vagrant virtualbox-dkms

Windows

On Windows (10, ⅞ should also be OK) you should download and install the following tools:

The Git installation requires a few changes to the defaults, make sure the following are selected in the installer:

  • Select Components: Use a TrueType font in all console windows)
  • Adjusting your PATH environment: Use Git and optional Unix tools from the Windows Command Prompt
  • Configuring the line ending conversions: Checkout Windows-style, commit Unix-style line endings)
  • Configuring the terminal emulator to use with Git Bash: Use MinTTY (the default terminal of MSYS2)
  • Configuring extra options: Enable symbolic links

Please note that to clone a Git repository which contains symbolic links (symlinks), you must start a shell (Microsoft PowerShell in this example, but a Command Prompt - cmd.exe - or Git Bash shell should work out fine) with elevated (Administrator) privileges. This is required in order for git to be able to create symlinks on Windows:

  • Start Powershell:
    1. In the Windows Start menu, type PowerShell
    2. Normally PowerShell will appear as the first option on the top as Best match
    3. Right click on it and select "Run as administrator"

See also the instructions and screenshots provided on this tutorial.

Python Virtualenv / Pyenv and Direnv

In order to have a consistent Python environment among the collaborators of this project, the use of virtualenv (a tool to create isolated virtual Python environments) is strongly encouraged. Based on the guidelines of this blog post, this project relies on two additional tools facilitating the setup of the python environment expected within this project:

Installation instructions -- see also this blog post

### TL;DR; installation
# Mac OS
brew install direnv pyenv pyenv-virtualenv
# Linux/WSL
sudo { apt-get | yum | ... } install direnv
curl -L https://raw.githubusercontent.com/pyenv/pyenv-installer/master/bin/pyenv-installer | bash
export PATH="$HOME/.pyenv/bin:$PATH"
pyenv root     # Should return $HOME/.pyenv

Assuming you have configured the XDG Base Directories in your favorite shell configuration (~/.bashrc, ~/.zshrc or ~/.profile), you can enable direnv and pyenv as follows

# XDG  Base Directory Specification
# See https://specifications.freedesktop.org/basedir-spec/latest/
export XDG_CONFIG_HOME=$HOME/.config
export XDG_CACHE_HOME=$HOME/.cache
export XDG_DATA_HOME=$HOME/.local/share
# [...]
# Direnv - see https://direnv.net/
if [ -f "$HOME/.config/direnv/init.sh" ]; then
    . $HOME/.config/direnv/init.sh
fi
# - pyenv: https://github.com/pyenv/pyenv
# - pyenv-virtualenv: https://github.com/pyenv/pyenv-virtualenv
export PYENV_ROOT=$HOME/.pyenv
export PATH="${PYENV_ROOT}/bin:${PYENV_ROOT}/plugins/pyenv-virtualenv/bin:$PATH"
if [ -n "$(which pyenv)" ]; then
   eval "$(pyenv init --path)"
   eval "$(pyenv init -)"
   eval "$(pyenv virtualenv-init -)"
   export PYENV_VIRTUALENV_DISABLE_PROMPT=1
fi

This is required to support pyenv shims, virtualenv and direnv. You can also create the following files:

If you're using oh-my-zsh, you probably want to enable the pyenv plugin

Finally, to setup direnv, simply run:

make setup-direnv

Running direnv allow (this will have to be done only once), you should automatically enable the virtualenv ulhpc technical docs based on the python version specified in .python-version. You'll eventually need to install the appropripriate Python version with pyenv:

pyenv versions   # Plural: show all versions
pyenv install $(head .python-version)
# Activate the virtualenv by reentering into the directory
cd ..
cd -
From that point, you should install the required packages using:

pip install -r requirements.txt

Alternatively, you can use make setup-python

Post-Installations checks

(Eventually) Make yourself known to Git

$> git config –-global user.name  "Firstname LastName"              # Adapt accordingly
$> git config –-global user.email "Firstname.LastName@domain.org"   # Adapt with your mail
# Eventually, if you have a GPG key, use the public key to sign your commits/tags
$> git config --global user.helper osxkeychain       # Only on Mac OS
$> git config --global user.signingkey <fingerprint> # Ex: git config --global user.signingkey 5D08BCDD4F156AD7
# you can get your key fingerprint (prefixed by 0x) with 'gpg -K --fingerprint | grep sec'

Now you can complete the setup of your copy of this repository by running

$> make setup    # Under Mac OS / Linux
Last update: May 28, 2021