Install Python modules as non-root user on HPC cluster
The key steps
1. Create a folder named “~/python-packages”, then create a path “~/python-packages/lib/python2.6/site-packages/” with
1 | mkdir -p python-packages/lib/python2.6/site-packages/ |
The above path containing the version number of Python should be adjust accordingly base on the actual Python version you are using.
This step sets up a directory structure that is needed by python modules.
2. Next, we need to export the newly created “~/python-packages” path to the environment variable PYTHONPATH, which is read by the installation scripts of many modules. This could be done by
1 | export PYTHONPATH=~/python-pacakges |
In the above command you may need to give full path instead of the ~ for your home directory.
You may want to write this into the ~/.bashrc for later use.
3. Install setuptools if you don’t have it. Just download the source and unpack it. Then run
1 | python setup.py install --prefix=~/python-pacakges |
in the unpacked directory. “–prefix” and “–install-dir” are options of setup.py.
4. Once we have setuptools installed, in ~/python-packages/bin you should see ‘easy_install’ program. You can also install ‘pip’ by downloading its source and installing it as described in previous step.
Note to install all other modules, similar command is needed, in which you have to specify the ‘–prefix’ to ~/python-packages folder.
e.g. to install ‘MDAnalysis’ package, we could use pip in this way:
1 | pip install --install-option="--prefix=~/python-packages" MDAnalysis |
5. Add ~/python-packages/bin to the PATH environment variable.
