Jupyter Notebooks, JupyterLab, TensorFlow, PyTorch

Table of Contents

    We recommend to use Open OnDemand https://apps.hpc.ntnu.no . There are several alternatives to start Jupyter.

    Start Jupyter from Anaconda environment module

    Anaconda Environment Module has preinstalled Jupyter and other python modules. You can change "Environment Setup" by adding this command:

    module load Anaconda3/2022.10

    Use python environment module and your locally installed Jupyter

    You will use one of preinstalled Python environment modules. And you need to installed install Jupyter to your home directory. Example commands:

    module load Python/3.11.3-GCCcore-12.3.0
    pip install --user notebook
    pip install --user jupyterlab

    These python modules will be installed to your cluster home directory: /cluster/home/USERNAME/.local/lib/python3.11/
    You will need to update "Environment Setup" to load the same environment module:

    Use your own setup, virtual environment, ...

    Do you already have your environment with Jupyter and other python modules?
    Steps for Python.
    Or you can leave "Environment Setup" empty but you will need to configure your cluster shell environment (.bashrc file). So Open OnDemand can start command: "jupyter notebook" or "jupyter lab".

    Example with Virtual Environment:

    Run Jupyter Notebook and TensorFlow (CUDA, GPU)

    There are several TensorFlow Environment Modules installed on IDUN:

    $ module avail TensorFlow
    TensorFlow/2.11.0-foss-2022a-CUDA-11.7.0
    TensorFlow/2.11.0-foss-2022a
    TensorFlow/2.13.0-foss-2023a

    This is example with GPU/CUDA:

    Check how many GPUs available:

    Run Jupyter Notebook and PyTorch

    There are several PyTorch Environment Modules installed on IDUN:

    $ module avail PyTorch
    PyTorch/1.12.0-foss-2022a-CUDA-11.7.0
    PyTorch/1.12.1-foss-2022a-CUDA-11.7.0
    PyTorch/1.13.1-foss-2022b
    PyTorch/2.0.1-foss-2022a

    Example with torchvision and GPU:

    Check available GPU:

    Jupyter with custom Python virtual environment

    Create python virtual environment PyEnv1 from command line:

    module load Python/3.12.3-GCCcore-13.3.0
python -m venv /cluster/home/[USERNAME]/PyEnv1
source /cluster/home/[USERNAME]/PyEnv1/bin/activate
pip install jupyterlab
pip install notebook
pip install matplotlib
pip install netCDF4
pip install geopandas
pip install scikit-learn
pip install seaborn
deactivate

    Set Environment Setup on https://apps.hpc.ntnu.no for Jupyter Notebook or Jupyter Lab:

    module load Python/3.12.3-GCCcore-13.3.0
source /cluster/home/[USERNAME]/PyEnv1/bin/activate

    Screenshot:

    Scroll to Top