Jupyter Notebooks, JupyterLab, TensorFlow, PyTorch

Table of Contents

    We recommend to use Open OnDemand https://apps.hpc.ntnu.no . There are several alternatives to start Jupyter.

    Start Jupyter from Anaconda environment module

    Anaconda Environment Module has preinstalled Jupyter and other python modules. You can change "Environment Setup" by adding this command:

    module load Anaconda3/2022.10

    Use python environment module and your locally installed Jupyter

    You will use one of preinstalled Python environment modules. And you need to installed install Jupyter to your home directory. Example commands:

    module load Python/3.11.3-GCCcore-12.3.0
    pip install --user notebook
    pip install --user jupyterlab

    These python modules will be installed to your cluster home directory: /cluster/home/USERNAME/.local/lib/python3.11/
    You will need to update "Environment Setup" to load the same environment module:

    Use your own setup, virtual environment, ...

    Do you already have your environment with Jupyter and other python modules?
    Steps for Python.
    Or you can leave "Environment Setup" empty but you will need to configure your cluster shell environment (.bashrc file). So Open OnDemand can start command: "jupyter notebook" or "jupyter lab".

    Example with Virtual Environment:

    Run Jupyter Notebook and TensorFlow (CUDA, GPU)

    There are several TensorFlow Environment Modules installed on IDUN:

    $ module avail TensorFlow
    TensorFlow/2.11.0-foss-2022a-CUDA-11.7.0
    TensorFlow/2.11.0-foss-2022a
    TensorFlow/2.13.0-foss-2023a

    This is example with GPU/CUDA:

    Check how many GPUs available:

    Run Jupyter Notebook and PyTorch

    There are several PyTorch Environment Modules installed on IDUN:

    $ module avail PyTorch
    PyTorch/1.12.0-foss-2022a-CUDA-11.7.0
    PyTorch/1.12.1-foss-2022a-CUDA-11.7.0
    PyTorch/1.13.1-foss-2022b
    PyTorch/2.0.1-foss-2022a

    Example with torchvision and GPU:

    Check available GPU:

    Jupyter with custom Python virtual environment

    Create python virtual environment PyEnv1 from command line:

    module load Python/3.12.3-GCCcore-13.3.0
    python -m venv /cluster/home/[USERNAME]/PyEnv1
    source /cluster/home/[USERNAME]/PyEnv1/bin/activate
    pip install jupyterlab
    pip install notebook
    pip install matplotlib
    pip install netCDF4
    pip install geopandas
    pip install scikit-learn
    pip install seaborn
    deactivate

    Set Environment Setup on https://apps.hpc.ntnu.no for Jupyter Notebook or Jupyter Lab:

    module load Python/3.12.3-GCCcore-13.3.0
    source /cluster/home/[USERNAME]/PyEnv1/bin/activate

    Screenshot:

    Scroll to Top