— Compute Servers

If you prefer a classic Linux terminal instead of an interactive notebook interface, you can select the "Terminal" option from the Launcher. Once logged into the JupyterHub interface, after selecting an environment, the Jupyter interface will appear. By pressing the '+' button in the tab bar, you can open the Launcher, and the Terminal option will be revealed.

If you prefer working from an SSH session instead of JupyterHub, you can do so. Read the next FAQ item on how to connect via SSH. After logging in, you can run conda env list to list all the available Anaconda environments.

The instances are also accessible through SSH (and SFTP). You can choose to connect in this manner to quickly upload datasets, or if you prefer working from a terminal or an IDE.

The following shell commands work on Windows, Linux, and macOS:

# on campus, connected to the VU's network:

$ ssh <VUnetID>@x.compute.vu.nl

# from home or outside campus:

$ ssh -J <VUnetID>@ssh.data.vu.nl <VUnetID>@x.compute.vu.nl

Replace x with either 1, 2, or 3, depending on your choice.

Similarly, scp can be used to copy files and directories:

# copy a file to my home directory:

$ scp my_file.txt <VUnetID>@x.compute.vu.nl:my_file.txt

# copy a file to an absolute path (such as /local/data, see next FAQ item):

$ scp my_file.txt scp://<VUnetID>@x.compute.vu.nl//local/data/<VUnetID>/my_file.txt

# notice the 'scp://' prefix in the command above

# copy an entire directory (such as a dataset) to /local/data:

$ scp -r my_dataset scp://<VUnetID>@x.compute.vu.nl//local/data/<VUnetID>/my_dataset

In case you are not on campus, or not connected to the VU's network, you can use the -J option as shown in the SSH example:

$ scp -J <VUnetID>@ssh.data.vu.nl ... ...

You can create a directory in /local/data/<VUNetID>/ (such as /local/data/abc123/my_dataset/), and place your dataset there using SSH/SFTP. Usually, this directory is automatically created for you on login. Inside you can find a README.txt file containing details about the /local folder. Be aware that these folders are local to each compute instance, and are not synchronized. Therefore, they are using the instance's own storage space.

Make sure that there is enough space for your dataset before placing it (you can verify by running df -h /local/data in the Terminal).

The compute instances make up a shared service. Please remove your folders from /local/data/<VUNetID>/ when you don't need them anymore. This way, you can leave storage space for others. The directories in /local/data are removed if the directory name is not a valid VUnetID. Every first day of the month, the directories are moved to /local/.old/. If your data is missing, check this directory first and move your relevant files back. You are responsible of keeping your data backed up.

Yes, Anaconda is available for use through an SSH session. You can use Anaconda by typing the command conda in the Terminal. You can list the provided environments by running conda env list. Anaconda is not available through the JupyterHub web interface.

If you want to create your own environment, be aware of your storage quota. Similar to how you'd proceed with datasets, you can create a conda environment in /local/data/<VUNetID>/. If your environment is small enough, you may create it in your home drive, to take advantage of your syncronised home directory.

You can create an environment in /local/data/<VUNetID>/ like so (note the trailing slash in each command!):

$ conda create -p /local/data/<VUNetID>/<my_environment>/ [packages...]

$ conda activate /local/data/<VUNetID>/<my_environment>/

# after you are done using it:

$ conda deactivate

$ conda env remove -p /local/data/<VUNetID>/<my_environment>/

The compute instances make up a shared service. Please remove your folders from /local/data/<VUNetID>/ when you don't need them anymore. This way, you can leave storage space for others. The directories in /local/data are removed if the directory name is not a valid VUnetID. Every first of the month, the directories are moved to /local/.old/. If your data is missing, check this directory first and move your relevant files back. You are responsible of keeping your data backed up.

If you use this custom environment very often, and/or you need it for teaching, you can make a request to have your environment predefined across all the compute instances.

Do you want to use JupyterHub for a course you are teaching? You can request a custom environment for your course by filling in this form:

• JupyterHub — Request an online development environment for your course.

Mention the software you'd like to have included (e.g. libraries for Python, R, Matlab) and their versions, if you need specific ones. For Python libraries, you can, for example, include the contents of your requirements.txt file, if you have one. Also mention other specific requirements for your course, such as a shared data directory.

The Compute Servers are provided by VU-IT. If you have any problems or questions, please send an email to servicedesk.it@vu.nl, mentioning "JupyterHub".

Some additional information can also be found here.