Using Globus Connect with the BRC Supercluster

This document describes how to use Globus Connect (formerly Globus Online) to transfer data between your computer (or any other computer on the Internet that is acting as a Globus "endpoint," and to which you have access) and the Berkeley Research Computing (BRC) supercluster, consisting of the Savio, Vector, and Cortex high-performance computing clusters at the University of California, Berkeley.

Globus Connect is ideal for transferring large files and/or large numbers of files at high speed. It allows you to start an unattended transfer and have confidence that it will be completed reliably; even sending you an email notification when it's done.

For information on how to use it, see Globus's Step-by-Step Getting Started Guide.

A quick summary for users:

  1. Visit the Globus website.
  2. Create a free account. (Click the Sign Up button at upper right.)
  3. Log into the Globus website.
  4. Click the "Transfer Files" link near the top of the home page.
  5. On the Transfer Files page:
    • Click in the Endpoint field at left. A new pop-up screen will appear, displaying a different (and very long) Endpoint field.
    • Enter ucb#brc into the Endpoint field. (This is the Globus endpoint for the BRC supercluster, which connects via the supercluster's Data Transfer Node, dtn.brc.berkeley.edu.)
    • Select the ucb#brc entry from the Endpoint list displayed below the Endpoint field. Doing so will return you to the original Transfer Files page.
    • If prompted to authenticate, click the "Continue" button and follow the on-screen instructions. (In brief, you'll be taken to an LBNL page where you can enter your BRC cluster username, along with your token PIN followed by a one-time password that you will generate via the Google Authenticator application. See Logging In for details on generating that password.)
    • Enter the path for the directory on the BRC cluster to/from which you'd like to copy files, into the Path field at left. (E.g. /~/ or /global/home/users/myusername for your home directory, where myusername is your BRC cluster username. For more details on storage locations, see the relevant User Guide page for the cluster on which you are working.)
    • Click in the Endpoint field at right. A new pop-up screen will appear, displaying a different (and very long) Endpoint field.
    • Enter the name of your second endpoint, where you'd like to copy files to/from the BRC supercluster, into the Endpoint field at right. (Globus provides two test endpoints, go#ep1 and go#ep2, each of which provides sample files that you can practice transferring to the cluster.)
    • Select the appropriate entry from the Endpoint list displayed below the Endpoint field. (For instance, if you entered the name of one of Globus's two test endpoints, select 'Globus Tutorial Endpoint 1' or 'Globus Tutorial Endpoint2', respectively, from that list.) Doing so will return you to the original Transfer Files page.
    • Authenticate as needed to that second endpoint. (Note: the Globus test endpoints, mentioned above, do not require authentication.)
    • In the list of files in either of the two endpoints, select one or more files or folders.
    • Click the appropriate "arrow" button to transfer these files to the other endpoint.

To transfer files between your computer and the BRC supercluster, you'll need to set up Globus Connect Personal. Here's a quick overview of how to install and use it:

  1. Log into the Globus website. (If you're not already logged in, that is.)
  2. Visit the Manage Endpoints page.
  3. Click the "add Globus Connect Personal" link.
  4. Follow the onscreen instructions on how to set up and install this software. You can also find detailed, official instructions for installing Globus Connect Personal for Mac OS X, Microsoft Windows, and Linux via links on this Globus Support "How To" page.
  5. Once you've installed Globus Connect Personal, and that software is actively running on your computer, click the "Transfer Files" link near the top of the Globus home page.
  6. On the Transfer Files page:
    • Click in the Endpoint field at left. A new pop-up screen will appear, displaying a different (and very long) Endpoint field.
    • Enter ucb#brc into the Endpoint field. (This is the Globus endpoint for the BRC supercluster, which connects via the supercluster's Data Transfer Node, dtn.brc.berkeley.edu.)
    • Select the ucb#brc entry from Endpoint list displayed below the Endpoint field. Doing so will return you to the original Transfer Files page.
    • If prompted to authenticate, follow the onscreen instructions. (See earlier instructions above for more details.)
    • Enter the path for the directory on the BRC cluster to/from which you'd like to copy files, into the Path field below that Endpoint field. (E.g. /~/ or /global/home/users/myusername for your home directory, where myusername is your BRC cluster username. For more details on storage locations, see the relevant User Guide page for the cluster on which you are working.)
    • Enter the endpoint name for your own computer into the other Endpoint field. (Note: the Globus Connect Personal application must be running on your computer in order to enable that Endpoint.)
    • In the list of files in either of the two endpoints, select one or more files or folders.
    • Click the appropriate "arrow" button to transfer these files to the other endpoint.