Large Data Transfers and Sharing

Globus is a reliable, high-performance file transfer platform allowing users to transfer large amounts of data seamlessly between systems. It aims to make automatic transfers with built-in fault recovery.

Globus can be used for both file transfers between the Neuroscience Gateway and:

  • An institution with Globus installed
  • A personal computer (known as a personal endpoint)

Neuroscience Gateway users with input or output files exceeding 5GB can use Globus for file transfers. Globus can also be used for sharing data with your collaborators by creating a group of existing Neuroscience users who can access your data. Data can also be shared with external collaborators who are not NSG users.

Using Globus with NSG

To use Globus for file transfers or sharing, please send an email describing your data needs to nsghelp@sdsc.edu. Once approved, you will receive an email with a link to your directories where you can transfer or retrieve your files.

Two directories will be created for managing public and personal datasets and will be made available when your job is running.

The directory paths for your public and personal directories are:

/expanse/projects/nsg/external_users/public/<your_username>

/expanse/projects/nsg/external_users/personal/<your_username>

Intended use of the directories

Public

  • To make data available to other NSG users to read and process.
  • Not recommended for writing output results from NSG jobs.

Personal

  • Recommended for personal input data to be used in your own NSG jobs.
  • Recommended for writing large output results from NSG jobs.
Transferring Data from a personal computer

To use Globus to transfer from a personal computer, you will need to install the Globus Connect Personal client on your computer following the steps outlined at Globus Connect Personal.

Once a personal endpoint is set up, you can transfer files from your personal computer to the directory that was created for you by the NSG administrators.

Please note that following the protocol of science gateways NSG operates using a community user account and technically it is possible for the community user account to access user's data with read/write permission.

Any questions should be emailed to nsghelp@sdsc.edu