The prefetch can download data through the Aspera or through Http without Aspera. The path information of Aspera connect needs to be provided in the command line for Aspera download. Aspera is more robust and faster for the downloading job with large data size. The documentation of prefetch can be found from here. IBM Aspera Connect is a free, install-on-demand application that plugs in to your Web browser to facilitate high-speed downloads with an IBM Aspera transfer server. It uses IBM Aspera's patented FASP transfer technology to achieve speeds that can be hundreds of times faster than FTP/HTTP. Aspera Connect helps you securely transfer files and folders of any size. Installation is free and easy! Tutorial to download and install IBM Aspera Connect browser plug-in using Mozilla Firefox on MacOS. For support and maintenance options, please visit: http:/.
File Transfer Protocol (FTP)
In order to upload your files to the NCBI server via FTP you need the following information:
- Address
- Username
- Password
Figure 1: Credentials for file uploading in the Submission Portal
FTP Command Line Basics
File transfer can be initiated in Windows Command Prompt (
Start -> Command Prompt
), Mac Terminal (Finder -> Applications -> Utilities -> Terminal
), Linux, and Unix.
![Aspera connect download for mac Aspera connect download for mac](/uploads/1/2/6/8/126869507/628633076.jpg)
Action | Command line |
---|---|
Change into directory with your data files | cd 'U:SRAexample-files' |
Invoke FTP protocol | ftp -i |
Access NCBI FTP server | open ftp-private.ncbi.nlm.nih.gov |
Provide username on 'User' prompt | Username |
Provide password on 'Password' prompt | Password |
Create directory | mkdir MYSUBMISSION |
Change into that directory with arbitrary name | cd MYSUBMISSION |
Transfer all files | mput * |
Useful FTP commands:
Command | Information |
---|---|
help, ? | Either command prints list of commands |
! | Toggles back and forth between the operating system and ftp. Once back in the operating system, type 'exit' to get back to the FTP command line. |
bye, quit | These commands terminate the FTP session and exit |
ls | Lists files in the remote directory |
put | Send one file |
FTP Command line Instructions for the SRA Submission Portal Wizard
After you connected to the NCBI FTP server:
-
Navigate (use command
cd
) to your account folderuploads/[email protected]_xxxxx
-
Create a submission folder (use command
mkdir
) for your data within your FTP account folder (uploads/[email protected]_xxxxx
) and deposit your files into the submission folder.If you upload your files in the account folder directly or place the files into a nested folder, you will not be able to see or select the submission folder as instructed below. The submission folder is a temporary holding area and it will be removed after the whole submission is finished processing (the button is clicked).
Using third party FTP clients
Many reliable FTP clients can be found on Internet. For example, Filezilla and NcFTP Client are free for download. Please refer to their documentation for usage instructions and troubleshooting tips.
Using FileZilla to transfer files when making submission in the SRA Submission Portal Wizard
When using FileZilla to upload files you need to paste your account folder's name into the Remote Site box on the right hand side of FileZilla interface (outlined in red) and hit Enter. Until you do this, you will see a message stating '
550 /: Permission denied
' or 'Fail to read the directory listing
'. We prevent directory listing in the default sign in folder for security reasons. From within your account folder you can then create your submission folder(s) for your submission(s).
Figure 2
Aspera Connect
Aspera Connect is software that allows download and upload via a web plugin for popular browsers on machines running Linux, Windows, and Macintosh. The software also includes a command line tool (
ascp
) that allows scripted data transfer. The software client is free for users exchanging data with NCBI.
Aspera Connect Fast and Secure Protocol (FASP) uses User Datagram Protocol (UDP) that eliminates and overcomes many shortcomings of other FTP clients and we recommend it for all medium to large submissions and slow or unreliable connections (especially from abroad).
Download and install Aspera Connect software from: http://downloads.asperasoft.com/en/downloads/8?list .
Aspera command line for the SRA Submission Portal Wizard
You may use the following command to upload files via Aspera command line:
<path to aspera executables> -i <path/to/key-file> -QT -l 100m -k1 -d <path/to/folder-with-file(s)> [email protected]:uploads/<[email protected]_xxxxx>
Aspera command line explanation:
<[email protected]_xxxxx>
is your personal account SRA folder generated for you and is listed under the Aspera Command-Line upload link;-
<path to aspera executables>
or in many cases, simply,ascp
:- Microsoft Windows:
C:Program FilesAsperaAspera Connectbinascp.exe
- Mac OS X:
/Applications/Aspera/Connect.app/Contents/Resources/ascp
(for admin's installation) or/Users/[username]/Applications/Aspera/Connect.app/Contents/Resources/ascp
(for non-admin's installation) - Linux:
/opt/aspera/bin/ascp
or/home/[username]/aspera/connect/bin/ascp
- Microsoft Windows:
i <path/to/key_file>
- absolute path and file name of the private SSH key (e.g.:/home/keys/aspera.openssh
)<key-file>
is provided upon your clicking on Get the key file link or sent upon request for direct SRA uploadsQ
- for adaptive flow control (needed for disk throttling)T
- to disable encryptionk1
- enables resuming of partial transfersd
-<path/to/folder-with-file(s)>
should include a directory name and so should not be '.' Or './'l100m
- designates maximum bandwidth of 100Mps, try 100m and move up if necessary to ensure good performance with unattended operations<path/to/folder-with-file(s)>
- names of files to transfer including path or path to a folder that contains all files
The environmental variable
ASPERA_SCP_PASS
can be used to store the private key path for a scripted series of bulk uploads.
A submission folder
<folder-with-file(s)>
will be created automatically by Aspera within your account folder <[email protected]_xxxxx>
when the option -d
is provided.
At the last step of your submission, the submissionfolder containing your files can be selected and the files will link to your submission.
Do not upload complex directory structures and files that do not contain sequence data!
The submission folder is a temporary holding area and it will be removed after the whole submission is finished processing (the button is clicked).
Firewall Requirements
UDP (User Datagram Protocol for Aspera Connect) transfer must be enabled for the following IP ranges (please consult your local network admin if you are unsure): 130.14.29.0/24 and 130.14.250.0/24. TCP (Transmission Control Protocol for FTP) port 22 and UDP port 33001 must be open for our subnet.
Contact SRA staff
Email [email protected] for help.