How to Use cURL to Download a File on Linux?

In the world of command-line tools, cURL is popularly used to transfer data using various network protocols. One of its most common use cases is downloading files from the internet.

Apart from using the -o parameter, there are other multiple ways to download files with cURL. 

Whether you're a system admin, a developer, or an enthusiast, you need to understand how to use cURL effectively to enhance your productivity. 

In this comprehensive guide, we will explore the fundamentals of cURL and provide step-by-step instructions on how to download files using this cURL tool to meet different requirements. 

What is cURL?

cURL, short for “client for URLs,” is a command-line tool and library that transfers data with URLs. It’s included in the libcurl library project. This open-source utility supports multiple protocols, such as FTP, HTTP, HTTPS, FTPS, SCP, SFTP, LDAP, and SMTP. 

Because of its versatility and ease of use, this command line utility is available on various operating systems. It is a popular choice for developers and system administrators working in diverse environments. Its seamless functionality and portability make it a tool that consistently delivers results without requiring extensive configuration. 

Additionally, since cURL is pre-installed on numerous systems, it eliminates the need for additional installations and ensures immediate accessibility across various platforms. It has solidified its position as the go-to tool for API documentation due to its ability to effortlessly perform tasks related to data transfer. 

Prerequisites

Before diving into the comprehensive guide on how to use cURL to download a file, there are a few prerequisites you should have. 

• A working operating system. Whether you're using Linux, macOS, or Windows, cURL is compatible with various operating systems. Ensure you have a working terminal or command prompt on your system.
• Confirm if cURL is already installed on your system. Most Linux distributions and macOS come with cURL pre-installed. For Windows, you might need to install it separately. 

To confirm if cURL is installed, use the -V command. If it's not installed, you can visit the official cURL website for installation instructions. Additionally, if your Linux operating system doesn’t have cURL, you can download it easily by using the package manager.

• A stable internet connection is necessary as cURL downloads files from the web. Ensure your internet connection is reliable to prevent interruptions during the download process. Since cURL supports multiple protocols, ensure that you are connected to a server that allows you to download files from one of these protocols.
• An optional text editor. While it's not mandatory, having a text editor can help you create and execute scripts using cURL commands. This allows you to write, edit, and save scripts for automated downloads.

A Step-By-Step Guide: How To Use cURL To Download a File

Here’s how to download a file with the cURL command.

Download a File Using cURL 

You can use cURL to download and save a file using the remote name. This is a straightforward process. Open your terminal or command prompt and use the following syntax:

Replace the URL part with the actual web address of the file you want to download. The -O option instructs cURL to save the file with its original name.

Apart from the -O option, you can also use the --remote-name option to save your output to a local file. Once you input this command, your files will be automatically downloaded and saved in the present working directory.

How To Use cURL Download Command To Save a File With Another Name

While the -O cURL download command saves your files using the same name from your server, you can opt to specify the output file name. 

To save the downloaded file with a different name, use the -o or - -output option followed by the desired filename. Use this basic syntax: 

In this command, replace output_filename with the name you want to give to the downloaded file.

How To Use a Non-Default Port

You can define a non-default port when using cURL to download a file. Most servers operate on standard ports, such as port 80 for HTTP, port 21 for FTP, and port 443 for HTTPS. 

However, many web servers host services on non-standard ports for various reasons, such as enhancing security or avoiding conflicts with other applications. In this case, you can use cURL to specify a non-default port, which helps you download files seamlessly.

To specify a non-default port, you need to use cURL to add the port number to the URL you are trying to access. This is added at the end of the IP address or server’s domain name. The syntax is as follows:

In this command above, replace http://example.com with the actual URL, PORT_NUMBER with the non-default port number, file.txt with the name of the file you want to download, and output_filename with the desired name for the downloaded file.

For example, if you're accessing a web server running on port 8080, the command would look like this:

How To Use cURL Download Commands To Restart Interrupted Downloads

When you have a weak network connection, it becomes challenging to download large files. Your downloads are more likely to be interrupted.

However, cURL has download commands that enable you to resume interrupted operations by using the -C - option. To resume a download, use the following syntax:

This command will resume the download from where it left off.

How To Download Multiple Files Using One Command 

Users can download multiple files using a single command by specifying multiple URLs. For example:

This command will download all three files specified (URL1, URL2, and URL3) into your current working directory.

Alternatively, if you have many URLs, you should have all of them on a text file and opt for the the xargs command together with cURL. 

How To Limit Download Speed

If you want to limit the download speed to avoid consuming too much bandwidth, you can use the --limit-rate option followed by the desired speed limit. You can indicate the maximum number of bytes downloaded per second. 

The rate can be abbreviated either using kilobytes (k or K), megabytes (m or M), and gigabytes (g or G).

For instance, to limit the download speed to 1mpbs, use the following command:

Remember that if you run cURL with the --limit-rate option, the speeds may be higher than the specified rate; however, this eventually levels off.

How To Terminate Slow Downloads

In cases when the download is slow, you can interrupt this connection automatically. Use the -y (--speed-time) or -Y ( -speed-limit) options to stop this download.

The -Y ( --speed-limit) specifies the speed, while the -y (or --speed-time) option defines the time, measured in seconds. cURL will abort the download if the speeds are lower than the defined speed and time. 

For example, to set a maximum download time of 30 seconds and a speed limit of 100 kilobytes per second, use the following command:

In this command, if the download speed falls below the specified limit (100K) and the download time exceeds 30 seconds, cURL will automatically abort the download.

How To Display The Download Status Bar

The -# or --progress-bar option in cURL enables the display of a download status bar. When you include this option in your cURL command, it replaces the regular download progress information with a dynamic progress bar, giving you a clear visual representation of the download progress. 

The command syntax for the progress bar option is as follows:

Replace output_filename with the desired name for the downloaded file and the URL with the actual web address of the file you want to download.

How To Follow Redirects With cURL

Sometimes, the file you want to download might be hosted on a different URL that redirects to the actual file location. To automatically follow redirects, use the -L or --location option:

This option instructs cURL to follow all redirections until it reaches the final destination and then download the file.

How To Use The Standard Authentication

If your file requires authentication, you can have a username and password. To achieve this, use the -u option. Follow this command:

How To Enable Insecure TLS/SSL Certificates

If a server has a self-signed or invalid certificate, use the -k parameter so that cURL can download a file without verifying your SSL/ TLS certificates.

Here is the simple syntax to use: 

How To Use a Proxy To Download a File

You can also use a proxy to download your files. To do this, use the -x parameter and then add the proxy address. This is how it should appear: 

How To Enter The Silent Mode

Sometimes, you might not want cURL to show an error or progress message. To enter the silent mode, use the --silent or -s option. Follow this syntax:

Why Use cURL?

The versatility and robustness of cURL make it an ideal tool of choice for a wide range of users, including developers and system administrators. Its popularity is due to these advantages:

Extensive Protocol Support

One of cURL's standout features is its ability to handle diverse protocols. Whether you need to download files via HTTP, HTTPS, FTP, FTPS, SCP, SFTP, LDAP, or other protocols, cURL has you covered. 

This flexibility means you can seamlessly interact with different services and platforms, ensuring that your download requirements are met. 

Seamless Automation

The command-line nature of cURL makes it highly conducive to automation. For developers, this means that your repetitive tasks can be streamlined and executed with minimal manual intervention. 

Users can save substantial time by incorporating cURL commands into scripts and automated processes. This allows them to focus on more complex and critical aspects of their work.

Customization

cURL offers multiple options and parameters, providing users with a high degree of customization. Whether you need to set specific timeouts, limit download speeds, or handle redirects, cURL's extensive configuration choices ensure that your download requests can be tailored precisely to your unique requirements. 

This level of customization empowers users to adapt cURL to fit their specific use cases. 

Enhanced Security Features

Security is essential, especially when dealing with sensitive data transfers. cURL addresses this concern by supporting secure protocols like HTTPS and FTPS. 

When downloading files that contain confidential or private information in your company, cURL ensures that the data transfers occur over encrypted connections. This will safeguard the integrity and confidentiality of the information being exchanged. 

This security assurance is vital in professional and enterprise settings where data protection is a top priority.

Summary

Mastering cURL for file downloads is a valuable skill for developers, system administrators, and anyone working with data transfers. This guide covered how to use cURL to download a file for common use cases. 

Various cURL download commands and options allow you to download files with different specifications, resume interrupted downloads, download multiple files, limit download speeds, and handle redirects effectively.

If you harness the power of cURL, you can automate tasks, streamline workflows, and efficiently manage file downloads from the command line.