Reading from a URL using URLConnection Class

In the previous article, we saw how we can create a URL object and get the information of any resource on the internet. But barely getting the state-information is not the true motive of a real world application. To retrieve the information, process it and sending the results back to the server, or just display the required information retrieved from the server is what we are aiming at. Consider, for example a small application which asks for a movie name from user and in turn returns the “imdb” rating of the movie or return all the links related to that movie. All of this can be achieved using the URLConnection class.

What is URLConnection class?

URLConnection is an abstract class whose subclasses form the link between the user application and any resource on the web. We can use it to read/write from/to any resource referenced by a URL object.

There are mainly two subclass that extends the URLConnection class-

• HttpURLConnection: If we are connecting to any url which uses “http” as its protocol, then HttpURLConnection class is used.
• JarURLConnection: If however, we are trying to establish a connection to a jar file on the web, then JarURLConnection is used.
• Once the connection is established and we have a URLConnection object, we can use it to read or write or get further information about when was the page/file last modified, content length etc.

Important Methods

• URLConnection openConnection(): opens the connection to the specified URL.
• Object getContent(): retrieves the content of the URLConnection.
• Map<String,List> getHeaderFields(): returns the map containing the values of various header fields in the http header.
• getContentEncoding(): Returns the value of the content-encoding header field.
• getContentLength(): returns the length of the content header field.
• getDate(): returns value of date in header field.
• getHeaderField(int i): returns the value of ith index of header.
• getHeaderField(String field): returns the value of field named “field” in header

To get the list of all header fields, read this.

• OutputStream getOutputStream(): returns the output stream to this connection.
• InputStream getInputStream(): returns the input stream to this open connection.
• setAllowUserInteraction(boolean): Setting this true means a user can interact with the page. Default value is true.
• setDefaultUseCaches(boolean): Sets the default value of useCache field as the given value.
• setDoInput(boolean): sets if the user is allowed to take input or not.
• setDoOutput(boolean): sets if the user is allowed to write on the page. Default value is false since most of the url dont allow to write.
• Lets look at a sample program, which uses the above methods to display the header fields and also print the source code of entire page on to the console window.

//Java Program to illustrate reading and writing
// in URLConnection Class
import java.io.*;
import java.net.*;
import java.util.ArrayList;
import java.util.Date;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

public class URLConnectionclass
{
public static void main(String[] args)
{
try
{
URL url = new URL("https://www.prutor.ai/java");

//open the connection to the above URL.
URLConnection urlcon = url.openConnection();

//Executing the below line would print the value of
// Allow User Interaction field.
// System.out.println(urlcon.getAllowUserInteraction());

//Executing the below line would print
// the value of Content Type field.
// System.out.println(urlcon.getContentType());

//Executing the below line would print the value
// of URL of the given connection.
// System.out.println(urlcon.getURL());

//Executing the below line would
// print the value of Do Input field.
// System.out.println(urlcon.getDoInput());

//Executing the below line would
// print the value of Do Output field.
// System.out.println(urlcon.getDoOutput());

//Executing the below line would
// print the value of Last Modified field.
// System.out.println(new Date(urlcon.getLastModified()));

//Executing the below line would
// print the value of Content Encoding field.
// System.out.println(urlcon.getContentEncoding());

//To get a map of all the fields of http header
Map<String, List<String>> header = urlcon.getHeaderFields();

//print all the fields along with their value.
for (Map.Entry<String, List<String>> mp : header.entrySet())
{
System.out.print(mp.getKey() + " : ");
System.out.println(mp.getValue().toString());
}
System.out.println();
System.out.println("Complete source code of the URL is-");
System.out.println("---------------------------------");

//get the inputstream of the open connection.
BufferedReader br = new BufferedReader(new InputStreamReader
(urlcon.getInputStream()));
String i;

//print the source code line by line.
while ((i = br.readLine()) != null)
{
System.out.println(i);
}
}

catch (Exception e)
{
System.out.println(e);
}
}
}
Result :

Keep-Alive : [timeout=5, max=100]
null : [HTTP/1.1 200 OK]
Server : [Apache/2.4.18 (Ubuntu)]
Connection : [Keep-Alive]
Last-Modified : [Wed, 16 Nov 2016 06:49:55 GMT]
Date : [Wed, 16 Nov 2016 10:58:34 GMT]
Accept-Ranges : [bytes]
Cache-Control : [max-age=3]
ETag : ["10866-541657b07e4d7"]
Vary : [Accept-Encoding]
Expires : [Wed, 16 Nov 2016 10:58:37 GMT]
Content-Length : [67686]
Content-Type :

Complete source code of the URL is-
--------------------------------------------------

...source code of the page...

Steps involved in the above process:

URL Creation: Create a URL object using any of the constructors given int
this article
Create Object: Invoke the openConnection() call to create the object of URLConnection.
Display the Content: Either use the above created object to display the information about the resource or to read/write contents of the file to console using bufferedReader and InputStream of the open connection using getInputStream() method.
Close Stream: Close the InputStream when done.