skillfindtech

What is HTTP?

HTTP (Hypertext Transfer Protocol)

HTTP (Hypertext Transfer Protocol) is a set of rules and procedures (a protocol) governing the actions of creating, reading, updating, and deleting World Wide Web hypermedia resources through exchanges of messages between two separate processes (instances of computer programs) where the process that initially sends the request message is called the client, and the process which sends the response message is called the server.

HTTP within Internet Protocol Suite

HTTP is part of the application layer (a software layer) within the Internet protocol suite (commonly referred to as TCP/IP).

Generally, TCP/IP includes the following layers:

  • the application layer - process-to-process communication, currently includes HTTP, FTP, SSH, and SMTP,
  • the transport layer - host-to-host (device-to-device) communication, governed by the rules of the Transmission Control Protocol (TCP) in HTTP/2 and User Datagram Protocol (UDP) in HTTP/3,
  • the internet layer - network-to-network communication, internetworking, and
  • the link layer - link-to-link communication over a single network.

HTTP Origins

TCP/IP, of which HTTP is part, was incepted in the late 1960s by Defense Advanced Research Projects Agency (DARPA) which created Advanced Research Projects Agency Network (ARPANET). In the 1980s versions of TCP/IP were adopted by IBM, AT&T, and other companies. In 1989 the University of California, Berkeley placed the TCP/IP code developed for BSD Unix in the public domain.

The development of HTTP itself was initiated by Sir Tim Berners-Lee at CERN in 1989 with version 0.9. It coincided with the development of HTML i.e. HyperText Markup Language.

The first HTTP server went live in 1990.

HTTP Versions

HTTP/0.9 (HTTP version 0.9), which was the first HTTP version, had only one method GET which could only retrieve HTML documents. HTTP/0.9 is currently considered obsolete.

HTTP/1 was finalized in 1996. It added HEAD and POST methods. In HTTP/1 a separate connection had to be made for each request and response message exchange. HTTP/1 is currently considered obsolete.

HTTP/1.1 was released in 1997. It added the PUT, DELETE, CONNECT, OPTIONS, and TRACE methods. In HTTP/1.1 a single connection can be reused for multiple request and response message exchanges. HTTP/1.1 is still used currently.

HTTP/2 was introduced in 2015. It is based to some extent on Google's SPDY binary protocol and enhances the speed of message exchange however generally retains compatibility with HTTP/1.1. Although HTTP/2 does not require encryption, the major browsers do for HTTP/2 message exchanges, making its usage with HTTP/2 de facto compulsory. HTTP/2 is currently the prevailing standard of HTTP.

HTTP/3 is currently under development. HTTP/3 generally uses the same message methods, status codes, and fields as HTTP/2. The main reason behind HTTP/3 development is increasing the speed of message exchange. It's being attained through employing a new transport layer (host-to-host) protocol called QUIC (based on the User Datagram Protocol) instead of the Transmission Control Protocol (TCP).

HTTPS (Hypertext Transfer Protocol Secure) is an extension to HTTP providing secure communication between processes using Transport Layer Security (TLS) or, previously, Secure Sockets Layer (SSL).

HTTP Statelessness

HTTP is stateless which means that no previous state is ever needed for an execution of a given request and response messages exchange.

The above does not mean that no data can be retained between message exchanges. Indeed, it can. The data can be retained between message exchanges in the form of HTTP cookies which are sent in responses by the server, stored by the client, and appended to the client's subsequent requests.

Connections & Ports

Before a request and response messages exchange is performed the client establishes a connection with the server.

Commonly used port for non-encrypted connections is 80, and for encrypted connections 443.

What is URL?

URL (Uniform Resource Locator)

URL (Uniform Resource Locator) - colloquially known as the web address - is a unique sequence of characters that provides means for locating digital information resources (digitally preserved intangible creations of the human intellect) over a network, including, but not limited to, the Internet, which then can be retrieved using various transfer protocols such as HTTP (Hypertext Transfer Protocol) for hypertext documents, FTP (File Transfer Protocol) for files, and mailto for emails.

URL as URI (Uniform Resource Identifier)

URL is a kind of URI (Uniform Resource Identifier).

URI (Uniform Resource Identifier)

URI (Uniform Resource Identifier) is a unique sequence of characters that provides means for identifying physical resources, such as physical objects, places, and people, and logical resources (intangible creations of the human intellect), such as ideas, concepts, written documents, books, songs, movies, games, and digital information resources.

URL Origins

The term URL was defined by the inventor of the World Wide Web, Tim Berners-Lee, in 1994.

The conceptualization of URL is interlinked with the development of the Hypertext Transfer Protocol (HTTP) initiated by Tim Berners-Lee at CERN in 1989. See What is HTTP? for further reading.

URI Syntax

As noted above, URL is a kind of URI.

The generic URI syntax is comprised of five components:

  • scheme (required),
  • authority (optional),
  • path (required),
  • query (optional), and
  • fragment (optional).
URI components
scheme://authority/path?query#fragment

The authority component is comprised of three subcomponents:

  • userinfo (optional),
  • host (required), and,
  • port (optional).
URI with userinfo, host & port subcomponents
scheme://userinfo@host:port/path?query#fragment

The userinfo subcomponent can be further divided into the following subcomponents:

  • username (required), and
  • password (optional).
Full URI syntax
scheme://username:password@host:port/path?query#fragment

It is worth remembering that the only required URI components are scheme and path.

URL Syntax

URL observes the generic URI syntax however the scheme component is referred to as the protocol. The protocol component can include such values as file, http, https, ftp, and mailto.

Therefore, an URL can be comprised of five components:

  • protocol (required),
  • authority (optional) consisting of optional userinfo, required host and optional port,
  • path (required),
  • query (optional), and
  • fragment (optional).

Of those five components, only protocol and path are required.

An URL referring to a local file is an example of the URL with only protocol and path specified.

An URL with only protocol and path specified
file:///Users/johndoe/Documents/mystuff.json

URL Syntax.

URL Protocol

URL protocol

URL protocol - the counterpart of URI scheme - is the first of the two required URL components (the second being the path) which denotes a set of rules and procedures governing the manner in which the URL resource should be accessed.

A common protocol value can be file, http (together with its encrypted counterpart https), ftp and mailto.

URL Authority

URL authority

URL authority is an optional URL component which provides directions to a server or servers from which the underlying URL resource is to be served, and optionally user authentication while accessing those servers and port through which those servers should be accessed.

The URL authority component is comprised of three subcomponents:

  • userinfo (optional),
  • host (required), and
  • port (optional).

URL Authority Userinfo

URL authority userinfo

URL authority userinfo is an optional subcomponent in optional URL authority component which provides required username and optional password for the purpose of authentication while accessing the URL host.

The data in the password subcomponent is provided as plain text and therefore its usage is advised against or even considered deprecated.

URL Authority Host

URL authority host

URL authority host is a required subcomponent in the optional URL authority component which provides directions to a server or servers from which the underlying URL resource is to be served.

The host component can be:

  • a registered domain name (e.g. example.net), which can be prefixed with an optional subdomain (e.g. www), or
  • an IP address (e.g. 127.0.0.1).

In the registered domain name the last subcomponent (e.g. com, net, co.uk) is referred to as the domain suffix.

A registered domain name is mapped to one or many IP addresses by Domain Name System (DNS).

On the other hand, many different registered domain names can map to one IP address when the so-called virtual hosting is involved.

An URL with a registered domain name as the host
https://www.example.net/movies/?director=scott
An URL with an IP as the host
http://127.0.0.1:8080/users

URL Authority Port

URL authority port

URL authority port is an optional subcomponent in the optional URL authority component which identifies the client process which accesses the URL host.

Some port numbers are deemed reserved and are supposed only to be used by processes using specific protocols.

Commonly used port for web servers is 8080, however when no port is specified HTTP uses implicitly port 80, and HTTPS uses implicitly 443. Those port numbers are reserved for those purposes.

An URL with the port number
https://example.net:443

URL Path

URL path

URL path is the second of the two required URL components (the first one being the protocol) which denotes the logical location to the contemplated URL resource (whereas the protocol denotes the rules and procedures regarding the manner of resource access).

Path can consist of one segment (e.g. /movies) or many segments (e.g. /movies/comedies).

URL Query

URL query

URL query is an optional URL component which denotes parameters that are to be used during the URL resource access.

In an URL the query component is preceded with the question mark (?).

The query syntax is not clearly defined but usually is a sequence of key-value pairs separated by the ampersand (&) delimiter.

An URL with query params
https://www.example.net/movies?director=scott&year=1982

URL Fragment

URL fragment

URL fragment is an optional URL component which denotes the logical location to the URL secondary resource such as an element in HTML document specified by the ID attribute.

In an URL the fragment subcomponent is preceded by the hash symbol (#).

An URL with query params and fragment
https://www.example.net/movies?director=scott&year=1982#actors

URLs in HTTP Requests

URLs are used in HTTP requests to indicate resources being accessed.

Resources that can be accessed using HTTP and URLs are primarily HTML documents.

URL Converting & Encoding

A given URL can only be for data transmission over the Internet when it consists of URL safe ASCII characters.

A non-ASCII character to be used within an URL needs to be converted to the so-called Punycode consisting only of ASCII safe characters.

An example of a non-ASCII character is ♥ which - to be used within an URL - needs to be converted into xn--g6h.

An unsafe ASCII character to be used within an URL needs to be encoded to a set of safe ASCII characters consisting of % prefix followed by a hexadecimal number.

An example of an unsafe ASCII character is the space character which needs to be encoded into + or %20.

IRI (Internationalized Resource Identifier)

IRI (Internationalized Resource Identifier)

IRI (Internationalized Resource Identifier) is an URL which for internationalization purposes can consist of Unicode characters (as opposed to standard URL safe ASCII characters).

Most modern browsers support IRIs.

For the purpose of Internet transmission, IRI non-ASCII characters are converted into the so-called Punycode which consists only of ASCII characters.

An example of a Unicode non-ASCII character is 人 which needs to be converted into xn--gmq.

A domain name in IRI with internationalized characters is known as an Internationalized Domain Name (IDN).

Messages Between Processes in HTTP

The process requesting a CRUD action (i.e. an action of creating, reading, updating, and/or deleting) a World Wide Web hypermedia resource, called a client, sends a request message to a separate process, called a server, which responds with a response message.

The process which typically requests a CRUD action is a user agent such as a web browser on the basis of user interactions with that browser. Other user agents include mobile apps, voice browsers, and web crawlers.

A request message includes:

  • URL, which stands for Uniform Resource Locator,
  • method,
  • headers, and
  • optional body.

A response message includes:

  • status,
  • headers, and
  • optional body