HTTP protocol hands-on

Here are some instructions / code snippets to play around HTTP.

How to start a simple HTTP server

const http = require('http');

const server = http.createServer((req, res) => {
  if (req.url === '/hello') {
    // Simulate a delay for the response
    setTimeout(() => {
      res.writeHead(200, { 'Content-Type': 'text/plain' });
      res.end('Hello World');
    }, 2000);
  } else {
    res.writeHead(404);
    res.end();
  }
});

const PORT = 8000;
const HOST = 'localhost';
server.listen(PORT, HOST, () => {
  console.log(`Server running at http://${HOST}:${PORT}/`);
});

How to open a TCP connection to a web server (using netcat on Linux)

nc localhost 8000

How to send a HTTP request on top of a TCP connection

(simply write the request: method, path, http version)

As you can see, with HTTP/1.0, the TCP connection is closed by the webserver after each request

Why is HTTP1.1 better than HTTP1.0 ?

It keeps the TCP connection opened for next request

As you can see, with HTTP/1.1 it is possible to use the same TCP connection for multiple requests (sequentially). The webserver decides whenever the connection will be closed (keep-alive headers). it is possible for the client to request specific parameters if needed

How to get the response as chunks

if the response is big, it could be beneficial to sent that big response chunk by chunk (for both the webserver and the client). Typical use cases would be video streaming (HTTP Live Streaming (HLS)).
Here is how a webserver can send a first chunk, wait 2 sec and send a second chunk to the client.

const http = require('http');

const server = http.createServer((req, res) => {
  if (req.url === '/hello') {
    // Inform the client that the response will be chunked
    res.writeHead(200, {
      'Content-Type': 'text/plain',
      'Transfer-Encoding': 'chunked'
    });

    // Send the word "Hello" as one chunk
    res.write('Hello ');

    // Wait 2 seconds before sending the next chunk
    setTimeout(() => {
      // Send the word "World" as another chunk
      res.write('World');

      // End the response, which sends a zero-length chunk to indicate the end of the response
      res.end();
    }, 2000);
  } else {
    res.writeHead(404);
    res.end();
  }
});

const PORT = 8000;
const HOST = 'localhost';
server.listen(PORT, HOST, () => {
  console.log(`Server running at http://${HOST}:${PORT}/`);
});

Request and response from client

How to encrypt traffic in transit with HTTPS ?

If you want to use HTTPS instead of HTTP, you can use openssl

When you use HTTPS, there are extra steps for the TCP handshake:
- The webserver will send back its certificate

openssl s_client -connect google.com:443

Some webserver wants to get the HTTP request on the TCP connection quickly (or they close the connection)

Keep in mind that webservers can decide whenever they want to close the TCP connection (maybe due to to delays, incorrect headers used, bot protection, etc …)
If you try to craft your own web requests with openssl against well known websites, chances are that you will struggle 😁

How to determine if HTTP2.0 is supported ?

you can ask to use HTTP 2.0 as a client during the TLS handshake (“ALPN” = Application Layer Protocol Negotation)

curl -I --http2 https://www.example.com

HTTP1.1 vs HTTP2.0 relationship with TCP connection

In HTTP1.1, multiple requests will be sent sequentially on the same TCP connection.
In HTTP2.0, multiple requests will be sent in parallel on the same TCP connection.

sudo tcpdump -i eth0 -n 'tcp[tcpflags] == tcp-syn and port 443'

This command in your terminal will make a new line appear whenever there is a new TCP connection opened to a remote server on port 443.

Now you can test 2 requests in parallel with HTTP1.1 (= 2 TCP connections)

curl -Z --http1.1 \
     -w "Time for resource1: %{time_total}\n" -o /dev/null https://www.google.com \
     -w "Time for resource2: %{time_total}\n" -o /dev/null https://www.google.com

And you can test 2 requests in parallel with HTTP2.0 (=1 TCP connection)

curl --http2 -Z \
     -w "Time for resource1: %{time_total}\n" -o /dev/null https://www.google.com \
     -w "Time for resource2: %{time_total}\n" -o /dev/null https://www.google.com