Streaming Data from Client to Cloud Run Server using HTTP/1
In this post, we’ll create a Cloud Run service using a Node.js server that logs incoming data chunks. We’ll also create a Node.js client that sends chunks of data to the server.
We’ll explore how to stream data from a client to server using HTTP/1 with chunked transfer encoding.
If you are anxious, you can check my git repo where I have all the files here.
Why?
The first thing you might be wondering, is the WHY.
HTTP/2 is generally the preferred option here. It offers significant performance benefits, reduced latency, and improved resource utilisation. But, while Cloud Run offers End-to-End HTTP/2 for improved performance, your application must also be capable of receiving HTTP/2 calls. Unfortunately, this capability isn’t universally available, posing a potential hurdle for some deployments:
- There is a need to support older clients that don’t support HTTP/2 or have known issues with it
- If your application relies on libraries or frameworks that specifically require chunked encoding and are not fully compatible with HTTP/2, using HTTP/1 might be the only option.
We will be using the Transfer-Encoding: chunked header which is a feature of HTTP/1.1 that allows the client to send data in chunks to the server without knowing the total size of the data upfront. It break the data into chunks and sends them separately. The server then reconstructs the data from these chunks.
This is particularly useful when the sender doesn’t know the length of the content when it starts transmitting, which is often the case for dynamic or streaming content.
It’s worth noting that HTTP/2 and HTTP/3 handle data transmission differently, using binary framing and multiplexing to allow multiple requests and responses to be in flight at the same time, which can lead to performance improvements, however you might not be able to change your application to accept HTTP/2.
Keep in mind that the concept of chunked transfer encoding doesn’t directly apply in HTTP/2 and HTTP/3 as it does in HTTP/1.1.
Streaming data directions
Streaming data from the server to the client and from the client to the server are two different concepts, each with its own use cases and techniques.
1. Server to Client (Server-Sent Events): This is typically used when the server has new information that it needs to push to the client.
For example, in a real-time application like a chat app or a live sports update app, the server might need to push new messages or updates to the client as soon as they’re available.
This is done using a technique called Server-Sent Events (SSE) in which the client opens a connection to the server and the server keeps that connection open, pushing updates down the connection whenever they’re available.
2. Client to Server (HTTP Streaming or Chunked Transfer Encoding): This is typically used when the client has a large amount of data to send to the server and wants to start the transmission before all the data is ready.
For example, in a file upload scenario, the client might want to start uploading a large file before the entire file has been read into memory.
This is done using a feature of the HTTP protocol called chunked transfer encoding, where the client sends the data in chunks, and the server processes each chunk as it arrives.
In both cases, the goal is to allow data to be sent and processed incrementally, instead of requiring all the data to be ready at the start of the request. This can lead to better performance and lower memory usage, especially when dealing with large amounts of data.
Remember, the choice between server to client and client to server streaming depends on the specific requirements of your application. In this post we are only focusing on Client to Server Streaming. If you want check server to client stream with gRPC you can check this Google Blog Post.
Server Setup
First, let’s create the server. We’ll use Express, a popular Node.js framework. Our server will listen for POST requests on the /upload endpoint and log any incoming data chunks.
Create the package.json file
{
"name": "stream-test",
"version": "1.0.0",
"description": "",
"main": "client.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1",
"start": "node server.js"
},
"keywords": [],
"author": "",
"license": "ISC",
"dependencies": {
"express": "^4.18.2"
}
}And here’s the server.js for our server:
// server.js
const express = require('express');
const app = express();
app.use(express.raw({ type: '*/*', limit: '5mb' }));
app.post('/upload', (req, res) => {
req.on('data', chunk => {
console.log(`Received chunk: ${chunk}`);
});
req.on('end', () => {
res.send('Upload complete');
});
});
app.listen(3000, () => console.log('Server listening on port 3000'));We can containerise this server using Docker. Here’s a simple Dockerfile for our server:
# Dockerfile.server
FROM node:14
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .
CMD [ "node", "server.js" ]Cloud Run Server
Ok, so let’s create the container images now, push to our Artifact Registry in GCP and deploy to our cloud run service. You might need to create your AR repository first:
REPOSITORY=us-central1-docker.pkg.dev/<GCP_PROJECT>/<REPO>
docker build -t $REPOSITORY/stream-server:1.0 -f Dockerfile.server .Deploy your server
gcloud run deploy stream-server --image $REPOSITORY/stream-server:1.0You should be able to see similar logs after the service is deployed
Now that we have our server running on Cloud Run, we need to get the Cloud Run URL:
export SERVER_URL=$(gcloud run services describe stream-server --region us-central1 --format 'value(status.url)')Client Setup
Let’s now create the client. Our client will send chunks of data to the server upon startup. Remember to replace the <SERVER_URL> in the hostname field with the actual server URL..
Here’s the client.js for our client:
// client.js
const https = require('https');
const options = {
hostname: '<SERVER_URL>',
port: 443,
path: '/upload',
method: 'POST',
headers: {
'Transfer-Encoding': 'chunked'
}
};
const req = https.request(options, (res) => {
res.on('data', (chunk) => {
console.log(`Response: ${chunk}`);
});
});
// write chunks of data to the request
req.write('chunk1');
req.write('chunk2');
req.write('chunk3');
req.end();We can also containerise this client using Docker. Here’s a simple Dockerfile for our client:
# Dockerfile.client
FROM node:14
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .
CMD [ "node", "client.js" ]And now Building the client image and running the code
docker build -t stream-client:1.0 -f Dockerfile.client .
docker run --rm stream-client:1.0
Let’s confirm this is working by now looking into the Cloud Run Logs on the server side:
Hold on, looks like the chunks are not being logged separated. The issue here is that the HTTP protocol doesn’t guarantee that each call to req.write() will result in a separate 'data' event on the server.
The chunks you’re sending are small and sent in quick succession, so they’re likely being received by the server in a single ‘data’ event.
To confirm that streaming is working, you can send a larger amount of data that won’t fit into a single TCP packet. This will force the data to be split into multiple chunks, which should then trigger multiple ‘data’ events on the server.
Let’s fix our client.js file adding a “for loop” to write data, build it again our container and test one more time.
// client.js
const https = require('https');
const options = {
hostname: 'stream-server-kdfodunfwq-uc.a.run.app',
port: 443,
path: '/upload',
method: 'POST',
headers: {
'Transfer-Encoding': 'chunked'
}
};
const req = https.request(options, (res) => {
res.on('data', (chunk) => {
console.log(`Response: ${chunk}`);
});
});
// write a large amount of data to the request
for (let i = 0; i < 1e6; i++) {
req.write(`This is chunk number ${i}\n`);
}
req.end();docker build -t stream-client:2.0 -f Dockerfile.client .
docker run --rm stream-client:2.0It looks like it’s working now, nice!
Conclusion
In this post, we’ve seen how to stream data between two Cloud Run instances using Node.js. This setup allows us to send data in chunks from the client to the server, with the server logging each chunk as it arrives.
Keep in mind that the HTTP protocol doesn’t guarantee that each call to req.write() will result in a separate 'data' event on the server. The chunks we were sending initially were quite small and sent in quick succession, so they’re were being received by the server in a single ‘data’ event.
We could confirm the streaming was working sending a larger amount of data that won’t fit into a single TCP packet. This forced the data to be split into multiple chunks, which then triggered multiple ‘data’ events on the server.
Again, HTTP/2 is generally preferred. Use it if possible
I hope you found this post informative! If you have any questions, please feel free to leave a comment below.
