Practical Guide To Not Blocking The Event Loop
JavaScript runs in a single-threaded environment with an event loop, an architecture that is very easy to reason about. It’s a continuous loop executing incoming work. Said work can schedule more of it for the future.
while (hasWorkToDo) {
/* Run timers, I/O callbacks,
check for incoming connections,
and so on... */
doWork();
}
Synchronous work runs immediately; asynchronous work runs when there is no synchronous work to left to perform (or simply, “later”). Ideally, the execution profile of the your application should allow the event loop to run frequently in order to perform background work (e.g. accepting new connections, running timers, etc).
gantt
title Ideal Execution Profile
dateFormat HH
axisFormat %H
todayMarker off
section Event Loop
Background Work :milestone, crit, 01, sameline
doWork() :01, 02
Background Work :milestone, crit, 02, sameline
doWork() :02, 03
Background Work :milestone, crit, 03, sameline
doWork() :03, 04
This design implies that performing synchronous work is a Big Deal: for every continuous moment it runs, the event loop cannot perform any work – none!
/* setImmediate registers a
callback on the event loop */
setImmediate(() => {
console.log("This will at some point in the future");
});
/* synchronous work that you
will never see the end of */
findNthPrime(9999999);
gantt
title Blocking Event Loop For a Long Time
dateFormat HH
axisFormat %H
todayMarker off
section User Code
setImmediate() :01, 03
findNthPrime(9999999) :03, 10
console.log(...) :10, 12
section Event Loop
Background Work :milestone, crit, 01, sameline
doWork() :01, 10
Background Work :milestone, crit, 10, sameline
doWork() :10, 12
In server contexts, one such request can block all others indefinitely.
/* If a request to /computePrimes
has been sent, this route will
never respond. It will timeout. */
app.get("/home", () => {
return response("Welcome Home!");
});
app.get("/computePrimes", () => {
/* synchronous work that you
will never see the end of. */
return response(findNthPrime(9999999));
});
gantt
title One Request Blocking Other Requests
dateFormat HH
axisFormat %H
todayMarker off
section Client
GET /home :01, 03
GET /computePrimes :crit, 02, 10
GET /home :active, 04, 12
GET /home :05, 14
section User Code
handle /home :01, 03
handle /computePrimes :crit, 03, 10
findNthPrime(9999999) :crit, 04, 09
handle /home :active, 10, 12
handle /home :12, 14
section Event Loop
Background Work :milestone, crit, 01, sameline
doWork() :01, 03
Background Work :milestone, crit, 03, sameline
doWork() :crit, 03, 10
Background Work :milestone, crit, 10, sameline
doWork() :active, 10, 12
Background Work :milestone, crit, 12, sameline
doWork() :12, 14
There are three solutions to these scenarios.
- Throw more nodes at it
- Refactoring
findNthPrimeto perform work asynchronously - Off-loading
findNthPrimeto another thread
Throw More Nodes At It!⌗
You've Got Mail!
No more than five emails per year, all densely packed with knowledge.
The industry term for “throw more resources at it” is Horizontal Scaling (as opposed to Vertical Scaling, where the name of the game is “throw better resources at it”). One of the features of Node.js that had people excited was built-in support for easy horizontal scaling via Clusters.
The general idea is to run multiple servers in parallel such that if one server is busy, another can take the incoming request. A pitfall with this approach is that it can bury the issue until the load catches up.
In our server implementation, the synchronous is slow to complete. If there is one node, it takes one request to put it out of commission. Scaling up the number of nodes will increase the capacity for those requests by the same number.
This approach is straightforward to implement but doesn’t avoid blocking the event loop; it simply adds more event loops into the mix. As a strategy, it works as long as the rate of incoming requests does not exceed the time it takes to process them.
Refactoring to Perform Work Asynchronously⌗
Asynchronous work is usually not CPU-bound. For example, if it takes 10ms to read a file, it is likely that less than 1ms was spent waiting for the CPU, and the rest was waiting for the disk.
Calculating primes, on the other hand, is entirely CPU-bound: it’s just basic mathematical operations.
On an event loop architecture, a long-running algorithm can be converted to an asynchronous job by chunking the work onto the event loop.
Consider the following findNthPrime implementation:
const findNthPrime = num => {
let i, primes = [2, 3], n = 5;
const isPrime = n => {
let i = 1, p = primes[i],
limit = Math.ceil(Math.sqrt(n));
while (p <= limit) {
if (n % p === 0) {
return false;
}
i += 1;
p = primes[i];
}
return true;
}
for (i = 2; i <= num; i += 1) {
while (!isPrime(n)) {
n += 2;
}
primes.push(n);
n += 2;
};
return primes[num - 1];
}
gantt
title Execution Profile Of findNthPrime()
dateFormat HH
axisFormat %H
todayMarker off
section User Code
findNthPrime() :01, 09
isPrime() :01, 03
isPrime() :03, 05
isPrime() :05, 07
isPrime() :07, 09
section Event Loop
Background Work :milestone, crit, 01, sameline
doWork() :01, 09
Background Work :milestone, crit, 09, sameline
The fundamental goal of this approach is to add gaps between blocks of synchronous execution, allowing the event loop to run while your algorithm executes. Where you want those gaps to appear depends on the performance profile you are looking for. If your algorithm blocks the event loop for over a second, adding gaps anywhere is worthwhile.
In this case, isPrime() does most of the work over multiple iterations. It is already conveniently isolated in a function, which makes it a prime candidate to defer it on the event loop.
gantt
title Target Execution Profile of findNthPrimeAsync()
dateFormat HH
axisFormat %H
todayMarker off
section User Code
findNthPrime() :01, 09
isPrime() :01, 03
isPrime() :03, 05
isPrime() :05, 07
isPrime() :07, 09
section Event Loop
Background Work :milestone, crit, 01, sameline
doWork() :01, 03
Background Work :milestone, crit, 03, sameline
doWork() :03, 05
Background Work :milestone, crit, 05, sameline
doWork() :05, 07
Background Work :milestone, crit, 07, sameline
doWork() :07, 09
Background Work :milestone, crit, 09, sameline
Promisify⌗
The first step is to isolate the portion of the code to move onto the event loop into a Promise:
const isPrime = n => new Promise(
resolve => {
let i = 1, p = primes[i],
limit = Math.ceil(Math.sqrt(n));
while (p <= limit) {
if (n % p === 0) {
return resolve(false);
}
i += 1;
p = primes[i];
}
return resolve(true);
}
)
// ...
while (!await isPrime(n)) {
//...
Turning sync code into a Promise does not make code asynchronous. For code to be asynchronous, it must be called from the event loop. setImmediate accepts a callback to do precisely that:
const isPrime = n => new Promise(
resolve => setImmediate(() => {
let i = 1, p = primes[i],
limit = Math.ceil(Math.sqrt(n));
while (p <= limit) {
if (n % p === 0) {
return resolve(false);
}
i += 1;
p = primes[i];
}
return resolve(true);
}
))
Complete Implementation⌗
const asyncInterval = setInterval(() => {
console.log("Event loop executed");
exCount++;
}, 1);
const findNthPrimeAsync = async num => {
let i, primes = [2, 3], n = 5;
const isPrime = n => new Promise(
resolve => setImmediate(() => {
let i = 1, p = primes[i],
limit = Math.ceil(Math.sqrt(n));
while (p <= limit) {
if (n % p === 0) {
return resolve(false);
}
i += 1;
p = primes[i];
}
return resolve(true);
}
));
for (i = 2; i <= num; i += 1) {
while (!await isPrime(n)) {
n += 2;
}
primes.push(n);
n += 2;
};
return primes[num - 1];
}
To prove that the code is now indeed on event loop, we can try scheduling on tasks on the event loop to see if they get executed:
console.log("Calculating Sync Prime...")
let syncCount = 0;
const syncInterval = setInterval(() => {
console.log("Event loop executed");
exCount++;
}, 1);
const sync = findNthPrime(nth);
console.log("Sync Prime is", sync)
clearInterval(syncInterval);
console.log("Intervals on event loop:", syncCount)
console.log("Calculating Async Prime...")
let asyncCount = 0;
const asyncInterval = setInterval(() => {
console.log("Event loop executed");
asyncCount++;
}, 1);
findNthPrimeAsync(nth)
.then(n => console.log("Async Prime is", n))
.then(() => clearInterval(asyncInterval))
.then(() => console.log("Intervals on event loop:", asyncCount));
It outputs:
Calculating Sync Prime...
Sync Prime is 541
Intervals on event loop: 0
Calculating Async Prime...
Event loop executed
Event loop executed
Event loop executed
Event loop executed
Event loop executed
Event loop executed
Async Prime is 541
Intervals on event loop: 6
For a visual, the execution profile looks like this:
gantt
title Execution Profile Of findNthPrime vs findNthPrimeAsync()
dateFormat HH
axisFormat %H
todayMarker off
section User Code
findNthPrime() :01, 09
isPrime() :01, 03
isPrime() :03, 05
isPrime() :05, 07
isPrime() :07, 09
findNthPrimeAsync() :09, 17
isPrime() :09, 11
isPrime() :11, 13
isPrime() :13, 15
isPrime() :15, 17
section Timers
interval :milestone, active, 11, sameline
interval :milestone, active, 13, sameline
interval :milestone, active, 15, sameline
section Event Loop
Background Work :milestone, crit, 01, sameline
doWork() :01, 09
Background Work :milestone, crit, 09, sameline
doWork() :09, 11
Background Work :milestone, crit, 11, sameline
doWork() :11, 13
Background Work :milestone, crit, 13, sameline
doWork() :13, 15
Background Work :milestone, crit, 15, sameline
doWork() :15, 17
Background Work :milestone, crit, 17, sameline
Off-Loading to Another Thread⌗
The last approach to processing a synchronous job without blocking the main thread is offloading it to another thread entirely. Worker pools optimize this strategy further.
The premise is to have a main thread dispatch a worker:
const nth = 100; // play with this value
const findNthPrimeWorker = num => new Promise(resolve => {
const worker = new Worker(require.resolve('./worker.js'), {
workerData: num
});
worker.on("message", d => resolve(d));
})
findNthPrimeWorker(nth)
with the worker performing the computation and sending the result:
// worker.js
const findNthPrime = num => {
// ...
}
parentPort.postMessage(findNthPrime(workerData));
gantt
title Execution Profile of findNthPrimeWorker()
dateFormat HH
axisFormat %H
todayMarker off
section User Code
findNthPrimeWorker() :01, 09
section Event Loop
Background Work :milestone, crit, 01, sameline
doWork() :01, 02
Background Work :milestone, crit, 02, sameline
doWork() :02, 03
Background Work :milestone, crit, 03, sameline
doWork() :03, 04
Background Work :milestone, crit, 04, sameline
doWork() :04, 05
Background Work :milestone, crit, 05, sameline
doWork() :05, 06
Background Work :milestone, crit, 06, sameline
doWork() :06, 07
Background Work :milestone, crit, 07, sameline
doWork() :07, 08
Background Work :milestone, crit, 08, sameline
doWork() :08, 09
Background Work :milestone, crit, 09, sameline
section Worker Thread
findNthPrime() :01, 09
isPrime() :01, 03
isPrime() :03, 05
isPrime() :05, 07
isPrime() :07, 09
Workers Limitations⌗
Workers are ideal for moving long-running, CPU-bound tasks off the main thread but are not a silver bullet. Their primary limitation is the data that can be sent to them. The restrictions are documented in port.postMessage()
Workers != Magic!⌗
You've Got Mail!
No more than five emails per year, all densely packed with knowledge.
An important callout with Workers is that by dedicating them to CPU-bound tasks, they are limited to the number of available threads. If your server has eight threads, running over eight workers will not make them run faster.
The benefit of Workers isn’t infinite parallelism but the guarantee that your main thread will always be free to perform quick work by off-loading non-so-quick work.