[Node.js] Single-threaded Event Loop

1) A simple analogy

What is single-threaded event loop? Hope the following analogy of a doctor visit will help you! For this analogy, we are doing comparison in context of server-side web architectures.

In a traditional thread-model, when you get to a receptionist, you stand and fill out your form for as long as it takes. While you are filling out your form, the receptionist just sits and wait for you; unable to serve other people behind you.

The only way to scale is to add more receptionists, which is costly - both in terms of labor cost and room allocation for the receptionist to sit.

In an event-based system (which is what single-threaded event loop relies on), when you get to the receptionist, you are given a form to fill and told to come back after you have completed it. You sit down and fill the form as the receptionist helps the next period in line. You are not blocking the receptionist from serving the next person.

When you are done, you line up again and hand in your form. If you have done something wrong, you repeat the earlier steps again. 

This system is more scalable (in fact, used in most doctor's visit). You can further increase performance by adding an additional receptionist. 

2) Problems NodeJS is trying to fix (via Event loop)

There are 2 main problems with the traditional multi-threaded client-per-connection server model that NodeJS attempts to solve:

2.1) I/O operations are expensive

More importantly, it's a waste of cpu cycles as it's waiting because it is not really doing anything. NodeJS' approach is to use asynchronous I/O operations via callbacks/promises/async&await.

2.2) Thread-per-connection is memory-expensive

Apache is multi-threaded and spawn thread per request. The problem is that it eats up the memories as the number of concurrent connections increase. Nginx and NodeJS are not multi-threaded, because threads (and processes) can carry a heavy memory cost. They are single-threaded, but event-based. This also eliminates the overhead of creating thousands of threads.

3) 8 Phases of Event Loop Cycle

To begin with, I will talk about the 6 phases of event loop and explain the last 2 later on. Each phase is associated with a queue of callbacks. That is the queue nodeJS will read from for a given phase.
  1. Timers
    1. Reads from a minMax heap of timer callbacks.
    2. Execute callbacks via setTimeout()[start after some ms] or setInterval()[repeat the callback after some ms.
  2. Pending Callbacks (called 'IO callbacks' in diagram)
    1. Callbacks from this queue are pushed from other phases.
    2. For example, if you try to write something in a TCP handler and the work is done, then the callback is pushed in this queue. 
  3. Idle, Prepare (Not showing in diagram, but exists)
    1. Only used internally.
  4. Check (or Polling) 
    1. Let's assume the queue for this phase is "watch_queue"
    2. If there is something in the "watch_queue", the callbacks are executed synchronously one after another until the queue is empty (or until system specific max limit).
    3. Else (if watch_queue is empty), nodeJS in this phase will wait for new connections/events. The time to 'wait' depends on various factors. 
  5. Set Immediate (or Check)
    1. Callbacks via setImmediate()
  6. Close Events
    1. All .on('close') event callbacks.

3.1) Diagram

3.2) Intermediate Queues

  • Additional to the 6 phases explained, there are 2 phase/queues called "nextTickQueue" and "microTaskQueue". As soon as one phase is completed, it will check these 2 (intermediate) phases and process them. 
  • nextTickQueue happens before microTaskQueue.
  • The actual flow is actually: Timer -> nextTickQueue -> microTaskQueue -> Pending -> nextTickQueue -> microTaskQueue -> Idle, Prepare -> ...etc

3.2.1) nextTickQueue

  • Callbacks added using process.nextTick function.

3.2.2) microTaskQueue

  • Callbacks as a result of resolved (native) promises.
  • However, if you are using a library promise, such as q or bluebird, you will observe an entirely different result (because they are not native).

Resources


https://nodejs.org/de/docs/guides/event-loop-timers-and-nexttick/
http://blog.mixu.net/2011/02/01/understanding-the-node-js-event-loop/
http://voidcanvas.com/nodejs-event-loop/
https://medium.com/the-node-js-collection/what-you-should-know-to-really-understand-the-node-js-event-loop-and-its-metrics-c4907b19da4c
http://code.danyork.com/2011/01/25/node-js-doctors-offices-and-fast-food-restaurants-understanding-event-driven-programming/
https://jsblog.insiderattack.net/event-loop-and-the-big-picture-nodejs-event-loop-part-1-1cb67a182810

Todo:
https://www.youtube.com/watch?v=zphcsoSJMvM&t=1s
https://www.youtube.com/watch?v=8aGhZQkoFbQ

Comments

Post a Comment

Popular posts from this blog

[Redis] Redis Cluster vs Redis Sentinel

1) In-Depth Guides to Redis Cluster and Sentinel For more in-depth explanation for Redis Cluster or Redis Sentinel, please read my other posts: Sentinel:  https://notafraidofwong.blogspot.com/2019/05/redis-redis-sentinel.html Cluster:  https://notafraidofwong.blogspot.com/2019/05/redis-redis-cluster.html 2) Deciding between Redis Sentinel and Redis Cluster Similarities : Both solutions provide high availability for your system. 2.1) Redis Cluster 2.1.1) Pros Shards data across multiple nodes Has replication support Has built-in failover of master 2.1.2) Cons Not every library supports it May not be as robust (yet) as Standalone Redis or Sentinel Setup and maintenance is more complicated 2.2) Redis Sentinel 2.2.1) Pros Automatically selects new master in case of failure Easy to setup, (seems) easy to maintain 2.2.2) Cons No data sharding; master might be overutilized. Another distributed system to maintain. 3) So how do I decide? In
Read more

[Unit Testing] Test Doubles (Stubs, Mocks....etc)

Image
In unit testing, isolation is key. The goal of unit testing is to test individual components, and not an entire system . The class/object/function you are testing is System Under Test (SUT) , and the other components of the system are Collaborators or Dependencies. Test Double  is a generic term for any kind of 'pretend' object used in place of a real object for testing purposes.  There are several types of Testing Doubles. We will start with: Fake Stub Mock Command Queries Mocks vs Stubs And then: Dummy Spies Spies vs Mocks I) Basic Testing Doubles a) Fake Fake are objects that have working implementations, but not same as production one. Usually they take some shortcut and have simplified version of production code. Note that Fake is a generic term - that can point to anything; usually mock and stubs. An example of this is an in-memory implementation of a database. This fake implementation will not access the actual database, but will u
Read more

[Node.js] Pending HTTP requests lead to unresponsive nodeJS

This is a recap conclusion to the potential problems of a NodeJS program I worked on previously. The program simulates thousands of clients navigating a website by sending thousands of http requests to a given API . The idea is to load test a particular API to make sure they can handle high traffic. Because we are simulating clients, it's important to keep track of the state of the simulated client as well. The problem with this program is it is buggy and becomes unresponsive from time to time (the only solution is to restart it). The problem was fixed by re-writing the entire program in Java, but it was regretful that I never had enough time to find the exact reason the nodeJS version of it failed. The following are potential causes I suspect: 1. Exceptions The NodeJS program was not written very neatly and stacked callbacks on top of callbacks on top of callbacks. This made it very difficult to keep track of exceptions and unexpected http responses are prone to happen. So
Read more