Node.js - Scaling Application

04 Mar

Node.js runs in a single-thread mode, but it uses an event-driven paradigm to handle concurrency. It also facilitates creation of child processes to leverage parallel processing on multi-core CPU based systems.
Child processes always have three streams child.stdin, child.stdout, and child.stderr which may be shared with the stdio streams of the parent process.
Node provides child_process module which has the following three major ways to create a child process.

exec − child_process.exec method runs a command in a shell/console and buffers the output.
spawn − child_process.spawn launches a new process with a given command.
fork − The child_process.fork method is a special case of the spawn() to create child processes.

The exec() method

child_process.exec method runs a command in a shell and buffers the output. It has the following signature −
```
child_process.exec(command[, options], callback)
```
Parameters

Here is the description of the parameters used −
command (String) The command to run, with space-separated arguments
options (Object) may comprise one or more of the following options −
cwd (String) Current working directory of the child process
env (Object) Environment key-value pairs
encoding (String) (Default: 'utf8')
shell (String) Shell to execute the command with (Default: '/bin/sh' on UNIX, 'cmd.exe' on Windows, The shell should understand the -c switch on UNIX or /s /c on Windows. On Windows, command line parsing should be compatible with cmd.exe.)
timeout (Number) (Default: 0)
maxBuffer (Number) (Default: 200*1024)
killSignal (String) (Default: 'SIGTERM')
uid (Number) Sets the user identity of the process.
gid (Number) Sets the group identity of the process.
callback The function gets three arguments error, stdout, and stderr which are called with the output when the process terminates.
cwd (String) Current working directory of the child process
env (Object) Environment key-value pairs
encoding (String) (Default: 'utf8')
shell (String) Shell to execute the command with (Default: '/bin/sh' on UNIX, 'cmd.exe' on Windows, The shell should understand the -c switch on UNIX or /s /c on Windows. On Windows, command line parsing should be compatible with cmd.exe.)
timeout (Number) (Default: 0)
maxBuffer (Number) (Default: 200*1024)
killSignal (String) (Default: 'SIGTERM')
uid (Number) Sets the user identity of the process.
gid (Number) Sets the group identity of the process.
callback The function gets three arguments error, stdout, and stderr which are called with the output when the process terminates.
The exec() method returns a buffer with a max size and waits for the process to end and tries to return all the buffered data at once.

Example

Let us create two js files named support.js and master.js −
File: support.js
console.log("Child Process " + process.argv[2] + " executed." );
File: master.js
const fs = require('fs');
const child_process = require('child_process');
for(var i=0; i<3; i++) {
var workerProcess = child_process.exec('node support.js '+i,function
(error, stdout, stderr) {
if (error) {
console.log(error.stack);
console.log('Error code: '+error.code);
console.log('Signal received: '+error.signal);
}
console.log('stdout: ' + stdout);
console.log('stderr: ' + stderr);
});
workerProcess.on('exit', function (code) {
console.log('Child process exited with exit code '+code);
});
}
Now run the master.js to see the result −
$ node master.js
Verify the Output. Server has started.
Child process exited with exit code 0
stdout: Child Process 1 executed.
stderr:
Child process exited with exit code 0
stdout: Child Process 0 executed.
stderr:
Child process exited with exit code 0
stdout: Child Process 2 executed.
The spawn() Method
child_process.spawn method launches a new process with a given command. It has the following signature −
child_process.spawn(command[, args][, options])
Parameters
Here is the description of the parameters used −
command (String) The command to run
args (Array) List of string arguments
options (Object) may comprise one or more of the following options −
cwd (String) Current working directory of the child process.
env (Object) Environment key-value pairs.
stdio (Array) String Child's stdio configuration.
customFds (Array) Deprecated File descriptors for the child to use for stdio.
detached (Boolean) The child will be a process group leader.
uid (Number) Sets the user identity of the process.
gid (Number) Sets the group identity of the process.
The spawn() method returns streams (stdout &stderr) and it should be used when the process returns a volume amount of data. spawn() starts receiving the response as soon as the process starts executing.
Example
Create two js files named support.js and master.js −
File: support.js
console.log("Child Process " + process.argv[2] + " executed." );
File: master.js
const fs = require('fs');
const child_process = require('child_process');
for(var i = 0; i<3; i++) {
var workerProcess = child_process.spawn('node', ['support.js', i]);
workerProcess.stdout.on('data', function (data) {
console.log('stdout: ' + data);
});
workerProcess.stderr.on('data', function (data) {
console.log('stderr: ' + data);
});
workerProcess.on('close', function (code) {
console.log('child process exited with code ' + code);
});
}
Now run the master.js to see the result −
$ node master.js
Verify the Output. Server has started
stdout: Child Process 0 executed.
child process exited with code 0
stdout: Child Process 1 executed.
stdout: Child Process 2 executed.
child process exited with code 0
child process exited with code 0
The fork() Method
child_process.fork method is a special case of spawn() to create Node processes. It has the following signature −
child_process.fork(modulePath[, args][, options])
Parameters
Here is the description of the parameters used −
modulePath (String) The module to run in the child.
args (Array) List of string arguments
options (Object) may comprise one or more of the following options −
cwd (String) Current working directory of the child process.
env (Object) Environment key-value pairs.
execPath (String) Executable used to create the child process.
execArgv (Array) List of string arguments passed to the executable (Default: process.execArgv).
silent (Boolean) If true, stdin, stdout, and stderr of the child will be piped to the parent, otherwise they will be inherited from the parent, see the "pipe" and "inherit" options for spawn()'s stdio for more details (default is false).
uid (Number) Sets the user identity of the process.
gid (Number) Sets the group identity of the process.
The fork method returns an object with a built-in communication channel in addition to having all the methods in a normal ChildProcess instance.
Example
Create two js files named support.js and master.js −
File: support.js
console.log("Child Process " + process.argv[2] + " executed." );
File: master.js
const fs = require('fs');
const child_process = require('child_process');
for(var i=0; i<3; i++) {
var worker_process = child_process.fork("support.js", [i]);
worker_process.on('close', function (code) {
console.log('child process exited with code ' + code);
});
}
Now run the master.js to see the result −
$ node master.js
Verify the Output. Server has started.
Child Process 0 executed.
Child Process 1 executed.
Child Process 2 executed.
child process exited with code 0
child process exited with code 0
child process exited with code 0
File: support.js
```
console.log("Child Process " + process.argv[2] + " executed." );
```
File: master.js
```
const fs = require('fs');
const child_process = require('child_process');
for(var i=0; i<3; i++) {
var workerProcess = child_process.exec('node support.js '+i,function
(error, stdout, stderr) {
if (error) {
console.log(error.stack);
console.log('Error code: '+error.code);
console.log('Signal received: '+error.signal);
}
console.log('stdout: ' + stdout);
console.log('stderr: ' + stderr);
});
workerProcess.on('exit', function (code) {
console.log('Child process exited with exit code '+code);
});
}
```
Now run the master.js to see the result −
```
$ node master.js
```
Verify the Output. Server has started.
```
Child process exited with exit code 0
stdout: Child Process 1 executed.
stderr:
Child process exited with exit code 0
stdout: Child Process 0 executed.
stderr:
Child process exited with exit code 0
stdout: Child Process 2 executed.
```
The spawn() Method

child_process.spawn method launches a new process with a given command. It has the following signature −
```
child_process.spawn(command[, args][, options])
```
Parameters

Here is the description of the parameters used −
command (String) The command to run
args (Array) List of string arguments
options (Object) may comprise one or more of the following options −
cwd (String) Current working directory of the child process.
env (Object) Environment key-value pairs.
stdio (Array) String Child's stdio configuration.
customFds (Array) Deprecated File descriptors for the child to use for stdio.
detached (Boolean) The child will be a process group leader.
uid (Number) Sets the user identity of the process.
gid (Number) Sets the group identity of the process.
cwd (String) Current working directory of the child process.
env (Object) Environment key-value pairs.
stdio (Array) String Child's stdio configuration.
customFds (Array) Deprecated File descriptors for the child to use for stdio.
detached (Boolean) The child will be a process group leader.
uid (Number) Sets the user identity of the process.

gid (Number) Sets the group identity of the process.
The spawn() method returns streams (stdout &stderr) and it should be used when the process returns a volume amount of data. spawn() starts receiving the response as soon as the process starts executing.

Example

Create two js files named support.js and master.js −
File: support.js

console.log("Child Process " + process.argv[2] + " executed." );

File: master.js

const fs = require('fs');
const child_process = require('child_process');
for(var i = 0; i<3; i++) {
var workerProcess = child_process.spawn('node', ['support.js', i]);
workerProcess.stdout.on('data', function (data) {
console.log('stdout: ' + data);
});
workerProcess.stderr.on('data', function (data) {
console.log('stderr: ' + data);
});
workerProcess.on('close', function (code) {
console.log('child process exited with code ' + code);
});
}

Now run the master.js to see the result −

$ node master.js

Verify the Output. Server has started

stdout: Child Process 0 executed.
child process exited with code 0
stdout: Child Process 1 executed.
stdout: Child Process 2 executed.
child process exited with code 0
child process exited with code 0

The fork() Method

child_process.fork method is a special case of spawn() to create Node processes. It has the following signature −

child_process.fork(modulePath[, args][, options])

Parameters

Here is the description of the parameters used −

modulePath (String) The module to run in the child.
args (Array) List of string arguments
options (Object) may comprise one or more of the following options −
cwd (String) Current working directory of the child process.
env (Object) Environment key-value pairs.
execPath (String) Executable used to create the child process.
execArgv (Array) List of string arguments passed to the executable (Default: process.execArgv).
silent (Boolean) If true, stdin, stdout, and stderr of the child will be piped to the parent, otherwise they will be inherited from the parent, see the "pipe" and "inherit" options for spawn()'s stdio for more details (default is false).
uid (Number) Sets the user identity of the process.
gid (Number) Sets the group identity of the process.
cwd (String) Current working directory of the child process.
env (Object) Environment key-value pairs.
execPath (String) Executable used to create the child process.
execArgv (Array) List of string arguments passed to the executable (Default: process.execArgv).
silent (Boolean) If true, stdin, stdout, and stderr of the child will be piped to the parent, otherwise they will be inherited from the parent, see the "pipe" and "inherit" options for spawn()'s stdio for more details (default is false).
uid (Number) Sets the user identity of the process.
gid (Number) Sets the group identity of the process.

Example

Create two js files named support.js and master.js −
File: support.js
```
console.log("Child Process " + process.argv[2] + " executed." );
```
File: master.js
```
const fs = require('fs');
const child_process = require('child_process');
for(var i=0; i<3; i++) {
var worker_process = child_process.fork("support.js", [i]);
worker_process.on('close', function (code) {
console.log('child process exited with code ' + code);
});
}
```
Now run the master.js to see the result −
```
$ node master.js
```
Verify the Output. Server has started.
```
Child Process 0 executed.
Child Process 1 executed.
Child Process 2 executed.
child process exited with code 0
child process exited with code 0
child process exited with code 0
```
Node.js - Packaging

JXcore, which is an open source project, introduces a unique feature for packaging and encryption of source files and other assets into JX packages.
Consider you have a large project consisting of many files. JXcore can pack them all into a single file to simplify the distribution. This chapter provides a quick overview of the whole process starting from installing JXcore.

JXcore Installation

Installing JXcore is quite simple. Here we have provided step-by-step instructions on how to install JXcore on your system. Follow the steps given below −

Step 1

Download the JXcore package from https://github.com/jxcore/jxcore, as per your operating system and machine architecture. We downloaded a package for Cenots running on 64-bit machine.
```
$ wget https://s3.amazonaws.com/nodejx/jx_rh64.zip
```
Step 2

Unpack the downloaded file jx_rh64.zipand copy the jx binary into /usr/bin or may be in any other directory based on your system setup.
```
$ unzip jx_rh64.zip
$ cp jx_rh64/jx /usr/bin
```
Step 3

Set your PATH variable appropriately to run jx from anywhere you like.
```
$ export PATH=$PATH:/usr/bin
```
Step 4

You can verify your installation by issuing a simple command as shown below. You should find it working and printing its version number as follows −
```
$ jx --version
v0.10.32
```
Packaging the Code

Consider you have a project with the following directories where you kept all your files including Node.js, main file, index.js, and all the modules installed locally.
```
drwxr-xr-x  2 root root  4096 Nov 13 12:42 images
-rwxr-xr-x  1 root root 30457 Mar  6 12:19 index.htm
-rwxr-xr-x  1 root root 30452 Mar  1 12:54 index.js
drwxr-xr-x 23 root root  4096 Jan 15 03:48 node_modules
drwxr-xr-x  2 root root  4096 Mar 21 06:10 scripts
drwxr-xr-x  2 root root  4096 Feb 15 11:56 style
```
To package the above project, you simply need to go inside this directory and issue the following jx command. Assuming index.js is the entry file for your Node.js project −
```
$ jx package index.js index
```
Here you could have used any other package name instead of index. We have used index because we wanted to keep our main file name as index.jx. However, the above command will pack everything and will create the following two files −
index.jxp This is an intermediate file which contains the complete project detail needed to compile the project.
index.jx This is the binary file having the complete package that is ready to be shipped to your client or to your production environment.
index.jxp This is an intermediate file which contains the complete project detail needed to compile the project.
index.jx This is the binary file having the complete package that is ready to be shipped to your client or to your production environment.

Launching JX File

Consider your original Node.js project was running as follows −
```
$ node index.js command_line_arguments
```
After compiling your package using JXcore, it can be started as follows −
```
$ jx index.jx command_line_arguments
```
To know more on JXcore, you can check its official website.

Node.js – Scaling Application

Course Curriculum

Node.js – Scaling Application

The exec() method

Parameters

Example

The spawn() Method

Parameters

Example

The fork() Method

Parameters

Example

Node.js - Packaging

JXcore Installation

Step 1

Step 2

Step 3

Step 4

Packaging the Code

Launching JX File