How I wrote Node.js bindings for GNU source-highlight – Pal's Blog
Categories
Software

How I wrote Node.js bindings for GNU source-highlight

GNU source-highlight is a C++ library for highlighting source code in several different languages and output formats. It reads from “language definition” files to lexically analyze the source code, which means you can add support for new languages without rebuilding the library. Similarly, “output format” specification files are used to generate the final document with the highlighted code.

I found GNU source-highlight while looking for a native highlighting library to use in webdoc. I added sources publishing support to webdoc 1.5.0, but Highlight.js more than 6xed webdoc’s single thread execution time; after running Highlight.js with 18 worker threads, the final result was still twice the original execution time. So I decided to finally learn how to write Node.js bindings for native C++ libraries!

The Node.js bindings are available on npm – source-highlight. Here, I want to share the process of creating Node.js bindings for native libraries.

Node Addon API – Creating the C++ addon

The node-addon-api is the official module for writing native addons for Node.js in C++. The entry point for addons is defined by registering an initialization function using NODE_API_MODULE:

#include <napi.h>

Napi::Object Init (Napi::Env env, Napi::Object exports) {
    // TODO: Initialize module and expose APIs to JavaScript

    return exports;
}

NODE_API_MODULE(my_module_name, Init);

Here, the Init function accepts two parameters:

  • env – This represents the context of the Node.js runtime you are working with.
  • exports – This is an opaque handle to module.exports. You can set properties on this object to expose APIs to JavaScript code.

To expose a hello-world “function”, you can set a property on the exports object to a Napi::Function.

// hello_world.cc

// Disable C++ exceptions since we are not using. Otherwise, we'd need to configure
// node-gyp to install them when compiling.
#define NAPI_DISABLE_CPP_EXCEPTIONS

#include <iostream>
#include <napi.h>

// Our C++ hello-world function. It takes a single JavaScript
// string and outputs "Hello <string>" to the standard output.
//
// It returns true on success, false otherwise.
Napi::Value helloWorld(const Napi::CallbackInfo& info) {
    // Extract our context early so we can use it to create primitive values.
    Napi::Env env = info.Env();

    // Return false if no arguments were passed.
    if (info.Length() < 1) {
        std::cout << "helloWorld expected 1 argument!";
        return Napi::Boolean::New(env, false);
    }

    // Return false if the first argument is not a string.
    if (!info[0].IsString()) {
        std::cout << "helloWorld expected string argument!";
        return Napi::Boolean::New(env, false);
    }

    // Convert the first argument into a Napi::String,
    // then cast it into a std::string.
    std::string msg = (std::string) info[0].As<Napi::String>();

    // Output our "hello world" message to the standard output.
    std::cout << "Hello " << msg;

    // Return true for success!
    return Napi::Boolean::New(env, true);
}

Napi::Object Init (Napi::Env env, Napi::Object exports) {
    // Wrap helloWorld in a Napi::Function
    Napi::Function helloWorldFn = Napi::Function::New<helloWorld>(env);

    // Set exports.helloWorld to our hello world function
    exports.Set(Napi::String::New(env, "helloWorld"), helloWorldFn);

    return exports;
}

NODE_API_MODULE(hello_world_module, Init);

helloWorld uses the the Napi::Boolean wrapper to create boolean values. The wrappers for all JavaScript values are listed here. All of these wrappers extend the abstract type Napi::Value.

Instead of declaring a Napi::String parameter and returning a Napi::Boolean directly, helloWorld accepts CallbackInfo reference and returns a Napi::Value. Using this generic signature is required to wrap it in Napi::Function. The CallbackInfo contains the arguments passed by the caller in JavaScript code. To protect from the native code from throwing an exception, the function validates the arguments.

After creating the binary module for this addon, it should be useable from JavaScript:

// hello_world.js

const { helloWorld } = require('./build/Release/hello_world');

helloWorld('world, you did it!');

node-gyp – Building the binary module from C++ code

node-gyp is a cross-platform tool for compiling native Node.js addons. It uses a fork of the gyp meta-build tool – “a build system that generates other build systems”. More specifically, node-gyp will configure the build toolchain specific to your platform to compile native code. Instead of creating a Makefile for Linux, Xcode project for macOS, and a Visual Studio project for Windows – you need to create a single binding.gyp file. node-gyp will handle the rest for you; this is particularly useful when you want the native code to compile on the user’s machine and not serve prebuilt binaries.

To use node-gyp and node-addon-api, you’ll need to create an npm package (run npm init). Then install node-addon-api locally and node-gyp globally,

npm install --save node-addon-api
npm install -g node-gyp

Now, to build the example hello-world addon, we’ll need a very simple binding.gyp configuration:

{
    "targets": [
        {
            "target_name": "hello_world",
            "sources": ["hello_world.cc"],
            "include_dirs": [
                "<!@(node -p \"require('node-addon-api').include_dir\")"
            ]
        }
    ]
}

This configuration defines one build-target: our “hello_world” addon. We have one source file “hello_world.cc”, and we want to include the header files provided by node-addon-api. Here, the <!@(...) directive tells node-gyp to evaluate the code ... and use the resulting string. node-addon-api exports the include_dir variable, which is the path to the directory containing its header files.

We can finally run node-gyp,

# This will create the Makefile/Xcode/MSVC project
node-gyp config

# This will invoke the platform-specific project's build toolchain and build the addon
node-gyp build

You can also run node-gyp rebuild instead of running two commands. node-gyp should now have created the binary module for the hello_world addon at ./build/Release/hello_world.node. You can require or import it like any other Node.js! Run the hello_world.js file to test it!

I created this Replit so you can run the hello_world addon right in your browser!

Linking your C++ addon with another native library

Now that we’ve created a simple addon, we want to write bindings for another library. There are two ways to do this:

  • Include the sources of the native library in your repository (using a git submodule) and include that in your node-gyp sources. node-libpng does this. It has two additional gyp configurations in the deps/ folder for compiling libpng and zlib. Since compiling libraries can slow down an npm install, node-libpng prebuilds the binaries and its install script downloads them. sharp also does this and falls back to locally compiling libvips if there isn’t a prebuilt binary for the client platform.
  • Statically link to library preinstalled on the client machine. The downside of this approach is that your user must install the native library before using your bindings. This might be necessary if the native library uses a sophisticated build system that’s hard to replicate using node-gyp. I did this for node-source-highlight because source-highlight depends on the Boost library.

To statically link to a preinstalled copy of the library on the client machine, you can add the following snippet to your addon target in binding.gyp:

"link_settings": {
    "libraries": [
        "-l<name>"
    ]
}

where <name> is the “name” in “libname” of the library you are linking. For example, you would use “-lsource-highlight” to link to “libsource-highlight”. Now, assuming you’ve correctly installed the native library on your machine, you can use its headers in your C++ code.

You can wrap the underlying APIs of a native library and expose them to JavaScript. In node-source-highlight, the SourceHighlight class wraps an instance of srchilite::SourceHighlight.

// SourceHighlight.h
#include <napi.h>

class SourceHighlight : public Napi::ObjectWrap<SourceHighlight> {
  public:
      static Napi::Object Init(Napi::Env env, Napi::Object exports);
      Napi::Value initialize(const Napi::CallbackInfo& callbackInfo);
  private:
      srchilite::SourceHighlight instance;
}

// SourceHighlight.cc
#include <napi.h>
#include "SourceHighlight.h"

Napi::Value SourceHighlight::initialize(const Napi::CallbackInfo& info) {
   this->instance.initialize();
   return info.Env().Undefined();
}

Napi::Object SourceHighlight::Init(Napi::Env env, Napi::Object exports) {
    Napi::Function fn = DefineClass(env, "SourceHighlight", {
       InstanceMethod("initialize", &SourceHighlight::initialize)
   });

   Napi::FunctionReference* constructor = new Napi::FunctionReference();
   *constructor = Napi::Persistent(func);

   env.SetInstanceData(constructor);

   exports.Set("SourceHighlight", func);

   return exports;
}

// Initialize native add-on
Napi::Object Init (Napi::Env env, Napi::Object exports) {
    SourceHighlight::Init(env, exports);
    return exports;
}

NODE_API_MODULE(sourcehighlight, Init);

In this snippet, SourceHighlight::Init does the heavy lifting of creating a class constructor function and attaching it to the exports. The SourceHighlight class holds the underlying srchilite::SourceHighlight instance and each method invokes the corresponding method on that instance after validating the arguments passed.

The full sources are available here.