Plugin Engine¶

ONO Plugin Engine enables you to extend and customize your ONO Lean Logistics cluster by writing simple plugins in JavaScript.

Features¶

Handling chain
Chain priority
Asynchronous code
Access main database
Persistent data
NPM dependencies
Message payload
Time based events

Architecture¶

The plugin engine gives plugins the chance to alter requests before they are handled by the controller and to alter responses before they are sent back to the client.

Every time the controller receives a request, it transfers it to the plugin engine and fires an event which will cause the engine to process it, potentially altering the request. Then the request is sent back to the controller which will properly handle it and generate a response. The controller transfers the response to the plugin engine firing yet another event which will cause the response to be altered by plugins. After that the response is finally sent back to the client.

`Before` and `After` events¶

For every request the controller receives, two plugin engine events are generated: one before the controller handles it and one after. The former event has a chance to alter the request object, the latter can alter the response.

Therefore, for each request the controller is able to handle, we can define two events:

BeforeX

Thrown before the controller handles the request (It's a pre-processor). Plugins can alter the request object during an event of this type.
AfterX

Thrown after the controller has handled the request (It's a post-processor). Plugins can alter the response object during an event of this type.

Handlers¶

Each plugin must advertise and register itself to the controller and declare some handlers. Handlers are functions associated with an event identified by its name and they are invoked each time an instance of that event is thrown.

Multiple plugins¶

The same event could be associated with multiple handlers, each one coming from a different plugin. When an instance of that event gets thrown, the engine will loop through each plugin that declared an handler to that event, and execute its handler.

The order in which handlers from different plugins are invoked, is dictated by their respective plugin' priority which is an integer number that ranges from $-\infty$ to $+\infty$ and defaults at $0$ . An handler from a plugin with priority $2$ is invoked before an handler from a plugin with priority $-1$ for instance.

Priorities are set on a per-plugin basis.

Payload¶

Request objects coming from clients and response objects generated by the server both include an extra field meant to be used for plugin communication. The payload is carried over the whole request-response cycle.

For instance: a pre-processor (an handler bound to a BeforeX event) could enrich the payload with some data that will then be accessible to all postprocessors for the same request (handlers bound to an AfterX event).

Time based events¶

The pluginengine is also capable of firing events periodically or at specific times.

These events have the syntax CRON <cronspec>, where <cronspec> is just a placeholder for a string that must be formatted according to these rules or generated using a tool such as crontab.guru.

Info

Cronspecs that crontab.guru marks as non-standard won't work.

For instance: CRON * * * * * is fired every minute, CRON 0 12 * * * is fired at noon.

Quickstart¶

Requisites¶

NodeJS
NPM

To get started, create a new directory for your plugin and initialize an npm package:

$ mkdir myPlugin && cd myPlugin
$ npm init -y

This should create a package.json file that looks something like this:

{  
    "name": "myPlugin",  
    "version": "1.0.0",  
    "main": "index.js",  
    "license": "MIT"  
}

What matters here is the main field: it tells the engine what source file it should load and begin executing first. Everything else that your plugin needs should be imported by the main file. The default is index.js, you can use that or change it to your likings.

Your directory should have this structure by now:

.
├─package.json
└─index.js

Let's write some code:

const {engine, Plugin} = require.main.require("./main.js");

const plugin = new Plugin("myPlugin");

engine.registerPlugin(plugin);

What we have done so far is: 1. Create a new plugin with id: myPlugin 2. Register it to the engine

Let's now edit index.js and define an handler:

const {engine, Plugin} = require.main.require("./main.js");

const plugin = new Plugin("myPlugin");

plugin.registerHandler("BeforeSearch", request => {
    return request;
});

engine.registerPlugin(plugin);

Info

The name of the variable that you assign new Plugin("plugin id goes here") is not important for the scope of the plugin engine. You should, however, make sure that all plugins have a unique plugin identifier which is the string you pass to the plugin constructor.

Warning

Always remember to return the object that gets passed to the handler! In this case the request object. Failing to do so could potentially result in the plugin engine crashing.

With this snippet, we're creating an handler for myPlugin that listens to BeforeSearch events, possibly producing side effects or altering the request object before the controller handles it.

We're not doing anything useful yet. Let's edit the code above so that this plugin will deny any search of anything but product types.

const {engine, Plugin} = require.main.require("./main.js");

const plugin = new Plugin("myPlugin");

plugin.registerHandler("BeforeSearch", request => {
    request.wantDrawers = false;
    request.wantProducts = false;
    return request;
});

engine.registerPlugin(plugin);

This will make sure that no matter what the client requested, only productTypes will ever be returned.

Now what's left to do is deploy this plugin. To do that you need access to your ONO controller, including linux username and password (and ssl private key if you have that set up).

A plugin is self-contained within a directory. Installing a plugin is as easy as placing that directory inside a specific path in the plugin engine.

To do that, get a copy of your plugin directory on the controller. Make sure no operator is using the wharehouse and no maintenance task is currently running. Then execute the following (assuming your plugin directory is named myPlugin and you copied it into the current working directory):

$ docker cp ./myPlugin /home/ono/ono/pluginengine/plugins/myPlugin
$ docker-compose restart pluginengine && sleep 5 && docker-compose restart ono

What does that do?

It copies your plugin folder to the correct path inside the virtual machine that hosts the plugin engine and reloads it.

Guide¶

This is a commented code snippet that will explain most of the features you can use when developing plugins.

// Import engine and Plugin class
const {engine, Plugin} = require.main.require("./main.js");

// Create a new plugin
const pluginID = "plugin id goes here";
const plugin = new Plugin(pluginID);

// Change the plugin priority to 3. This plugin'
// handlers will be executed before any other
// handler from a plugin with priority < 3.
plugin.priority = 3;

// Create a basic `Before` handler
plugin.registerHandler("BeforeX", request => {
    // TODO Do something here, maybe side effects, maybe alter
    // the request object.

    // Always return the request object at the end of a `Before'
    // handler, this will get passed on to the next handler down
    // the pipeline, based on priorities.
    return request;
});

// Create a basic `After` handler
plugin.registerHandler("AfterX", response => {
    // TODO Do something here, maybe side effects, maybe alter the,
    // response object, maybe even add something to the JSON payload.

    // Always return the response object at the end of a `After'
    // handler, this will get passed to the next handler
    // down the pipeline based on priorities.
    return response;
});

// `After` handler that enriches the JSON payload.

// WARNING: A plugin can only declare one handler for each event.
// If you declare multiple handlers, only the last one will be used
// by the engine. Here we cannot declare another handler for `AfterX`.
plugin.registerHandler("AfterY", response => {
    response.payload = {x: 10};
    // NOTE this will overwrite any previous attributes added by other
    // plugins, if you just want to add new attributes use:
    response.payload = {...response.payload, y: 10};

    return response;
});

// External dependencies
// To declare external dependencies, add them to your `package.json`
// or better yet: use the `--save` flag when installing with npm
// or use yarn. Either way will update `package.json` automatically.
// The plugin engine will make sure all plugins' dependencies are installed.

// NOTE It's good JavaScript practice to make sure all imports are among
// the very first lines of your source code. We're putting it here just
// for demonstration purposes
const {filter} = require("lodash");
plugin.registerHandler("ExternalDependencies", response => {
    response.someArrayField = filter(response.someArrayField, e => e.isOk);

    return response;
});

// Asynchronous code
// To use asynchronous code, instead of returning
// the request/response object, return a Promise that resolves
// to it. The engine will wait for that promise to resolve before
// moving on to the next handler.
const axios = require("axios");
plugin.registerHandler("AsyncCode", response => {
    return axios.get("http://example.com").then(res => {
        response.payload = {status: res.statusCode};
        return response;
    });
});
plugin.registerHandler("AsyncCodeWithAsyncAwait", async response => {
    const res = await axios.get("http://example.com");
    response.payload = {status: res.statusCode};
    return response;
});

// Persistent data
const fs = require("fs");
const path = require("path");
plugin.registerHandler("PersistentData", async response => {
    // All the engine can do is provide you with the path
    // to a directory whose content gets preserved between restarts.
    // It's up to you to decide how to use that space.
    // You could, for instance, save data to a JSON file,
    // use a SQLite database or a tiny file-based
    // database like LowDB.

    // Each store directory is identified by its name.
    const storePath1 = engine.getStorepath("store name");

    // You can also call the same method on the plugin object
    // and it will yield the same result.
    const storePath2 = plugin.getStorepath("store name");
    // If you go this route, you can also omit the store
    // name and it will default to the ID of the plugin
    // you're calling the method on.
    const storePath3 = plugin.getStorepath();

    // Let's write a JSON file for example:
    fs.writeFileSync(path.join([storePath3, "mydb.json"], JSON.stringify({x: 10}));

    return response;
});

// Time based events
// Handlers bound to time based events must not take any parameters nor return anything
// unlike their standard counterpart.
// The cronspec for this tells us that the event is fired every second.
// This handler will print the lyrics for Daft Punk - Around the World.
plugin.registerHandler("CRON * * * * * *", () => {
    console.log("Around the world, around the world");
});

// Payload
plugin.registerHandler("BeforeZ", request => {
    request.pluginPayload = {message: "Hello World!"};
    return request;
});
plugin.registerHandler("AfterZ", response => {
    console.log(response.pluginPayload.message);
    return response;
});

// At last, register the plugin to the engine
engine.registerPlugin(plugin);

Performance¶

A sample request takes, on average, 140ms with plugins and 115ms without.

With plugins:

Without plugins:

A gRPC callsingle request to the plugin engine takes 6ms on average. Keep in mind that a single request to the controllmain server usually requires more that one round trip between itself and the plugin engine though.

Quite surprisingly, taking logging away reduced the avarage request time further by 2ms.

From our testings, it seems like the plugin engine is not that big of an overhead. It is important to note, though, that the extra time and resources that are required to process a request heavily depens on the number and the quality of the plugins that you installed on the engine.