MLAND Series – Tips III – Loopback RabbitMQ Usage

The purpose of this post is to help you get up and running with RabbitMQ integrated into your Loopback API.

Prerequisite

You have a working Loopback API Project.

RabbitMQ

Install the Loopback component

Inside your Loopback project folder run:

npm install loopback-component-mq --save

This will install the component and all it´s dependencies as you are used to

Configure the RabbitMQ component

Register it

Loopback loads components per default only if they are inside known folders. External components are in different locations and need be registered.
You do so by adding "../node_modules/loopback-component-mq/lib/mixins" to the mixins array in the model-config.json.

Component configuration

Loopback checks if a configuration is available for a component inside component-config.json.
Therefore add the following template to component-config.json:

{
"loopback-component-mq": {
    "path": "loopback-component-mq",
    "options": {
      "restPort": 15672,
      "acls": [
        {
          "accessType": "*",
          "principalType": "ROLE",
          "principalId": "$unauthenticated",
          "permission": "DENY"
        }
      ]
    },
    "topology": {
      "connection": {
        "uri": "amqp://$username:$password@$host:$port/$vhost", (1)
        "timeout": 30000
      },
      "exchanges": [
        {
          "name": "my_first_exhange",
          "type": "topic",
          "persistent": true
        }
      ],
      "queues": [
        {
          "name": "my_first_queue",
          "subscribe": true,
          "limit": 1
        },
        {
          "name": "my_second_queue",
          "limit": 1
        }
      ],
      "bindings": [
        {
          "exchange": "my_first_exchange",
          "target": "my_first_queue",
          "keys": [
            "my_first_queue"
          ]
        },
        {
          "exchange": "my_first_exchange",
          "target": "my_second_queue",
          "keys": [
            "my_second_queue"
          ]
        }
      ],
      "logging": {
        "adapters": {
          "stdOut": {
            "level": 5,
            "bailIfDebug": true
          }
        }
      }
    }
  }

Explanation:
The path entry reflects the component name.
You can configure where the management interface is located (restPort) and the acls you like via the options entry.

In topology we define the exchanges and queues we want to use inside this API.

Inside connection you define with the uri where your RabbitMQ Service is located and with the timeout how long the system should wait until a connect call fails.
I strongly suggest defining it, especially if you connect over the internet. Otherwise you instantly receive a connection error.
In exchanges you define the exchanges you want to use, with their name, their type and if you want them to persist after the api disconnects (key persistent). To learn more about
the Types have a look at (2).
queues contains all queues you want to use with the name, if you want to subscribe and if you want to limit the amount of concurrently processed messages.
If you subscribe to a queue you will retrieve all messages inside it, so make sure you handle them all. Otherwise you wonder why you number of messages grow and the api outputs errors. Use this if you implement a consumer for all messages you send to this queue (I show you later how it´s done).
I strongly advise you to use limits, otherwise the api might just stop working when too many message fight for resources. Additionally this impacts the performance of your API.
bindings model the connection between exchanges and queues while defining which routing key (key keys) you use.
exchange is self explanatory, target is the queue you want to connect. You can add keywords or themes in the keys array, I just put in the queue name I´d like to use.
You can configure the logging you want to have inside logging. I added the configuration I use myself for debugging.

Mixin configuration

The RabbitMQ component uses a mixin. This way you can configure the consumers and producers per model.

Add a similar structure like this to your model.json file:

"mixins": {
    "MessageQueue": {
      "consumers": {
        "consumerMessage": {
          "queue": "my_first_queue",
          "type": "$company.$type.$subtype"
        }
      },
      "producers": {
        "producerGreet": {
          "exchange": "my_first_exchange",
          "options": {
            "routingKey": "my_first_queue",
            "type": "$company.$type.$subtype"
            "contentType": "application/json"
          }
        },
      }
    }
  },

If you already have a mixins key in the model.json just add the inner structure, beginning at "MessageQueue".

Inside “MessageQueue” you can define a consumers and producers object, in which you define the consumers and producers respectively.

A consumer has a name ( in the example consumerMessage) and needs to know from which queue it should get it´s messages (key queue) and which message type it is responsibly for (key type). If only one message type will occur in queue you need only one consumer, else you need more. The name of the consumer (consumerMessage in the example) is the name of the method you have to implement for this model. I come to this in a bit.

A producer has a name too, is connected to an exchange (key exchange) and has some options. Here comes the keys from the component-config.json into play, where I said I use the queue name I want to target. They need to match the routingKey. At last we set contentType, for me this is normally json.
You don´t need to implement a producer, the component does it for you. In a few moment I will show you how you can call it.

Usage

Consumer

As mentioned you need to implement a consumer yourself. The syntax is (ES6 Syntax):

{
$model.consumername = (payload) => {
  // If your message comes from another source than a loopback-component-mq based API
  const { message } = JSON.parse(payload)

  // Otherwise you can simplify it to
  const { message } = payload

  // Do something
  ....

  if (error) {
    // Depending on your architecture you might want to reject the message if an error occurs.
    // This will not acknowledge the message and it will be re-delivered to you. So you can use this if you have a temporary problem, but the message is important
    return Promise.reject(error)
  } else {
    // If everything is alright acknowledge this message 
    return Promise.return()
  }
}

You tell the queue that you handled a message by returning a Promise.resolve(). If you want the message to be re-delivered you send Promise.reject().

Producer

You can use a producer anywhere inside the scope of your model this way:

{
$model.greet = (name) => {
  return $model.producerGreet({greeting: 'Hi', name: name}))
}

That´s it already. The producer send this message with a JSON payload to the defined exchange with the defined routingKey.

Conclusion / Lessons learned

That´s it for today. In this tip you learned:

How to install the Loopback RabbitMQ Component
How to register the Loopback RabbitMQ Component
How to configure the Loopback RabbitMQ Component
How to configure consumers and producers for the Loopback RabbitMQ Component
How to implement a consumer
How to use a producer

If you have any question post them into the comments. Or feel free to send my an email.

Yours sincerely,

Frank

Sources
(1) https://www.rabbitmq.com/uri-spec.html
(2) https://www.rabbitmq.com/getstarted.html