Difference between revisions of "Edge++: Datonis Edge"

Revision as of 09:17, 12 June 2020

Basic Flow:

• The source data coming to Edge from various Adapters is pushed into Redis input-queues for processing and the processed data is put to the Redis output-queue which is used for transmitting data & alerts to Datonis and executing instructions.
• The events are processed using Ruby Faas Functions that are present in the Rails framework of Edge++. It is also used to for showing operator screens using this data.
• Redis is used as a shared data-store for Edge and the Rails Framework
• The number of Redis input queues is a configurable entity that allows us to process data parallelly. It has to be configured in the Rails framework.
• For example - if there are 30 things configured on Edge, then the number of Redis Input queues can be 5.
• For one Redis Input Queue, there is one long-running Rake Task which checks if there is any data available in a particular Redis input Queue. The input queue in which a particular thing’s data should go is consistent throughout. This is achieved by distributing the things between the number of configured Redis input queues. The distribution is done using the unique hashCode of the thingKeys.
• The long-running RakeTask for each input queue has to determine the type of the popped message from the input queue viz. Data, instruction. And the event is processed using the process faasFunction provided along with the event JSON. This faasFunction processes the event and returns the processed values. The output is pushed into the output queue which is read by the Edge. We can also return null which will not push the event to the output queue. This can be decided by analyzing the processing logic.

Configuration:

Edge++ Service:

Edge++ Service offers support of Redis Data-store which is used for exchanging data between Edge and the Faas Framework. Edge++ Service enables us to configure the settings related to the Redis store.

• Send-to-Datonis:
  • This flag is true if the events in the output-queue are to be transmitted to Datonis.
  • Make this flag false if the data should not be transmitted to Datonis.

EdgeCloudService:

Once the access-key and secret-key are configured, enable Edge++ setup. This will update the access-key in the Redis which will be used by the Rails framework in setting up its components.

Thing Settings:

In thing Settings for in any adapter configuration of a particular thing, you can find the following Edge++ related settings.

1. Publish Mode - Send/scan Edge++: Sends the events (scan/send data, instruction, alerts) to the Redis input queue, and the FaasFunction will process and push the processed event into the Output queue.
2. Sync Process Functions: This button populates the Data Process Function and the Instruction Process Function dropdown which are configured on the rails framework.
3. Data Process Function: Select the function which you want to use to process the thing’s data event.
4. Instruction Process Function: Select the function which you want to use to process the thing’s instruction event. You can select “--Not-Selected--” option which does not send instruction event to Redis.

JSON Structures:

JSON formats used for pushing data into Redis output-queue.

Data JSON:

• data: should be a map of metrics (Configured on Datonis)  <String, value> You can add new datonis metrics not defined on Edge.
• data_faas_function : Helper faas function name for Edge++.
• instruction_faas_function : Helper faas function name for Edge++
• type : “data”

{

"thing_key":"d9a3cb4f52",

"timestamp":1590067159361,

"data":{

"T":1,

"H":2

},

"data_faas_function":"data_faas_function#12",

"instruction_faas_function":"instruction_faas_function#11",

"type":"data"

}

Instruction JSON:

You can insert instructions in the output queue using this format.

{

"thing_key":"d9a3cb4f52",

"timestamp":1591308083000,

"alert_key":"877c5e3cc4",

"instruction_wrapper":{

"instruction":{

"T":"1"

}

},

"type":"instruction",

"instruction_faas_function":"instruction_faas_function#11"

}

Alert JSON:

Alert_type:

INFO - 0, WARNING - 1,  ERROR - 2, CRITICAL - 3

{

"alert":{

"thing_key":"d9a3cb4f52",

"timestamp":1589294688742,

"alert_key":"a28cb7bt36",

"alert_type":0,

"message":"[]",

"data":{

"execution_status":"success"

}

}

}