Difference between revisions of "Edge++: Datonis Edge"

From Datonis
Jump to: navigation, search
(Created page with " == Basic Flow: == * The source data coming to Edge from various Adapters is pushed into Redis '''input-queues''' for processing and the processed data is put to the Redis '''...")
 
(→‎Data JSON:)
 
(13 intermediate revisions by 2 users not shown)
Line 1: Line 1:
  
−
== Basic Flow: ==
+
== Edge++ Configuration on Datonis Edge: ==
−
* The source data coming to Edge from various Adapters is pushed into Redis '''input-queues''' for processing and the processed data is put to the Redis '''output-queue''' which is used for '''transmitting data & alerts to Datonis''' and '''executing instructions'''.
+
=== Edge++ Service: ===
−
* The events are processed using Ruby Faas Functions that are present in the Rails framework of Edge++. It is also used to for showing operator screens using this data.
+
Edge++ Service offers support of Redis Data-store which is used for exchanging data between Edge and the Faas Framework. Edge++ Service lets us configure the settings related to the Redis store.
−
* Redis is used as a shared data-store for Edge and the Rails Framework
 
−
* The '''number of Redis input queues''' is a configurable entity that allows us to process data parallelly. It has to be configured in the Rails framework.
 
−
* For example - if there are 30 things configured on Edge, then the number of Redis Input queues can be 5.
 
−
* For '''one''' Redis Input Queue, there is '''one''' long-running '''Rake Task''' which checks if there is any data available in a particular Redis input Queue. The input queue in which a particular thing’s data should go is consistent throughout. This is achieved by distributing the things between the number of configured Redis input queues. The distribution is done using the unique hashCode of the thingKeys.
 
−
* The long-running RakeTask for each input queue has to determine the type of the popped message from the input queue viz. Data, instruction. And the event is processed using the process faasFunction provided along with the event JSON. This faasFunction processes the event and returns the processed values. The output is pushed into the output queue which is read by the Edge. We can also return null which will not push the event to the output queue. This can be decided by analyzing the processing logic.
 
  
−
== Configuration: ==
+
'''Send-to-Datonis''':
 +
* This flag is true if the events in the output-queue are to be transmitted to Datonis.
 +
* Make this flag false if the data should not be transmitted to Datonis.
 +
[[File:Edge++ Service.png|1360x1360px|border]]
 +
 
 +
=== EdgeCloudService: ===
 +
Once the access-key and secret-key are configured, set '''Enable Edge++ Setup''' flag to true. This will update the Datonis access-key in the Redis Store on save.
  
−
=== Edge++ Service: ===
+
This will be used by the Rails framework to set up its components.
−
Edge++ Service offers support of Redis Data-store which is used for exchanging data between Edge and the Faas Framework. Edge++ Service enables us to configure the settings related to the Redis store.
 
−
* '''Send-to-Datonis''':
 
−
** This flag is true if the events in the output-queue are to be transmitted to Datonis.
 
−
** Make this flag false if the data should not be transmitted to Datonis.
 
  
−
=== EdgeCloudService: ===
+
[[File:Enable-edge++ 1.png|1360x1360px|border]]
−
Once the access-key and secret-key are configured, '''enable Edge++ setup'''. This will update the access-key in the Redis which will be used by the Rails framework in setting up its components.
 
  
 
=== Thing Settings: ===
 
=== Thing Settings: ===
−
In thing Settings for in any adapter configuration of a particular thing, you can find the following Edge++ related settings.
+
In Settings Tab for in any adapter configuration of a particular Thing, you can find the following Edge++ related settings.
 
# '''Publish Mode - Send/scan Edge++''': Sends the events (scan/send data, instruction, alerts) to the Redis input queue, and the FaasFunction will process and push the processed event into the Output queue.
 
# '''Publish Mode - Send/scan Edge++''': Sends the events (scan/send data, instruction, alerts) to the Redis input queue, and the FaasFunction will process and push the processed event into the Output queue.
−
# '''Sync Process Functions:''' This button populates the Data Process Function and the Instruction Process Function dropdown which are configured on the rails framework.
+
# '''Sync Edge++ Functions:''' This button populates the Data Process Function and the Instruction Process Function dropdown which are configured on the rails framework.
 
# '''Data Process Function:''' Select the function which you want to use to process the thing’s data event.
 
# '''Data Process Function:''' Select the function which you want to use to process the thing’s data event.
 
# '''Instruction Process Function''': Select the function which you want to use to process the thing’s instruction event. You can select “--Not-Selected--” option which does not send instruction event to Redis.
 
# '''Instruction Process Function''': Select the function which you want to use to process the thing’s instruction event. You can select “--Not-Selected--” option which does not send instruction event to Redis.
 +
Screen for '''Sync Edge++ Functions'''
 +
 +
[[File:Sync Edge++.png|1360x1360px|border]]
 +
 +
Screen for '''Thing Specific Edge++ Settings'''
 +
 +
[[File:Edge++ settings 1.png|1360x1360px|border]]
  
 
== JSON Structures: ==
 
== JSON Structures: ==
Line 32: Line 35:
  
 
=== Data JSON: ===
 
=== Data JSON: ===
−
* data: should be a map of metrics (Configured on Datonis)  <String, value>  You can add new datonis metrics not defined on Edge.
+
* data: should be a map of metrics (Configured on Datonis)  <String, value>  You can add new Datonis metrics not defined on Edge.
 
* data_faas_function : Helper faas function name for Edge++.
 
* data_faas_function : Helper faas function name for Edge++.
 
* instruction_faas_function : Helper faas function name for Edge++
 
* instruction_faas_function : Helper faas function name for Edge++
 
* type : “data”
 
* type : “data”
−
<code>{</code>
+
{
−
 
+
    "thing_key":"d9a3cb4f52",
−
<code>"thing_key":"d9a3cb4f52",</code>
+
    "timestamp":1590067159361,
−
 
+
    "data":{
−
<code>"timestamp":1590067159361,</code>
+
      "T":1,
−
 
+
      "H":2
−
<code>"data":{</code>
+
    },
−
 
+
    "data_faas_function":"data_faas_function#12",
−
<code>"T":1,</code>
+
    "type":"data"
−
 
+
}  
−
<code>"H":2</code>
 
−
 
 
−
<code>},</code>
 
−
 
 
−
<code>"data_faas_function":"data_faas_function#12",</code>
 
−
 
 
−
<code>"instruction_faas_function":"instruction_faas_function#11",</code>
 
−
 
 
−
<code>"type":"data"</code>
 
−
 
 
−
<code>}</code>
 
  
 
=== Instruction JSON: ===
 
=== Instruction JSON: ===
 
You can insert instructions in the output queue using this format.
 
You can insert instructions in the output queue using this format.
−
 
+
{
−
<code>{</code>
+
    "thing_key":"d9a3cb4f52",
−
 
+
    "timestamp":1591308083000,
−
<code>"thing_key":"d9a3cb4f52",</code>
+
    "alert_key":"877c5e3cc4",
−
 
+
    "instruction_wrapper":{
−
<code>"timestamp":1591308083000,</code>
+
      "instruction":{
−
 
+
          "T":"1"
−
<code>"alert_key":"877c5e3cc4",</code>
+
      }
−
 
+
    },
−
<code>"instruction_wrapper":{</code>
+
    "type":"instruction",
−
 
+
    "instruction_faas_function":"instruction_faas_function#11"
−
<code>"instruction":{</code>
+
}  
−
 
 
−
<code>"T":"1"</code>
 
−
 
 
−
<code>}</code>
 
−
 
 
−
<code>},</code>
 
−
 
 
−
<code>"type":"instruction",</code>
 
−
 
 
−
<code>"instruction_faas_function":"instruction_faas_function#11"</code>
 
−
 
 
−
<code>}</code>
 
  
 
=== Alert JSON: ===
 
=== Alert JSON: ===
Line 89: Line 69:
  
 
INFO - 0, WARNING - 1,  ERROR - 2, CRITICAL - 3
 
INFO - 0, WARNING - 1,  ERROR - 2, CRITICAL - 3
−
 
+
{
−
<code>{</code>
+
    "alert":{
−
 
+
      "thing_key":"d9a3cb4f52",
−
<code>"alert":{</code>
+
      "timestamp":1589294688742,
−
 
+
      "alert_key":"a28cb7bt36",
−
<code>"thing_key":"d9a3cb4f52",</code>
+
      "alert_type":0,
−
 
+
      "message":"[]",
−
<code>"timestamp":1589294688742,</code>
+
      "data":{
−
 
+
          "execution_status":"success"
−
<code>"alert_key":"a28cb7bt36",</code>
+
      }
−
 
+
    },
−
<code>"alert_type":0,</code>
+
    "type":"alert"
−
 
+
}
−
<code>"message":"[]",</code>
 
−
 
 
−
<code>"data":{</code>
 
−
 
 
−
<code>"execution_status":"success"</code>
 
−
 
 
−
<code>}</code>
 
−
 
 
−
<code>}</code>
 
−
 
 
−
<code>}</code>
 

Latest revision as of 06:20, 30 July 2020

Edge++ Configuration on Datonis Edge:

Edge++ Service:

Edge++ Service offers support of Redis Data-store which is used for exchanging data between Edge and the Faas Framework. Edge++ Service lets us configure the settings related to the Redis store.

Send-to-Datonis:

  • This flag is true if the events in the output-queue are to be transmitted to Datonis.
  • Make this flag false if the data should not be transmitted to Datonis.

Edge++ Service.png

EdgeCloudService:

Once the access-key and secret-key are configured, set Enable Edge++ Setup flag to true. This will update the Datonis access-key in the Redis Store on save.

This will be used by the Rails framework to set up its components.

Enable-edge++ 1.png

Thing Settings:

In Settings Tab for in any adapter configuration of a particular Thing, you can find the following Edge++ related settings.

  1. Publish Mode - Send/scan Edge++: Sends the events (scan/send data, instruction, alerts) to the Redis input queue, and the FaasFunction will process and push the processed event into the Output queue.
  2. Sync Edge++ Functions: This button populates the Data Process Function and the Instruction Process Function dropdown which are configured on the rails framework.
  3. Data Process Function: Select the function which you want to use to process the thing’s data event.
  4. Instruction Process Function: Select the function which you want to use to process the thing’s instruction event. You can select “--Not-Selected--” option which does not send instruction event to Redis.

Screen for Sync Edge++ Functions

Sync Edge++.png

Screen for Thing Specific Edge++ Settings

Edge++ settings 1.png

JSON Structures:

JSON formats used for pushing data into Redis output-queue.

Data JSON:

  • data: should be a map of metrics (Configured on Datonis)  <String, value> You can add new Datonis metrics not defined on Edge.
  • data_faas_function : Helper faas function name for Edge++.
  • instruction_faas_function : Helper faas function name for Edge++
  • type : “data”
{
   "thing_key":"d9a3cb4f52",
   "timestamp":1590067159361,
   "data":{
      "T":1,
      "H":2
   },
   "data_faas_function":"data_faas_function#12",
   "type":"data"
}

Instruction JSON:

You can insert instructions in the output queue using this format. 

{
   "thing_key":"d9a3cb4f52",
   "timestamp":1591308083000,
   "alert_key":"877c5e3cc4",
   "instruction_wrapper":{
      "instruction":{
         "T":"1"
      }
   },
   "type":"instruction",
   "instruction_faas_function":"instruction_faas_function#11"
}

Alert JSON:

Alert_type:

INFO - 0, WARNING - 1,  ERROR - 2, CRITICAL - 3 

{
   "alert":{
      "thing_key":"d9a3cb4f52",
      "timestamp":1589294688742,
      "alert_key":"a28cb7bt36",
      "alert_type":0,
      "message":"[]",
      "data":{
         "execution_status":"success"
      }
   },
   "type":"alert"
}
Retrieved from "https://docs.datonis.io/mediawiki/index.php?title=Edge%2B%2B:_Datonis_Edge&oldid=1415"