We need a structured list of all messages and commands

For developers
41

So the info in the files in the data folder is not parsed from the PDFs?
For instance to add a new command message it should be added in Velbus_data_protocol_messages.pm manually?
Probably not all info can be parsed automatically from the PDFs (like on how to convert), but like you said, I also think most of it can be parsed. Eg: the info from the tables, like I did here.
I’d like to help here, but not sure what’s the best way to keep this easy.

Like @MDAR said, everybody has his own interests. I’m now working on Node-RED nodes to update colors of Edge-lit buttons. I also want to send memo texts to them…
So we have to find a way to for any developer to easily add these niche commands, so everybody can benefit from it.

1 Like
42

Indeed, that’s a manual step. The effort to parse the protocol file 100% correctly does not outweighs the time needed to read the pdf and put in the needed data manually.

Parsing the pdf and putting the text in a json is something different then what I did. I don’t care about the text

“databyte2”:“0 =disabled / 1 = enabled”

No, I want something like

{
“0” = “disabled”,
“1” = “enabled”
}

And this needs to 100% accurate and for all the possible cases in all the protocol files the same.
So I prefer to this manually for the few places where the content of the databyte is relevant.

Stef

43

I don’t care about alarms and programs because I use OpenHAB for automation.
But if the demand is there, can always take a look at the messages and see if I can understand them and add them to the json and my scripts.

Stef

1 Like
44

Thanks Stef for making that JSON available. There’s a lot of information in that. :slight_smile:

I don’t think that Stef’s JSON should be responsible to know the whole structure of every command and their databytes. If you need that, I would wait for our own revision of the protocol manuals and parse that.

Here’s an example of what we have in mind (at this moment):

{
 "id": "02-2",
 "name": "'Switch relay on' command received",
 "modules": [
   "VMB4RYLD",
   "VMB4RYNO"
 ],
 "type": "Request",
 "priority": {
   "name": "Highest priority",
   "value": 0
 },
 "addressRequired": "Module address",
 "rtr": false,
 "databytes": [
   {
     "description": "Command Type",
     "displayAs": "hex",
     "options": [
       {
         "description": "COMMAND_SWITCH_RELAY_ON",
         "pattern": "02"
       }
     ]
   },
   {
     "description": "Relay bit number",
     "displayAs": "bin",
     "options": [
       {
         "description": "Channel 1",
         "pattern": "00000001"
       },
       {
         "description": "Channel 2",
         "pattern": "00000010"
       },
       {
         "description": "Channel 3",
         "pattern": "00000100"
       },
       {
         "description": "Channel 4",
         "pattern": "00001000"
       },
       {
         "description": "Virtual channel 5",
         "pattern": "00010000"
       }
     ]
   }
 ]
}

{
"id": "FB-3",
"name": "Transmits the relay status",
"remarks": [
  "[DATABYTE6][DATABYTE7][DATABYTE8] contain a 24-bit time in seconds"
],
"modules": [
  "VMB4RYLD",
  "VMB4RYNO"
],
"type": "Response",
"priority": {
  "name": "Lowest priority",
  "value": 11
},
"addressRequired": "Module address",
"rtr": false,
"databytes": [
  {
    "description": "Command Type",
    "displayAs": "hex",
    "options": [
      {
        "description": "COMMAND_RELAY_STATUS",
        "pattern": "FB"
      }
    ]
  },
  {
    "description": "Relay bit number",
    "displayAs": "bin",
    "options": [
      {
        "description": "Channel 1",
        "pattern": "00000001"
      },
      {
        "description": "Channel 2",
        "pattern": "00000010"
      },
      {
        "description": "Channel 3",
        "pattern": "00000100"
      },
      {
        "description": "Channel 4",
        "pattern": "00001000"
      },
      {
        "description": "Virtual Channel 5",
        "pattern": "00010000"
      }
    ]
  },
  {
    "description": "Disable/inhibit/Forced on setting",
    "displayAs": "bin",
    "options": [
      {
        "description": "Channel normal",
        "pattern": "xxxxxx00"
      },
      {
        "description": "Channel inhibited",
        "pattern": "xxxxxx01"
      },
      {
        "description": "Channel forced on",
        "pattern": "xxxxxx10"
      },
      {
        "description": "Channel disabled",
        "pattern": "xxxxxx11"
      }
    ]
  },
  {
    "description": "Relay status",
    "displayAs": "bin",
    "options": [
      {
        "description": "Relay channel off",
        "pattern": "xxxxxx00"
      },
      {
        "description": "Relay channel on",
        "pattern": "xxxxxx01"
      },
      {
        "description": "Relay channel interval timer on",
        "pattern": "xxxxxx11"
      }
    ]
  },
  {
    "description": "Led status",
    "displayAs": "bin",
    "options": [
      {
        "description": "LED off",
        "pattern": "00000000"
      },
      {
        "description": "LED on",
        "pattern": "10000000"
      },
      {
        "description": "LED slow blinking",
        "pattern": "01000000"
      },
      {
        "description": "LED fast blinking",
        "pattern": "00100000"
      },
      {
        "description": "LED very fast blinking",
        "pattern": "00010000"
      }
    ]
  },
  {
    "description": "high byte of current delay time",
    "displayAs": "none"
  },
  {
    "description": "mid byte of current delay time",
    "displayAs": "none"
  },
  {
    "description": "low byte of current delay time",
    "displayAs": "none"
  }
 ]
}

We have plans to generate a HTML based on this, eventually replacing the PDFs. Please note that the above structure is not final and subject to change.

In this example, you can find the needed packets for VMB4RYLD and VMB4RYNO allowing to turn the relay on. ( That’s a simple example, I know :slight_smile: )

For example, to turn relay 1 on:

  • Send out one packet with command type: 0x02 (Turn relay on) 0x01 (Relay 1).
  • Wait for one packet with command type: 0xFB (Relay status) and check the second databyte to be 1 and check if the first 2 bits of the third databyte to be 01.

I can find what is needed to turn on a relay in Stef’s JSON.

I think we should not define channel types to a channel, since you have a lot of channel types and chances are big that it will be extended with new channel types quite soon. :wink:

To give an example, some of our (newer) relay modules support inhibited, forced on, forced off and normal state. If we extend the JSON to support that, I assume you need to make a new channel type.

I think a list of possible “features” (switch relay on, force relay on, inhibit, switch to program 1, switch programs off, …) should be defined per module. An integration, supporting a relay, can look in the feature list if the required features are available and make the mapping.

Food for thought.

2 Likes
45

Hi,

There is a difference in making a json file from the protocol file and making a json file to implement the velbus protocol.

I know that I had a lot of trouble in converting the message data to a channel number. I will have to dig in my code to document it. But it was messy sometimes!

There should be a mapping between the message and channel number, not channel name. I don’t care if it’s a virtual channel, or an alarm or a button.

What I miss in your code is a channel number that’s always consistent for all messages and all modules:

And it can even simplified, instead of giving a sum of mappings, just say this:

When Content=Channel the byte contains the channel number. And we can find the channel number by converting it from ‘counting bit’ notation to decimal (I cal this ChannelBit in my code).
I did this for every message that I needed to parse so I can get the Channel number in a consistent way.

Stef

46

@VEL524
The biggest problem i face is that the same command codes are used with different data inside of it for certain modules.
How would this be represented in the velbus json?
Do you have an eta on when this would be available?

And to be honest i like the format from vel524 a bit more then the format from Stef_Coene as it has less logic inside

47

I have per module type and per message the configuration on how to interpret the command.
I try to reuse as much code as possible, that’s why I generate the json file based on (in my case) perl code that contains that logic.

It depends.
If you want to know the content of a message, then yes. But if you want to do something with message, then you need the logic to understand the message.
An example is channel numbering. Make it always the same. No in text like ‘Relay 1’. But in real numbering.

And we also need to change or add information. Example:

“description”: “Relay channel on”,

But for a programmer, this needs to be translate in a state he can use and not in some text:

“state”: “ON”

And for a programmer, this code:

"options": [
  {
    "description": "Channel 1",
    "pattern": "00000001"
  },
  {
    "description": "Channel 2",
    "pattern": "00000010"
  },
  {
    "description": "Channel 3",
    "pattern": "00000100"
  },
  {
    "description": "Channel 4",
    "pattern": "00001000"
  },
  {
    "description": "Virtual Channel 5",
    "pattern": "00010000"
  }
]

Should be:

"Map": {

“00000001”: “CH01”,
“00000010”: “CH02”,
“00000100”: “CH03”,
“00001000”: “CH04”,
“00010000”: “CH05”,
}

Stef

48

The more I think about it, the more I’m sure that my json layout is what I need.
The layout started very basic and step by step I added what I needed.
I’m sure that some parts can be done better, but at the end it is what I need.

I will try to document my json layout and how my scripts are using the json file.
I will also try to document the workaround I needed to implement at some places.

Stef

49

Maybe i don’t really understand your json, but i think its way to complex …

@Stef_Coene maybe it would be usefull to give us a walkthrough about how you use it?
for example:

1- how to use it for a scan
2- how to know what message to generate for turning a realy channel on
3- if we receive a couterStatus message for a vmb7in channel

50

Finding the right balance is hard; universal reusabe code versus detailed described readable…
If I may choose, I prefer the actual JSON format with wide application.

51

Thanks for all your valuable input, guys. :ok_hand:

I think we need both and I don’t see a reason why we can’t merge both needs in one JSON.
Eg:

"options": [
  {
    "channelNumber": 1,
    "description": "Channel 1",
    "pattern": "00000001"
  },
  {
    "channelNumber": 2,
    "description": "Channel 2",
    "pattern": "00000010"
  },
  {
    "channelNumber": 3,
    "description": "Channel 3",
    "pattern": "00000100"
  },
  {
    "channelNumber": 4,
    "description": "Channel 4",
    "pattern": "00001000"
  },
  {
    "channelNumber": 5,
    "description": "Virtual Channel 5",
    "pattern": "00010000"
  }

Personally I think that having an array of objects (as the above example) is better than using an object with keys: You have more freedom to find an item based on whatever key/value, as with an object you are stuck with the fixed key. The only downside is a little bit more verbose code. Speed is neglectable as long as you don’t have thousansds of items.

Example in JS code (ES6):

options.find(object => object.pattern === "00000010") //returns the second object

But you can also search on channelNumber:
options.find(object => object.channelNumber === 2) //also returns the second object

So you are not stuck on a predefined key to find an item.

@Stef_Coene do you see a downside of using arrays of objects?

Some examples might help, indeed.

52

In case someone is still following this discussion and is interested in some more information about how my scripts are parsing messages, see this document:

Stef

3 Likes
53

@Stef_Coene ik denk dat volgende lijst nog niet verwerkt is in uw json

54

My mistake. I generated a new json file but I forgot to push it to github.

Stef

1 Like
55

now the json is much better :slight_smile:

56

I agree. The Messages section is very valuable to me.

@Stef_Coene: Can you define what exactly should be done to add a Data section to messages that don’t have one yet?
May be I can fork and do a pull request to test that out?

Thanks!

57

The source for the json for message information is this file:
https://github.com/StefCoene/moduleprotocol/blob/master/data/Velbus_data_protocol_messages.pm

What data do you miss?

Stef

58

I just want to check how more complicated messages should be added and would work with the existing format.

Eg. when I want to add the “Set Edge Color command received” (COMMAND_SET_PB_BACKLIGHT) for the VMBELO, what should be added?
I guess I should extend the ModuleGeneral.EdgeLit and add ModuleTypes[37].Messages[AC]?
Is this correct?

Tx!

59

Nothing, it’s already defined in the json.

If I look at the protocol file, message COMMAND_SET_PB_BACKLIGHT is D4, not AC.
There is no message AC for a VMBELO.

But anyway, message COMMAND_SET_PB_BACKLIGHT = D4 is already defined for VMBELO = 37.
This can be found in Velbus_data_protocol_messages.pm:

$json{ModuleTypes}{‘37’}{Messages}{‘D4’}{General} = “EdgeLit” ;

So the message D4 for type 37 is mapped to the General definition EdgeLit.
I did this because there multiple modules that uses the same command and I don’t wanted to repeat the same definitions and risks typing errors.

Search for this in the json for the definition of EdgeLit:

$json{ModuleGeneral}{Messages}{EdgeLit}

And it’s funny how you picked one of the more difficult commands because there are 2 versions of the command. One with 3 data bytes and one with 5 data bytes. That’s why there are 2 definitions, search for this to find them:

$json{ModuleGeneral}{Messages}{EdgeLit}{Data}{‘PerByte:5’}
$json{ModuleGeneral}{Messages}{EdgeLit}{Data}{‘PerByte:3’}

Each version of the command has some Match lines per byte to extract the information we want.

Example: this will take the second byte from the 5 byte long message. It contains the Red channel of the color and we need to convert the byte to decimal:

$json{ModuleGeneral}{Messages}{EdgeLit}{Data}{‘PerByte:5’}{‘2’}{Name} = “Red” ;
$json{ModuleGeneral}{Messages}{EdgeLit}{Data}{‘PerByte:5’}{‘2’}{Match}{‘%.’}{Convert} = “Decimal” ;

Example: this will take the first byte from the 3 byte long message. And if the 1 is in the correct place, it means the message was for the bottom led:

$json{ModuleGeneral}{Messages}{EdgeLit}{Data}{‘PerByte:3’}{‘1’}{Match}{‘%…1…’}{Value} = “bottom” ;

To be honest, I don’t own a edge lit panel myself but I tested my code on a remote setup with a webcam. So it’s possible that’s not working 100%.
And there is a lot of information in the messages that I don’t parse. Mostly because it’s of no use in my case.

I just noticed that’s not very visible in the Velbus_data_protocol_messages.pm files, but lines starting with a # are commented out.

Stef

2 Likes
60

@Stef_Coene
for the vmbgpod module we mis channel 33 in the json

any idea why?