We need a structured list of all messages and commands

For developers
1

Dear Velbus enthusiasts and developers,

The module protocol PDFs are a valuable source of information for each developer wanting to code a Velbus app.

The info in there is good, but what we lack is a structured, program friendly format of this info.

So I started merging all the pdfs and did some regEx magic to generate the below JSON list:

Link to JSON file

Not too bad to start from, but some important info is still missing, like

  • what commands are used on what module
  • is it a message or a command
  • RTR
  • Priority
  • The structure can be better, like putting the DataBytes in an array.
  • There are a lot of duplicates, as most commands are reused in several modules.
  • extra info/metadata, like how a byte should be used, …

IMPORTANT: this JSON is semi-automatically generated from the PDFs, so probably not error-free!

Here are my questions about this JSON:

  1. Does anybody already have a similar list? Or a better one to start from?
  2. Is there any interest to set up a community driven github to keep this up to date?
  3. Can Velleman help us out?
  4. Is JSON the right format?
  5. Did I miss something?
  6. Who is interested to help on this?

For sure it would help me a lot! How about you?

Gert

3 Likes
2

Would a cross referenced table work for seeing which commands are used by which modules?

I know Google can share spreadsheets, is there a way to do something similar in GitHub?

Or is it just easier to create a table in GitHub?

3

The word is out; thank you @GertSt

There is this weird thing; a lot of information is published with efforts to keep it difficult to create a clear overview…
The value of an active community might be under/over estimated?
Knowledge is… developers pleasure!

4

I like the idea of using a shared Google sheet. That’s easy to update for everyone and there are plenty of tools to export to whatever other format, like JSON.

The fork and pull system in GitHub is better do keep code clean, as there is only one or a few supervisors that can merge cod. But it might be overkill for what we need?

2 Likes
5

I don’t understand the strategy behind it. On one hand they do provide protocol info from day one. Very cool and unseen >10 years ago. On the other hand why don’t we still have an API after >10 years?

Velbus keeps struggling for years now with their KNX Gateway / Signum interface. Why?

I’m sure that velbus could have been much further if there was an open API.
A relative small company like Velleman cannot keep pace with the fast evolutions in home automation if they do not have an easy way to interface with the rest of the world. We really need that!

So, back on topic: we need to join forces here! Looking forward to read the opinion of Velleman about this here on the forum! I’m pretty sure they stand behind it and can help us a lot.

Gert

1 Like
6

i would love something like this

i already started something for the python-velbus lib https://github.com/thomasdelaet/python-velbus/blob/master/velbus/data.json

i wanted to extend this to support all message types and other data. But this is a big task so i would love to have some help? We can always change the data format. So i really like the idea

7

This is indeed another type of info we need per module.

Lets wait a few days to see if Velleman responds and agrees to help.

8

I also did something like you did:
See https://github.com/StefCoene/velserver/blob/master/bin/pdf2txt.pl
This will process all protocol files and extract as much information as possible.
It generates the file Velbus_data_protocol_auto.pm

I also created some extra files manually:
Velbus_data_protocol_channels.pm: channel information: name, type, address mapping, …
Velbus_data_protocol_memory.pm: memory information: modulename, build version, …
Velbus_data_protocol_messages.pm: supported messages per module type, …
The files can be found in:

I was in touch with the developers of the OpenHAB and the homeassistant binding to coordinate our efforts, but I didn’t had the time to work further on it…

What if I spin this off this in a new project and generate whatever file is needed?
Then we all work with the same information?

Stef

1 Like
9

Very valuable info, Stef! I think it covers a lot more that what I have.
It would be great to use this as a starting point for a more language agnostic format.

10

Not sure what file format should be used to store all module data …
Let us know your preference in the below poll. Answers are visible after voting.

  • A JSON file managed by a single person on Github, allowing forks/pulls
  • A Google Spreadsheet open for everyone to edit
  • A Google Spreadsheet; read-only but updates can be requested on the forum
  • Something much better? Let us know!

0 voters

11

Is there a way to use a JSON file but make it easier to update than a git fork/pull?
I prefer the easy update of a Google Sheet and prefer de JSON format :face_with_raised_eyebrow:

12

as long as the output is in an easy parsable format (json or yaml) i don’t care how its inserted.

13

It’s not that hard to change my script to generate any output format you want.
json, xml, java, perl, python, …

And I recommend to NOT put it in a google sheet but in a github project.
This is better for tracking changes and bug reports.

I don’t know if you already found it tout, but Velbus has an official github account:

They also maintain a github project with all protocol files:

In fact, I once cloned that project. The idea was to clone te project and add my parsing script to it.
That way I can keep the protocol files in sync with the Velbus protocol repository and generate the files I need.
Maybe I just have to continue that project and generate the files I need and at the same time generate files suitable for other projects.

Stef

2 Likes
14

I like this idea!
Unfortunately I don’t know perl and the language looks so exotic to me that I can’t help you with these scripts :neutral_face:

15

if we do this in perl it will indeed be hard, as i can’t help you either.

as long as the output file is computer readable i’m fine with it

16

Indeed, this is something we as Velbus are also working on to provide, the format will be JSON.

3 Likes
17

That’s very good news!
Any timeframe on when can we expect this?

18

This would be really good news :slight_smile:

19

We can’t provide a timeframe for this at the moment. It’s a very low priority at this moment. Keep in mind, that converting protocol manuals released over the last 10 years isn’t done in a few days. :slight_smile: Perhaps we may need to release part by part. After reading through the comments, I think the need is the biggest for the commands specifically.

The ideas in this topic are particulary interesting and it would be cool if the community can discuss what is needed in a data file. That would mean that the concepts are generalized and that an integration no longer needs to implement the functions of the module and their channels, but knows enough with the datafile. At this point, I’m not entirely sure if that would be possible, therefore the discussion. :wink:

We can’t make promises that everything needed would be included in the datafile that we produce. Our goal is to provide the protocol manuals in a structured way.

3 Likes
20

would it be an idea if the community helps filling in the data file?
if you add this to a git repo i’m certainly willing to fill in some data (via PR).

what i think that iss needed:

  • every commandcode and the associated message structure
    • every possible version of the message
    • the supported modules
  • for every module
    • the modulename
    • the moduleCode
    • the number of modules
    • the support messages
    • location of important data in the memorymap (stuff that is not requestable via messages)
      • the modulename
      • for the vmb7in the parameters for the counter channels

I think this would be a really good start

3 Likes