MDF download in REST

[grodansparadis]

Add MDF download in the REST interface to ease cross domain download problems.

[BlueAndi]

:+1:

[BlueAndi]

And provide a XQuery interface via REST for the MDFs, which will make the development of wizards much easier.

[grodansparadis]

With not much thought on it, it feels like it belongs in the Javascript domain.

Andreas Merkle skrev den 2015-07-09 23:23:

And provide a XQuery interface via REST for the MDFs, which will make the development of wizards much easier.

— Reply to this email directly or view it on GitHub https://github.com/grodansparadis/vscp_software/issues/161#issuecomment-120148021.

Be Hungry - Stay Foolish! Åke Hedman, Grodans Paradis AB / Paradise of the Frog Brattbergavägen 17, 82050 LOS, SWEDEN Phone: +46 (0)840011835 CellPhone: +46 (0)730-533146 http://www.grodansparadis.com http://www.vscp.org

---

Detta e-postmeddelande har sökts igenom efter virus med antivirusprogram från Avast. https://www.avast.com/antivirus

[BlueAndi]

No, its a typical server feature. XQuery for xml databases is similar to Query for SQL databases. If you implement it on client side, the client have to download all data, before it can execute it. That would be horrible slow ... think about collections of xml files.

[grodansparadis]

Yes it could be if the world of VSCP really was a server-is-always-available world. But it isn't and there is a lot of effort put into VSCP for it to not need to work that way. "things" are self contained and groups are self contained. A server connected to a group of things just add to functionality. It should by all needs be a required component to do certain things.

This is even more true for Ethernet and wireless devices.

Yes I think it is no problem if the web interface can use services if the daemon such as what is done with the websocket interface and the rest interface and the functionality they export. Building interfaces for a home or whatever is the perfect fit here.

But for things that is done by works today I think they should go into the built in interface or be done so they can live and work also without the server. Yes it is a mess to write but it becomes a tremendous thing for usability.

A typical real world example I have here is the Frankfurt BLE module. It is a CAN4VSCP module and typically connects to a group of other CAN4VSCP modules to form some functionality. But most of the time it is not connected to a server but the group should still be possible to be configured with a phone/table/pc and therefore discovery, mdf read out, register read/write must be done without a server. Same is true also for Ethernet and wireless devices.

That not said that it can't be there on the server to. But it should not be an assumption of usercode as it get unusable for most device configuration.

So my point is that if it is done in the javascript domain it is usable for all usercases and there is no need to rewrite this for every other setup.

/Ake

Den 2015-08-09 kl. 14:05, skrev Andreas Merkle:

No, its a typical server feature. XQuery for xml databases is similar to Query for SQL databases. If you implement it on client side, the client have to download all data, before it can execute it. That would be horrible slow ... think about collections of xml files.

— Reply to this email directly or view it on GitHub https://github.com/grodansparadis/vscp_software/issues/161#issuecomment-129171290.

Be Hungry - Stay Foolish! Åke Hedman, Grodans Paradis AB / Paradise of the Frog Brattbergavägen 17, 82050 LOS, SWEDEN Phone: +46 (0)840011835 CellPhone: +46 (0)730-533146 http://www.grodansparadis.com http://www.vscp.org

---

Detta e-postmeddelande har sökts igenom efter virus med antivirusprogram från Avast. https://www.avast.com/antivirus

[BlueAndi]

If there is no server (and the daemon is one), there is no web interface which needs the benefits of the XQuery functionality. vscpWorks can live without that kind of functionality, but in my eyes this feature will push the web interface development (provided by the daemon) fast forward and is a must have dealing with xml.

If you want to use the same html5/javascript code for web interfaces, provided by nodes, every node has to provide the daemon websocket/REST API. Otherwise it won't work with the same source code. If they have so many space in the flash, why they don't provide the XQuery functionality too?

If you can access the nodes without internet access, they will have the mdf included. Perfect ... because the html5/javascript source doesn't need to differ where the mdf comes from. It uses only the provided REST/websocket API and the XQuery command.

Another advantage is the fast loading web interface. There is no need to download the whole mdf and download huge javascript libraries to evaluate them.

How did my first example look like without XQuery:

1. Retrieve the mdf url.
2. Download the mdf.
3. Convert the mdf xml to a javascript object.
4. Walk through the object and check every single part with "if"-clauses.

My two cents about ...

[grodansparadis]

A phone/table/PC all usually have Internet access. At least in most parts of the world. And for a phone/tablet there is also wifi/Bluetooth so a node can this be configured without a server using the system that is in place now. So a group of devices like this

can be configured by loading the mdf url (or the mdf) from the device and configure it on the device. The same is true for device lookup etc.

The BLE node in this example just export the simple tcp/ip interface as specified in the spec. Accessed over a BLE serial interface. In the wifi case it is a normal socket. Other cases may use something else. So same setup can be done for a wifi or other interface to a group of device which have no server. They just can have a server if needed. An Ethernet or Wifi device (or something else) may even on it's own export this interface.

I don't argue against the usefulness of XQuery , it fit's perfectly with the planed automation interface of the daemon where the mdf is downloaded when a node is discovered. But it is special case. The server is for sure useful but I just don't want so see a reliance on it always being there because it will make people make assumptions they should not take.

At least you must agree that it would be stupid to build configuration and discovery code that we will have to do again and again for every setup and interface of the system. With just a little thought and a little layered thinking and some more work the same code (or at least most of it) can be used for all.

Also remember that most phones and tablets (and pc's) have way more power than the a typical thing a daemon is running at (RPi/Beaglebode/old Linux PC). So also in that respect it makes sense to do most work on the client.

I just don't want this to be another "everyone doing things in parallel and nothing is moving forward" project.

If you want mdf handling and discovery on the server it really belongs in the embedded web interface (/vscp) and not in external files. Thus should be embedded. It's terrible coding and terrible maintenance in this but the end result is good (== worth the effort). But of course I can't stop anyone from going there own way with things but it does not help the project a bit.

/Ake

Den 2015-08-10 kl. 21:26, skrev Andreas Merkle:

If there is no server (and the daemon is one), there is no web interface which needs the benefits of the XQuery functionality. vscpWorks can live without that kind of functionality, but in my eyes this feature will push the web interface development (provided by the daemon) fast forward and is a must have dealing with xml.

If you want to use the same html5/javascript code for web interfaces, provided by nodes, every node has to provide the daemon websocket/REST API. Otherwise it won't work with the same source code. If they have so many space in the flash, why they don't provide the XQuery functionality too?

If you can access the nodes without internet access, they will have the mdf included. Perfect ... because the html5/javascript source doesn't need to differ where the mdf comes from. It uses only the provided REST/websocket API and the XQuery command.

Another advantage is the fast loading web interface. There is no need to download the whole mdf and download huge javascript libraries to evaluate them.

How did my first example look like without XQuery:

1. Retrieve the mdf url.
2. Download the mdf.
3. Convert the mdf xml to a javascript object.
4. Walk through the object and check every single part with "if"-clauses.

My two cents about ...

— Reply to this email directly or view it on GitHub https://github.com/grodansparadis/vscp_software/issues/161#issuecomment-129574721.

Be Hungry - Stay Foolish! Åke Hedman, Grodans Paradis AB / Paradise of the Frog Brattbergavägen 17, 82050 LOS, SWEDEN Phone: +46 (0)840011835 CellPhone: +46 (0)730-533146 http://www.grodansparadis.com http://www.vscp.org

---

Detta e-postmeddelande har sökts igenom efter virus med antivirusprogram från Avast. https://www.avast.com/antivirus

[BlueAndi]

Of course, I agree with the single source idea.

If the node discovery and mdf handling is done in the embedded web interface, I still need access functions via REST/websocket to the discovered nodes and to the mdf data itself. Or not?

Do you have some class diagrams about the classes and their dependencies or deployments, e.g. how the daemon is deployed with which classes? I have the feeling I miss the overview.

And maybe offtopic, but during reading your answers you wrote several project goals. That is something which is very important for all to understand the decisions later. If there are some minutes ... it may be a good idea to write them down and present them on vscp.org on a separate page.

Andi

[grodansparadis]

It more or less cooks down to the register abstraction model. How do you read a register? Yes a tcp/ip interface can be exported by a daemon, a wfifi node, an Ethernet node, etc. Phones/tablets handle that fine in Javascript. But the same interface could be exported through wbsocket/REST also as you mention but not only from the daemon but also from a node that have these capabilities. But BLE is a serial link that exports the lower parts of the tcp/ip interface. tablets/phones can handle that in Javascript. Oh well you get the picture. But to level things would be easy if the read register is the upper or at least a middle abstraction. How it is read/written is determined by lower levels.

Class diagram. It's more about interfaces. CANAL, TCP/IP, websockets and REST is what we have now on logical level. Expect MQTT and CoAP added to that. Under that it is physical formats. I wil try to write something about it.

Project goals.

Most people don't care. You are the first to ask of such a thing. As you have notices I don't even care to ask people on the list what they think anymore. I never get a response. We used to have hard open discussions in the past. It took time and a lot of words where written but was fun and useful. I will try to write something up.

[BlueAndi]

Hey ... I was the first reading the whole specification (seems so) and the first asking the important thing, the project goals. :-) I don't believe that really all people don't care about the project goals. In business I see often that a project without any clear communicated project or work package goal didn't work.

But this is now really offtopic LOL.

[grodansparadis]

I have that clear in my head sine the beginning of the 80th. Remember this has been a one man operation. But I think it is pretty clear from the specification where this is heading. But as you say this is off topic.

Den 2015-08-13 kl. 21:18, skrev Andreas Merkle:

Hey ... I was the first reading the whole specification (seems so) and the first asking the important thing, the project goals. :-) I don't believe that really all people don't care about the project goals. In business I see often that a project without any clear communicated project or work package goal didn't work.

But this is now really offtopic /LOL/.

— Reply to this email directly or view it on GitHub https://github.com/grodansparadis/vscp_software/issues/161#issuecomment-130809555.

Be Hungry - Stay Foolish! Åke Hedman, Grodans Paradis AB / Paradise of the Frog Brattbergavägen 17, 82050 LOS, SWEDEN Phone: +46 (0)840011835 CellPhone: +46 (0)730-533146 http://www.grodansparadis.com http://www.vscp.org