Remote configuration framework: Add support for JSON

vanvught commented 3 years ago

Adding support for JSON makes it easer to work with standard (mobile) applications.

[x] lib-remoteconfig -> Add JSON output, make it possible to parse JSON. With MCU implementations in mind, there must be a minimal of RAM being used.
[x] lib-remoteconfig -> Add a HTTP server for JSON GET/PUT requests. With MCU implementations in mind, there is just one HTTP connection possible. Also, this single connections makes it easier to work with exclusive read/writes to the SPI flash -> non-blocking. Support for HTTP 1.1 only.
[x] lib-network -> Add support for TCP. This will be a minimum RFC793 implementation for the server side only.

TCP Non-implemented features:

dynamic receive window
URG flag and urgent pointer
delayed ACK
queueing out-of-order TCP segments
security/compartment
precedence
user timeout
retransmission (subject for future implementation).

cc: @hippyau @rbarreiros

vanvught commented 3 years ago

Need to run as sudo as the http server port is below 1024

sudo ./linux_e131 eno1

arjanvanvught@MacBook-Air tmp % wget nuc-i7/json/e131.txt                                                        
--2021-09-02 20:19:38--  http://nuc-i7/json/e131.txt
Herleiden van nuc-i7 (nuc-i7)... 192.168.2.107
Verbinding maken met nuc-i7 (nuc-i7)|192.168.2.107|:80... verbonden.
HTTP-verzoek is verzonden; wachten op antwoord... 200 OK
Lengte: 324 [application/json]
Wordt opgeslagen als: ‘e131.txt’

e131.txt                       100%[==================================================>]     324  --.-KB/s    in 0s      

2021-09-02 20:19:38 (8,13 MB/s) - '‘e131.txt’' opgeslagen [324/324]

arjanvanvught@MacBook-Air tmp % cat e131.txt 
{"e131.txt":{"direction":"output","universe":1,"universe_port_a":1,"universe_port_b":2,"universe_port_c":3,"universe_port_d":4,"merge_mode":"htp","merge_mode_port_a":"htp","merge_mode_port_b":"htp","merge_mode_port_c":"htp","merge_mode_port_d":"htp","network_data_loss_timeout":2.5,"disable_merge_timeout":0,"priority":100}}

arjanvanvught@MacBook-Air tmp % vi e131.txt

{"e131.txt":{"direction":"output","universe":14,"universe_port_a":1,"universe_port_b":2,"universe_port_c":3,"universe_port_d":4,"merge_mode":"htp","merge_mode_port_a":"htp","merge_mode_port_b":"htp","merge_mode_port_c":"htp","merge_mode_port_d":"htp","network_data_loss_timeout":2.5,"disable_merge_timeout":0,"priority":100}}

arjanvanvught@MacBook-Air tmp % wget --post-file=e131.txt --header=Content-Type:application/json nuc-i7/submit
--2021-09-02 20:25:34--  http://nuc-i7/submit
Herleiden van nuc-i7 (nuc-i7)... 192.168.2.107
Verbinding maken met nuc-i7 (nuc-i7)|192.168.2.107|:80... verbonden.
HTTP-verzoek is verzonden; wachten op antwoord... 200 OK
Lengte: 91 [text/html]
Wordt opgeslagen als: ‘submit’

submit                         100%[==================================================>]      91  --.-KB/s    in 0s      

2021-09-02 20:25:34 (2,17 MB/s) - '‘submit’' opgeslagen [91/91]

arjanvanvught@MacBook-Air tmp % cat submit 
<!DOCTYPE html>
<html>
<head><title>Submit</title></head>
<body><h1>OK</h1></body>
</html>

[x] The Java util does not work anymore at this time -> need to fix this.

@rbarreiros @hippyau

hippyau commented 3 years ago

Awesome!! :1st_place_medal:

hippyau commented 3 years ago

It seems to be working great, this is awesome!

hippyau commented 3 years ago

Hey Arjan, I've got a public repo for the flutter app I'm working on, just for your info... https://github.com/hippyau/RNC. It basically grabs .txt files, cleans and formats the keys, lets user edit the values... Does not send back to node yet, of discover nodes.

screenshot

vanvught commented 3 years ago

Example: http://allwinner_240813.local/get/version

Note: mDNS is enabled.

vanvught commented 3 years ago

@hippyau Hi Hip,

of discover nodes.

What do you think about using mDNS discovery services for finding the node's? For example: _http._tcp services registration. Or is there a JSON mDNS services ? I will do some research.

Groet, Arjan

hippyau commented 3 years ago

Hey Arjan, I forgot you support mDNS... there is good support in Flutter I can see, as everyone wants it to use their Chromecasts on IOS/Android.

I'll crack on having a go at that :)

Cheers!

vanvught commented 3 years ago

Hi Hip, I've added mDNS Service record for HTTP. The TXT field contains the node type.

I have ran linux_e131 in Docker -> docker run --rm -dit --network pub_net -p 5353/udp -p 80/tcp e131mon

/docker/e131mon# cat Dockerfile 
FROM ubuntu:latest
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
COPY . /usr/src/app
CMD [ "./linux_e131", "eth0", "1","0" ]

hippyau commented 3 years ago

TCP Non-implemented features:

* dynamic receive window
* URG flag and urgent pointer
* delayed ACK
* queueing out-of-order TCP segments
* security/compartment
* precedence
* user timeout
* retransmission (subject for future implementation).

I saw this in my dmesg occasionally, it doesn't seem to be a problem, just thought I'd mention it.
TCP: request_sock_TCP: Possible SYN flooding on port 80. Sending cookies. Check SNMP counters. It wasn't an error, just a message. Still, seems to work fine. Very impressed with your progress here, it does seems to work great!

RE mDNS, I have some discovered some services on _http._tcp.local already in Flutter/Dart, it should be straight forward thing to get going. I can see localhost OLA instance, but not linux_e131 (no docker, pull/compile 5 minutes ago). I killed OLA and still nope. I am a docker noob in every way, virgin even, kind of avoided it.... now I will need it :) ... have some tooling to do, follow your previous instructions to emulate multiple nodes.

hippyau commented 3 years ago

Hey Arjan, I'm going to suggest maybe nice if all the JSON comes to/from /json/something[.txt]? https://github.com/vanvught/rpidmx512/blob/2d3e51a2696c911c813a4da431869634d70df172/lib-remoteconfig/src/httpd/httd.cpp#L297

hippyau commented 3 years ago

Example: http://allwinner_240813.local/get/version

Note: mDNS is enabled.

vanvught commented 3 years ago

Hey Arjan, I'm going to suggest maybe nice if all the JSON comes to/from /json/something[.txt]?

Ok. I will do some redesign. Btw: I've found an issue with the HTTP mDNS registration; the Answer contains: allwinner_240813._http._udp.local Where there is udp instead of tcp

hippyau commented 3 years ago

Ok. I will do some redesign.

No stress, it's working, just a thought :) Work in progress!

Btw: I've found an issue with the HTTP mDNS registration

I still can't see it, trying to work out my docker issues :)

I see...

vanvught commented 3 years ago

wget --post-file=e131.txt --header=Content-Type:application/json nuc-i7/json
--2021-09-05 19:47:16--  http://nuc-i7/json
Herleiden van nuc-i7 (nuc-i7)... 192.168.2.107
Verbinding maken met nuc-i7 (nuc-i7)|192.168.2.107|:80... verbonden.
HTTP-verzoek is verzonden; wachten op antwoord... 200 OK
Lengte: 91 [text/html]
Wordt opgeslagen als: ‘json’

json                             100%[=======================================================>]      91  --.-KB/s    in 0s      

2021-09-05 19:47:16 (4,13 MB/s) - '‘json’' opgeslagen [91/91]

 cat json 
<!DOCTYPE html>
<html>
<head><title>Submit</title></head>
<body><h1>OK</h1></body>
</html>

/json/get/version

hippyau commented 3 years ago

Awesome, i'm going to get my post code going now....

You are the best to advise here... (excuse the terrible lines!)

So far, These are the only hard-coded things in my app.... If there is enough space/ram in your opinion, can we have a 'directory' JSON for the firmwares, simple as returning a hardcoded string to suit the the firmware version, advising what the .txt files are called.

eg: {"directory.txt": {"network.txt": "Network", "e131.txt": "sACN (E1.31)",...... }}

Then the app is totally dynamic, showing tabs for all the files available based on the firmware it's talking to?

I use local processing to pretty print the setting names, so consistency you already have like "usedhcp" ... `usecould indicate a 0/1 bool (we draw a switch) andmask/ip,_gateway,server` can be a IP address entry fields etc....

vanvught commented 3 years ago

Hi Hip,

eg: {"directory.txt": {"network.txt": "Network", "e131.txt": "sACN (E1.31)",...... }}

Whererconfig.txt, network.txt are always available, I have added the GET /json/list API

Note that I have removed /get from URI.

The keywords for type can be found here:

Is this working for you?

Groet, Arjan

vanvught commented 3 years ago

Second thought; I should add the .txt files to the list output. We have plenty enough SPI flash 🙂

hippyau commented 3 years ago

i love you :)

I was getting at, just a raw literal string per firmware, returning example...

{
  "directory": {
    "network.txt": "Network",
    "e131.txt": "sACN (E1.31)",
    "rconfig.txt": "Remote Settings",
    "version": "Node Version",
    "uptime": "Node Uptime"
  }
}


uint16_t json_get_directory(char *pOutBuffer, const uint16_t nOutBufferSize) {
    const char* directory = "";  // per firmware, as above...
    const uint16_t nLength = static_cast<uint16_t>(snprintf(pOutBuffer, nOutBufferSize, "{\"directory\":%s}", directory));
    return nLength;
}

vanvught commented 3 years ago

GET /json/directory API

Returning an array of file objects.

vanvught commented 3 years ago

Example rebooting the system:

wget --post-file=reboot --header=Content-Type:application/json 192.168.2.139/json/action

cat reboot                                                                              
{"reboot":1}

When running on Linux, then please be aware that this will reboot your system.

hippyau commented 3 years ago

Hey Arjan,

I hope you might be okay with this: "directory": { "filename" : "description",... }

This way, we return a file called 'directory' with just filenames and descriptions of what the file is. All JSON parsing engines can make a map of this, so order the files are displayed are also on the device.

where filename contains .txt is ediitable...

uint16_t json_get_directory(char *pOutBuffer, const uint16_t nOutBufferSize) {
    const uint16_t nLength = static_cast<uint16_t>(snprintf(pOutBuffer, nOutBufferSize,
            "{\"directory\":{"
            "\"network.txt\":\"Network\","

#if defined (NODE_ARTNET)
            "\"artnet.txt\":\"Art-Net\","
#endif
#if defined (NODE_E131)
            "\"e131.txt\":\"sACN E1.31\","
#endif
#if defined (NODE_OSC_CLIENT)
            "\"oscclnt.txt\":\"OSC Client\","
#endif
#if defined (NODE_OSC_SERVER)
            "\"osc.txt\":\"OSC Server\","
#endif
#if defined (NODE_LTC_SMPTE)
            "\"ltc.txt\":\"Linear Time Code\","
            "\"ldisplay.txt\":\"LED Display\","
            "\"tcnet.txt\":\"TC-Net\","
            "\"gps.txt\":\"GPS\","
#endif
#if defined(NODE_SHOWFILE)
            "\"show.txt\":\"Show File\","
#endif
#if defined(NODE_DDP_DISPLAY)
            "\"ddpdisp.txt\":\"DDP\","
#endif
#if defined (OUTPUT_DMX_SEND)
            "\"params.txt\":\"DMX Parameter\","
#endif
#if defined (OUTPUT_DMX_PIXEL)
            "\"devices\":\"DMX Pixel\","
#endif
#if defined (OUTPUT_DMX_MONITOR)
            "\"mon.txt\":\"Monitor\","
#endif
#if defined (OUTPUT_DMX_SERIAL)
            "\"serial.txt\":\"DMX Serial\","
#endif
#if defined (OUTPUT_RGB_PANEL)
            "\"rgbpanel.txt\":\"RGB Panel\","
#endif
#if defined (RDM_RESPONDER)
            "\"sensors.txt\":\"RDM Sensors\","
            "\"subdev.txt\":\"RDMSub Devices\","
#endif
#if defined(DISPLAY_UDF)
            "\"display.txt\":\"Display\","
#endif
            "\"rconfig.txt\":\"Remote\","
            "\"version\":\"Version\","
            "\"uptime\":\"Uptime\""
            "}}"
            ));
    return nLength;
}

Returns:

{"directory":{"network.txt":"Network","e131.txt":"sACN E1.31","mon.txt":"Monitor","rconfig.txt":"Remote","version":"Version","uptime":"Uptime"}}

hippyau commented 3 years ago

ezgif-7-549d0f214c6a

hippyau commented 3 years ago

When running on Linux, then please be aware that this will reboot your system.

Confirmed :)

hippyau commented 3 years ago

Switches for boolean values, identified by "use_"* (any others?)

vanvught commented 3 years ago

Hi Hip,

I've made the change that it is now an array with name and label.

hippyau commented 3 years ago

Hey Arjan,

OK but my solution, the label and name tags are not really necessary and it leads to less elegant code on this end, just natively in Dart I can have just one JSON parser, that is generating the UI completely from the raw JSON. The only hardcoded thing is a call to fetch the /json/*

But with the name: and label: tags it breaks the model of all the other JSON responses (even version and uptime), so I'd need to duplicate code and have a separate parser code for just the directory file using this structure.

I guess I can do this, but I thought it was pretty obvious the "rgbpanel.txt": "RGB Panel" format is name/label.
If my suggestion is no good then that's okay, just seemed to stick with the existing theme a bit better.

Cheers, Hip

vanvught commented 3 years ago

Hi Hip,

There is an object files, where a file has a name and a label. The name has a value and the label has a value. This gives the opportunity to write clean code. For example JavaScript:

async function renderDirectory() {
            let directory=await getJSON('directory');
            let html=""
            for(let x in directory.files) {
                html+="<option value="+directory.files[x].name+">"+directory.files[x].label+"</option>";
            }
            document.getElementById("idDirectory").innerHTML=html;
}

The code is not aware of any content of the directory object; it just doing a for loop for all files -> object(name, label).

Similar for .txt files:

async function get_txt(sel) {
    let txt=await getJSON(sel);
    let html="";
    Object.keys(txt[sel]).forEach(function(key) {
        var value = txt[sel][key];
         html+="<tr><td>"+key+"</td><td>"+value+"</td></tr>";
    });
    document.getElementById("idTxt").innerHTML=html;
}

The UI is completely de-coupled from any firmware specific implementations.

vanvught commented 3 years ago

Hey Hip,

Ok :-) Reading the JavaScript code for txt files, we can do the same with directory. I will go back to my development environment. Give me some hours ;-)

Groet, Arjan

hippyau commented 3 years ago

hey no stress...... all good. My hack is working good, but I can deal with it :)

hippyau commented 3 years ago

just more specific code in the UI, and a appreciation of separation of UI and business, but when the device could return it in a nice natural order too without any adverse reaction for machines, it just make life easier. Love your work!

vanvught commented 3 years ago

I am learning JavaScript as well ;-)

async function renderDirectory() {
    let directory=await getJSON('directory');
    let html=""
    let filenames=Object.keys(directory["files"]);
    filenames.forEach(function(key) {
        var value = directory["files"][key];
        html+="<option value="+key+">"+value+"</option>";
    });
    document.getElementById("idDirectory").innerHTML=html;
}

hippyau commented 2 years ago

Hey Arjan, the files is working great :)

I'm having some kind of problem posting though. It seems to me maybe with ParseHeaderField() src/httpd/httd.cpp, line 293: m_nRequestContentLength=326

Which is present when wget sends the http post, but not when flutter sends the http post. I've tried adding it manually to the header...

    res = await http.post(
      Uri.parse(addr),
      headers: <String, String>{
        'Content-Type': 'application/json',
        'RequestContentLength': postbody.length.toString(),
      },
      body: postbody,
    );

but doesn't seem to have helped...

I understand is not your problem because wget works fine, I am just wondering where m_nRequestContentLength=326 comes from? I can see m_nFileDataLength=326 is valid in my post... ¯_(ツ)_/¯

I've captured this output of linux_e131 with a wget post that works...

Run() src/httpd/httd.cpp, line 94: m_Status=520, m_RequestMethod=2
ParseHeaderField() src/httpd/httd.cpp, line 293: m_nRequestContentLength=326
HandlePost() src/httpd/httd.cpp, line 381: m_nBytesReceived=523, m_nFileDataLength=326, m_nRequestContentLength=326 -> hasDataOnly=N
HandlePost() src/httpd/httd.cpp, line 408: 326|{"e131.txt": {"direction":"output","universe":6,"universe_port_a":1,"universe_port_b":2,"universe_port_c":3,"universe_port_d":4,"merge_mode":"htp","merge_mode_port_a":"htp","merge_mode_port_b":"htp","merge_mode_port_c":"htp","merge_mode_port_d":"htp","network_data_loss_timeout":2.5,"disable_merge_timeout":0,"priority":100}}
|->0

--> src/remoteconfig.cpp:HandleTxtFile:1005
HandleTxtFile() src/remoteconfig.cpp, line 1030: 
HandleTxtFile() src/remoteconfig.cpp, line 1033: JSON
0x55b05c418c05:280
0000  23 65 31 33 31 2e 74 78  74 0a 64 69 72 65 63 74  #e131.tx t.direct
0010  69 6f 6e 3d 6f 75 74 70  75 74 0a 75 6e 69 76 65  ion=outp ut.unive
0020  72 73 65 3d 36 0a 75 6e  69 76 65 72 73 65 5f 70  rse=6.un iverse_p
0030  6f 72 74 5f 61 3d 31 0a  75 6e 69 76 65 72 73 65  ort_a=1. universe
0040  5f 70 6f 72 74 5f 62 3d  32 0a 75 6e 69 76 65 72  _port_b= 2.univer
0050  73 65 5f 70 6f 72 74 5f  63 3d 33 0a 75 6e 69 76  se_port_ c=3.univ
0060  65 72 73 65 5f 70 6f 72  74 5f 64 3d 34 0a 6d 65  erse_por t_d=4.me
0070  72 67 65 5f 6d 6f 64 65  3d 68 74 70 0a 6d 65 72  rge_mode =htp.mer
0080  67 65 5f 6d 6f 64 65 5f  70 6f 72 74 5f 61 3d 68  ge_mode_ port_a=h
0090  74 70 0a 6d 65 72 67 65  5f 6d 6f 64 65 5f 70 6f  tp.merge _mode_po
00a0  72 74 5f 62 3d 68 74 70  0a 6d 65 72 67 65 5f 6d  rt_b=htp .merge_m
00b0  6f 64 65 5f 70 6f 72 74  5f 63 3d 68 74 70 0a 6d  ode_port _c=htp.m
00c0  65 72 67 65 5f 6d 6f 64  65 5f 70 6f 72 74 5f 64  erge_mod e_port_d
00d0  3d 68 74 70 0a 6e 65 74  77 6f 72 6b 5f 64 61 74  =htp.net work_dat
00e0  61 5f 6c 6f 73 73 5f 74  69 6d 65 6f 75 74 3d 32  a_loss_t imeout=2
00f0  2e 35 0a 64 69 73 61 62  6c 65 5f 6d 65 72 67 65  .5.disab le_merge
0100  5f 74 69 6d 65 6f 75 74  3d 30 0a 70 72 69 6f 72  _timeout =0.prior
0110  69 74 79 3d 31 30 30 0a                           ity=100.
--> src/remoteconfigstatic.cpp:GetIndex:74
GetIndex() src/remoteconfigstatic.cpp, line 75: nLength=326
0x55b05c418c06:16
0000  65 31 33 31 2e 74 78 74  0a 64 69 72 65 63 74 69  e131.txt .directi
GetIndex() src/remoteconfigstatic.cpp, line 89: nLength=8, i=3
<-- src/remoteconfigstatic.cpp:GetIndex:90
--> src/remoteconfig.cpp:HandleTxtFileE131:1239
src/e131paramsdump.cpp::Dump 'e131.txt':
 universe=6
 universe_port_a=1
 universe_port_b=2
 universe_port_c=3
 universe_port_d=4
<-- src/remoteconfig.cpp:HandleTxtFileE131:1261
<-- src/remoteconfig.cpp:HandleTxtFile:1147
Run() src/httpd/httd.cpp, line 163: m_nContentLength=91
read failed: Resource temporarily unavailable

Next, my flutter app does a post....


Run() src/httpd/httd.cpp, line 94: m_Status=520, m_RequestMethod=2
HandlePost() src/httpd/httd.cpp, line 381: m_nBytesReceived=520, m_nFileDataLength=326, m_nRequestContentLength=0 -> hasDataOnly=N
Run() src/httpd/httd.cpp, line 163: m_nContentLength=109
read failed: Resource temporarily unavailable
Run() src/httpd/httd.cpp, line 94: m_Status=520, m_RequestMethod=2
--> src/remoteconfig.cpp:HandleGet:603
--> src/remoteconfigstatic.cpp:GetIndex:74
GetIndex() src/remoteconfigstatic.cpp, line 75: nLength=1435
0x55b05c418b4a:16
0000  65 31 33 31 2e 74 78 74  00 48 54 54 50 00 31 2e  e131.txt .HTTP.1.
GetIndex() src/remoteconfigstatic.cpp, line 89: nLength=8, i=3
<-- src/remoteconfigstatic.cpp:GetIndex:90
--> src/remoteconfig.cpp:HandleGetE131Txt:779
--> src/e131paramssave.cpp:Save:86
--> src/e131paramssave.cpp:Builder:47
Builder() src/e131paramssave.cpp, line 81: nSize=324
<-- src/e131paramssave.cpp:Builder:82
<-- src/remoteconfig.cpp:HandleGetE131Txt:784
<-- src/remoteconfig.cpp:HandleGet:733
HandleGetTxt() src/httpd/httd.cpp, line 364: nBytes=324
Run() src/httpd/httd.cpp, line 163: m_nContentLength=324
read failed: Resource temporarily unavailable

Also, this is new in the last two commits: read failed: Resource temporarily unavailable occurs often... it seems to work with get requests as they work fine.

I also noticed when I killed the linux_e131 after a post, my all caused an exception like the connection was still open? weird, it shouldn't be?

hippyau commented 2 years ago

0000 65 31 33 31 2e 74 78 74 00 48 54 54 50 00 31 2e e131.txt .HTTP.1. Just realized clearly something is wrong here... I'll keep looking.

vanvught commented 2 years ago

Hi Hip. that is correct debug output. With working with MCU's in mind, I am reusing buffers. You would see that the txt file name is '\0'terminated. And then it is showing the remaining left over data from the buffer.

vanvught commented 2 years ago

I understand is not your problem because wget works fine

Well it is my problem when you have an issue with our firmware ;-)

I have tested the POST with wget only. Can you grab the Wireshark capture for your application ? I have the idea that it might working different than the wget POST.

hippyau commented 2 years ago

:)

This is what it looks like in my debug...

flutter: set new value: Setting universe = 14 in /json/e131.txt
flutter: Post /json/e131.txt to the device...

flutter: Post to 'http://192.168.1.96/json': 

{"e131.txt": {"direction":"output","universe":14,"universe_port_a":1,"universe_port_b":2,"universe_port_c":3,"universe_port_d":4,"merge_mode":"htp","merge_mode_port_a":"htp","merge_mode_port_b":"htp","merge_mode_port_c":"htp","merge_mode_port_d":"htp","network_data_loss_timeout":2.5,"disable_merge_timeout":0,"priority":100}}

flutter: Post Return: <!DOCTYPE html>
<html>
<head><title>400 Bad Request</title></head>
<body><h1>Bad Request</h1></body>

flutter: Read /json/e131.txt from device

{"e131.txt":{"direction":"output","universe":1,"universe_port_a":1,"universe_port_b":2,"universe_port_c":3,"universe_port_d":4,"merge_mode":"htp","merge_mode_port_a":"htp","merge_mode_port_b":"htp","merge_mode_port_c":"htp","merge_mode_port_d":"htp","network_data_loss_timeout":2.5,"disable_merge_timeout":0,"priority":100}}

my post is 1 character longer ({"e131.txt": {"direction":"output", vs. {"e131.txt":{"direction":"output"), but this is not the problem as wgetting that from a file works too.

hippyau commented 2 years ago

Can you grab the Wireshark capture for your application

I don't have wireshark on this machine yet, i'm doing this right now....

vanvught commented 2 years ago

A hint the wget capture, where wget is sending the headers and payload in different TCP packages.

vanvught commented 2 years ago

Plain Java is also sending 2 TCP frames.

public static void main(String[] args) {
        String url = "http://192.168.2.139/json";
        try {
            URL obj = new URL(url);
            HttpURLConnection con = (HttpURLConnection) obj.openConnection();

            con.setRequestMethod("POST");
            con.setRequestProperty("Content-Type", "application/json");

            String postJsonData = "{\"network.txt\":{\"use_dhcp\":1,\"dhcp_retry_time\":0,\"ip_address\":\"192.168.2.139\",\"net_mask\":\"255.255.255.0\",\"default_gateway\":\"192.168.2.1\",\"hostname\":\"allwinner_240813\",\"ntp_server\":\"192.168.2.120\",\"ntp_utc_offset\":2.0}}";

            con.setDoOutput(true);
            DataOutputStream wr = new DataOutputStream(con.getOutputStream());
            wr.writeBytes(postJsonData);
            wr.flush();
            wr.close();

            int responseCode = con.getResponseCode();

            System.out.println("Sending 'POST' request to URL : " + url);
            System.out.println("Post Data : " + postJsonData);
            System.out.println("Response Code : " + responseCode);

            BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
            String output;
            StringBuffer response = new StringBuffer();

            while ((output = in.readLine()) != null) {
                response.append(output);
            }
            in.close();

            System.out.println(response.toString());
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

Sending 'POST' request to URL : http://192.168.2.139/json
Post Data : {"network.txt":{"use_dhcp":1,"dhcp_retry_time":0,"ip_address":"192.168.2.139","net_mask":"255.255.255.0","default_gateway":"192.168.2.1","hostname":"allwinner_240813","ntp_server":"192.168.2.120","ntp_utc_offset":2.0}}
Response Code : 200
<!DOCTYPE html><html><head><title>Submit</title></head><body><h1>OK</h1></body></html>

hippyau commented 2 years ago

It looks like I sent this...

0000   00 00 00 00 00 00 00 00 00 00 00 00 08 00 45 00   ..............E.
0010   00 f6 22 65 40 00 40 06 93 8c c0 a8 01 60 c0 a8   .."e@.@......`..
0020   01 60 e4 3e 00 50 75 38 06 1c 4f 43 89 d0 80 18   .`.>.Pu8..OC....
0030   02 00 84 f9 00 00 01 01 08 0a 2f 67 ab 8b 2f 67   ........../g../g
0040   ab 86 50 4f 53 54 20 2f 6a 73 6f 6e 20 48 54 54   ..POST /json HTT
0050   50 2f 31 2e 31 0d 0a 75 73 65 72 2d 61 67 65 6e   P/1.1..user-agen
0060   74 3a 20 44 61 72 74 2f 32 2e 31 33 20 28 64 61   t: Dart/2.13 (da
0070   72 74 3a 69 6f 29 0d 0a 63 6f 6e 74 65 6e 74 2d   rt:io)..content-
0080   74 79 70 65 3a 20 61 70 70 6c 69 63 61 74 69 6f   type: applicatio
0090   6e 2f 6a 73 6f 6e 3b 20 63 68 61 72 73 65 74 3d   n/json; charset=
00a0   75 74 66 2d 38 0d 0a 72 65 71 75 65 73 74 63 6f   utf-8..requestco
00b0   6e 74 65 6e 74 6c 65 6e 67 74 68 3a 20 33 32 36   ntentlength: 326
00c0   0d 0a 61 63 63 65 70 74 2d 65 6e 63 6f 64 69 6e   ..accept-encodin
00d0   67 3a 20 67 7a 69 70 0d 0a 63 6f 6e 74 65 6e 74   g: gzip..content
00e0   2d 6c 65 6e 67 74 68 3a 20 33 32 36 0d 0a 68 6f   -length: 326..ho
00f0   73 74 3a 20 31 39 32 2e 31 36 38 2e 31 2e 39 36   st: 192.168.1.96
0100   0d 0a 0d 0a                                       ....

can see extra fields in the header...

hippyau commented 2 years ago

yep there is two packets... the post and then the data... the post is the first one in the screen shot and the data is the selected one.

hippyau commented 2 years ago

vanvught commented 2 years ago

You set 'Content-Type': 'application/json', however I see 'content-type' which is not recognized. Hence the BAD_REQUEST

vanvught commented 2 years ago

My bug :-)

From RFC 2616 - "Hypertext Transfer Protocol -- HTTP/1.1", Section 4.2, "Message Headers":

Each header field consists of a name followed by a colon (":") and the field value. Field names are case-insensitive.

hippyau commented 2 years ago

Ahh.... I defo set....

'Content-Type': 'application/json',

the library must be lower-case it.... maybe the standard is lower?

hippyau commented 2 years ago

vanvught commented 2 years ago

Hi Hip, the bug is fixed. Just did the commit.

hippyau commented 2 years ago

Arjan, you are an absolute legend....... 3 minutes and bam!

<html>
<head><title>Submit</title></head>
<body><h1>OK</h1></body>
</html>

hippyau commented 2 years ago

This shows is working. When a setting is modified, you click 'Save' to upload. The settings are then immediately read back... There is little/no validation on the client side, so I'll throw some crazy values at it.

Note the drop down tabs appear completely in order of files object from /json/directory, which is why I put network.txt first in json_get_directory, as every node will have it?

simplescreenrecorder-2021-09-14_23 31 09

vanvught commented 2 years ago

With #110 I am about to remove "Universe" and "Merge Mode". And then we have port specific settings only. I can not guarantee that it won't break backwards compatibility. But for easy of software management, we really need to remove the single port DMX references.

vanvught / rpidmx512

Remote configuration framework: Add support for JSON #209