terminate called after throwing an instance of 'std::out_of_range

[sweetw0r]

<?xml version='1.0' encoding='UTF-8'?>
<Status>
  <Maestro>
    <PlayingCompositions>
      <Composition>
        <InstanceID>8248-6FD02E10-314B-42E1-BF48-EA61BAE30E11</InstanceID>
        <CompositionPath></CompositionPath>
        <CompositionName>Draft of All Systems created on October 9, 2019 6:08:45 PM UTC</CompositionName>
        <StartingUser>admin</StartingUser>
        <PlayList />
        <Status>Playing</Status>
      </Composition>
      <Composition>
        <InstanceID>8249-DDA207D3-7595-4D2F-B2AD-4204189D903E</InstanceID>
        <CompositionPath></CompositionPath>
        <CompositionName>Draft of All Systems created on October 9, 2019 6:08:45 PM UTC</CompositionName>
        <StartingUser>admin</StartingUser>
        <PlayList />
        <Status>Playing</Status>
      </Composition>
    </PlayingCompositions>
  </Maestro>
</Status>

Error message:

bash:~ $ jtm -d file.xml
terminate called after throwing an instance of 'std::out_of_range'
  what():  map::at
Aborted

[sweetw0r]

bash :/usr/local/bin $ gdb ./jtm 
GNU gdb (Raspbian 8.2.1-2) 8.2.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "arm-linux-gnueabihf".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./jtm...done.
(gdb) catch throw
Catchpoint 1 (throw)
(gdb) run
Starting program: /usr/local/bin/jtm 

Catchpoint 1 (exception thrown), 0x00043140 in __cxa_throw ()
(gdb) bt
#0  0x00043140 in __cxa_throw ()
#1  0x0006b180 in std::__throw_out_of_range(char const*) ()
#2  0x00023c70 in Getopt::parseInputArgs_(int, char**, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ()
#3  0x00023dc4 in Getopt::parse(int, char**, char const*) ()
#4  0x00010c44 in main ()
(gdb) 

[ldn-softdev]

Hi, did you compile jtm on your system or used a precompiled binary? In the former case, did you get any compiler warnings?

the exception occurs in the Getopt::parseInputArgs_ method at om_.at(option) = ... place when parsing option -d (without that option jtm probably would convert your xml just fine). A parameter option is received from the system call getopt, However, by that time the std::map om_ should have been already defined with all the options - thus, such throw is entirely unexpected.

That means that either a system call getopt(...) did not work as expected, or something happened earlier in the code when options were getting defined (at the beginning of the main() function) - all that long before xml parsing kicks in.

That issue is not reproducible on my systems, if you like to debug what happens there on your system, could you add one output line, like in that snippet below (in Getopt::parseInputArgs_ call):

...
 while((option = getopt(argc, argv, fmt.c_str())) != -1) {      // read next option character
  if(option == ':')
   throw EXP(opt_argument_missing);                             // missing argument, forced ':'
  std::cerr << "fmt: '" << fmt << "', option: '" << option << "'" << std::endl;
...

then recompile and run it again with -d option, let me see the output - it'll be clearer what happens there.

[sweetw0r]

I compiled in my system. No warnings:

bash:~/jtm $ gcc --version
gcc (Raspbian 8.3.0-6+rpi1) 8.3.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

bash:~/jtm $ ls
jtm.cpp  jtm-linux-32.v2.08  jtm-linux-64.v2.08  jtm-macos-32.v2.08  jtm-macos-64.v2.08  lib  LICENSE  README.md

bash:~/jtm $ c++ -o jtm -Wall -std=gnu++14 -static -Ofast jtm.cpp
bash:~/jtm $ ls
jtm  jtm.cpp  jtm-linux-32.v2.08  jtm-linux-64.v2.08  jtm-macos-32.v2.08  jtm-macos-64.v2.08  lib  LICENSE  README.md

bash:~/jtm $ sudo mv ./jtm /usr/local/bin/
bash:~/jtm $ ls
jtm.cpp  jtm-linux-32.v2.08  jtm-linux-64.v2.08  jtm-macos-32.v2.08  jtm-macos-64.v2.08  lib  LICENSE  README.md

bash:~/jtm $ jtm # ... all that long before xml parsing kicks in
terminate called after throwing an instance of 'std::out_of_range'
  what():  map::at
Aborted 

bash:~/jtm $ jtm ~/file.xml 
terminate called after throwing an instance of 'std::out_of_range'
  what():  map::at
Aborted

I updated getoptions.hpp with:

std::cerr << "fmt: '" << fmt << "', option: '" << option << "'" << std::endl;

Output:

bash:~ $ jtm -d ~/file.xml 
fmt: 'a:defi:nqrh', option: 'd'
fmt: 'a:defi:nqrh', option: '?'
terminate called after throwing an instance of 'std::out_of_range'
  what():  map::at
Aborted
bash:~ $ jtm
fmt: 'a:defi:nqrh', option: '?'
terminate called after throwing an instance of 'std::out_of_range'
  what():  map::at
Aborted

[ldn-softdev]

okay, I think I know what's going on there.

getopt() at the end of processing of all the options (in this case a:defi:nqrh), must return the value -1, but apparently, in Raspberry Pi it returns some other value (less than -1) - that's why the loop:

while((option = getopt(argc, argv, fmt.c_str())) != -1)

does not get terminated with != -1 condition and processes returned negative value as an option. on this output:

fmt: 'a:defi:nqrh', option: '?'

the char '?' represents not a question mark character per se, but rather some other negative value. Because if it was really the char ?, then it would definitely hit these conditions:

  if(option == '?')                                             // '?' means invalid option
   if(throwException_) {                                         // if throwing is allowed
    exception_ = optopt;                                        //   optopt = either unknown option,
    throw fmt.find(exception_) == std::string::npos?            //   (if not found in fmt string)
           EXP(invalid_option): EXP(opt_argument_missing);      //   or missing arg, throw anyway
   }                                                            // otherwise let user handle it

and exception invalid_option or opt_argument_missing would be thrown.
However, it sneaks past that code (which proves that it's not a question mark character) and continues further down the code until it hits this line, where it throws at map:at:

   om_.at(option).hit();                                        // increment hit for boolean

You can confirm my theory by altering the line that I asked you to insert into this one:

  std::cerr << "fmt: '" << fmt << "', option: '" << (int)option << "'" << std::endl;

and see what value the option holds. If it's indeed some other value below -1, then the fix would be easy - alter the loop condition like this:

 while((option = getopt(argc, argv, fmt.c_str())) >= 0) {      // read next option character

- let me know, /Dmitry

[sweetw0r]

Hi Dmitry,

Thanks for looking into this! I tied and got 100 and 255:

bash:~ $ jtm -d
fmt: 'a:defi:nqrh', option: '100'
fmt: 'a:defi:nqrh', option: '255'
terminate called after throwing an instance of 'std::out_of_range'
  what():  map::at
Aborted
bash:~ $ jtm -d ~/file.xml 
fmt: 'a:defi:nqrh', option: '100'
fmt: 'a:defi:nqrh', option: '255'
terminate called after throwing an instance of 'std::out_of_range'
  what():  map::at
Aborted

[ldn-softdev]

so, my suspicions were right, if we convert int 255 into a signed 1-byte value we arrive to -127, thus indeed, unlike on other systems, on Raspberry Pi getopt() returns other than -1 value.

Thus the fix I told you was a correct one:

 while((option = getopt(argc, argv, fmt.c_str())) >= 0) {      // read next option character

for a better compatibility/portability, I'll update it also in my source.

btw, you're using 2.08 version while the latest available is 2.09 - I suggest you to use that one. - I will update the release section and trim up the git source upon publishing my next version 2.10 (where, btw, I'm going to introduce a couple of new features: streamed read support, so that jtm could be used in streams (in-flight) conversions, like ... | jtm <args> | ... and also will add a conversion from JSON to csv format (unlike most of other convertors it'll be capable of converting irregular JSON to .csv format).

[sweetw0r]

while((option = getopt(argc, argv, fmt.c_str())) >= 0) {      // read next option character
  if(option == ':')
   throw EXP(opt_argument_missing);                             // missing argument, forced ':'
  std::cerr << "fmt: '" << fmt << "', option: '" << (int)option << "'" << std::endl;
  if(option == '?')                                             // '?' means invalid option

Having this did not fix it unfortunately:

bash:~ $ jtm -d
fmt: 'a:defi:nqrh', option: '100'
fmt: 'a:defi:nqrh', option: '255'
terminate called after throwing an instance of 'std::out_of_range'
  what():  map::at
Aborted
bash:~ $ jtm -d ~/file.xml
fmt: 'a:defi:nqrh', option: '100'
fmt: 'a:defi:nqrh', option: '255'
terminate called after throwing an instance of 'std::out_of_range'
  what():  map::at
Aborted

Here is output of getopt() bore it gets aborted:

std::cerr << "fmt: '" << fmt << "', getopt(): '" << getopt(argc, argv, fmt.c_str()) << "'" << std::endl;

bash:~/jtm $ jtm -d
fmt: 'a:defi:nqrh', getopt(): '-1'
fmt: 'a:defi:nqrh', getopt(): '-1'
terminate called after throwing an instance of 'std::out_of_range'
  what():  map::at
Aborted

bash:~/jtm $ jtm -d
fmt: 'a:defi:nqrh', getopt(): '-1'', argc: '2'', argv: '0x7ebd7694'
--------------------
fmt: 'a:defi:nqrh', getopt(): '-1'', argc: '2'', argv: '0x7ebd7694'
--------------------
terminate called after throwing an instance of 'std::out_of_range'
  what():  map::at
Aborted

[ldn-softdev]

You're right. Don't know what I was thinking, but indeed, converting int 255 to a single byte signed value produces -1, not -127, sorry for that.

After scratching my head for a while, how comes that in your last output, getopt(...) returns int -1 value, while when it's getting assigned to the option in the loop, it's 255 suddenly, I realized, that I was looking at a different version source - in my latest source (not published yet) - that issue is already addressed - i.e. option there is declared as int while in the published versions, it's still declared as char;

so, declare option variable with int type (the line above the while loop):

 int option;

- that should fix the issue, let me know, kind regards, /Dmitry

[sweetw0r]

Excellent! int option; worked!

bash:~ $ jtm ~/file.xml 
[
    {
        "?xml":{
            "attributes":{
                "encoding":"UTF-8",
                "version":"1.0"
            }
        }
    },
    {
        "Status":{
            "Maestro":{
                "PlayingCompositions":[
                    {
                        "Composition":[
                            {
                                "InstanceID":"8248-6FD02E10-314B-42E1-BF48-EA61BAE30E11"
                            },
                            {
                                "CompositionPath":[]
                            },
                            {
                                "CompositionName":"Draft of All Systems created on October 9, 2019 6:08:45 PM UTC"
                            },
                            {
                                "StartingUser":"admin"
                            },
                            {
                                "PlayList/":null
                            },
                            {
                                "Status":"Playing"
                            }
                        ]
                    },
                    {
                        "Composition":[
                            {
                                "InstanceID":"8249-DDA207D3-7595-4D2F-B2AD-4204189D903E"
                            },
                            {
                                "CompositionPath":[]
                            },
                            {
                                "CompositionName":"Draft of All Systems created on October 9, 2019 6:08:45 PM UTC"
                            },
                            {
                                "StartingUser":"admin"
                            },
                            {
                                "PlayList/":null
                            },
                            {
                                "Status":"Playing"
                            }
                        ]
                    }
                ]
            }
        }
    }
]

Spasibo Dmitry! 😍 Looking forward to that feature:

so that jtm could be used in streams (in-flight) conversions, like ... | jtm <args> | ...