TimescaleDB support - Githubissues

Diego91RA commented 5 years ago

Hello again.

After compiling CM from "master" branch I've tried to execute the server and got such output:

./bin/hdb++cm-srv mpd-ecal-sc-1 terminate called after throwing an instance of 'std::logic_error' what(): basic_string::_M_construct null not valid Aborted

Also I've tried to compile from integrated-build branch, and noticed include, which is different from master.

include <hdb++/HdbClient.h>

It's missing and I don't know how to find it.

So I've tried to replace HdbConfigurationManager.cpp and .h with files from master branch with old include style and compile the project again. Now I have output like this:

tango@mpd-ecal-sc-1:~/Downloads/hdbpp-cm/build$ make [ 16%] Linking CXX executable hdb++cm-srv /usr/bin/ld: CMakeFiles/hdbpp_cm.dir/src/HdbConfigurationManager.cpp.o: in function HdbConfigurationManager_ns::HdbConfigurationManager::delete_device()': HdbConfigurationManager.cpp:(.text+0x1e15): undefined reference toHdbClient::~HdbClient()' /usr/bin/ld: CMakeFiles/hdbpp_cm.dir/src/HdbConfigurationManager.cpp.o: in function HdbConfigurationManager_ns::HdbConfigurationManager::init_device()': HdbConfigurationManager.cpp:(.text+0x2888): undefined reference toHdbClient::HdbClient(std::vector<std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::cxx11::basic_string<char, std::char_traits, std::allocator > > >)' /usr/bin/ld: CMakeFiles/hdbpp_cm.dir/src/HdbConfigurationManager.cpp.o: in function HdbConfigurationManager_ns::HdbConfigurationManager::attribute_add()': HdbConfigurationManager.cpp:(.text+0xc147): undefined reference toHdbClient::configure_Attr(std::__cxx11::basic_string<char, std::char_traits, std::allocator >, int, int, int, unsigned int)' collect2: error: ld returned 1 exit status make[2]: [CMakeFiles/hdbpp_cm.dir/build.make:150: hdb++cm-srv] Error 1 make[1]: [CMakeFiles/Makefile2:73: CMakeFiles/hdbpp_cm.dir/all] Error 2 make: *** [Makefile:130: all] Error 2

chedburgh commented 5 years ago

Do not use the integrated-build branch yet. This is a work in progress across several repositories.

What LibConfiguration parameters are you using? I think this error is coming from libhdbpp-timescale (logging).

Thanks Stuart

Diego91RA commented 5 years ago

I use almost the same as for ES, except for username.

connect_string=user=hdb_cfg_man password=hdbpp host=localhost port=5432 dbname=hdb logging_level=error log_file=false log_console=false libname=libhdb++timescale.so

chedburgh commented 5 years ago

Two things that can help find the problem.

Get a core trace:

ulimit -c unlimited

Then get the stack trace using gdb.

Another thing that may help is to change log_console=true and logging_level=debug, this way, when you run the binary from the command line, it will give a full trace from the timescale back end.

I'll try test the config in the meantime and see if I can solve it.

Diego91RA commented 5 years ago

From console with -v5 option:

.... 1573220274 [139961893630272] DEBUG sys/tools/hdb++cm-srv HdbConfigurationManager::init_device() create device sys/tools/hdb++cm-srv 1573220274 [139961893630272] DEBUG dserver/hdb++cm-srv/mpd-ecal-sc-1 SubDevDiag::get_associated_device() entering ... 1573220274 [139961893630272] DEBUG dserver/hdb++cm-srv/mpd-ecal-sc-1 SubDevDiag::get_associated_device() found : sys/tools/hdb++cm-srv 1573220274 [139961893630272] DEBUG dserver/hdb++cm-srv/mpd-ecal-sc-1 SubDevDiag::register_sub_device() dev_name = sys/tools/hdb++cm-srv sub_dev_name = sys/tools/hdb++es-srv terminate called after throwing an instance of 'std::logic_error' what(): basic_string::_M_construct null not valid Aborted (core dumped)

From gdb:

Starting program: /home/tango/Downloads/hdbpp-cm/bin/hdb++cm-srv mpd-ecal-sc-1 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [New Thread 0x7ffff68fa700 (LWP 8539)] [New Thread 0x7ffff5ed8700 (LWP 8540)] [New Thread 0x7ffff56d7700 (LWP 8541)] [New Thread 0x7ffff4ed6700 (LWP 8542)] [New Thread 0x7fffeffff700 (LWP 8543)] [New Thread 0x7fffef7fe700 (LWP 8544)] [New Thread 0x7fffeeffd700 (LWP 8545)] terminate called after throwing an instance of 'std::logic_error' what(): basic_string::_M_construct null not valid Thread 1 "hdb++cm-srv" received signal SIGABRT, Aborted. __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50 50 return ret; (gdb) backtrace

0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50

1 0x00007ffff6fb7535 in __GI_abort () at abort.c:79

2 0x00007ffff737f983 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6

3 0x00007ffff73858c6 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6

4 0x00007ffff7385901 in std::terminate() () from /lib/x86_64-linux-gnu/libstdc++.so.6

5 0x00007ffff7385b34 in __cxa_throw () from /lib/x86_64-linux-gnu/libstdc++.so.6

6 0x00007ffff73817d3 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6

7 0x000055555557f3e1 in void std::__cxx11::basic_string<char, std::char_traits, std::allocator >::_M_construct<char const>(char const, char const*, std::forward_iterator_tag) ()

8 0x0000555555581bb7 in ?? ()

9 0x0000555555582181 in ?? ()

10 0x0000555555582b48 in ?? ()

11 0x00005555555839ef in ?? ()

12 0x000055555557aaa3 in ?? ()

13 0x00007ffff7d24f0f in Tango::DServer::init_device (this=0x55555560c320) at /usr/include/c++/8/bits/stl_vector.h:930

14 0x00007ffff7d25f30 in Tango::DServer::DServer (this=0x55555560c320, cl_ptr=, n=,

d=, s=, st=, __in_chrg=, __vtt_parm=) at dserver.cpp:106

15 0x00007ffff7d29b6c in Tango::DServerClass::device_factory (this=0x5555556096e0, devlist_ptr=0x7fffffffe260)

at /usr/local/include/omniORB4/stringtypes.h:374

16 0x00007ffff7d2b5a7 in Tango::DServerClass::DServerClass (this=0x5555556096e0, s=...) at dserverclass.cpp:1574

17 0x00007ffff7d2b7e7 in Tango::DServerClass::init () at dserverclass.cpp:1623

18 0x00007ffff7e4d0d5 in Tango::Util::server_init (this=0x5555555f2720, with_window=) at utils.cpp:1857

19 0x0000555555571923 in ?? ()

20 0x00007ffff6fb909b in __libc_start_main (main=0x555555571910, argc=2, argv=0x7fffffffe448, init=,

fini=, rtld_fini=, stack_end=0x7fffffffe438) at ../csu/libc-start.c:308

21 0x0000555555571a1a in ?? ()

chedburgh commented 5 years ago

Did you also set log_console=true and logging_level=debug in the LibConfiguration property?

chedburgh commented 5 years ago

Eg:

connect_string=user=hdb_cfg_man password=hdbpp host=localhost port=5432 dbname=hdb logging_level=debug log_file=false log_console=true libname=libhdb++timescale.so

Diego91RA commented 5 years ago

Yes, but without -v5 it does nothing.

chedburgh commented 5 years ago

Ok, thanks. That means its not getting as far as loading the shared library (so its not the issue I thought it may be).

Can you list any other properties set on the device server please. I'll see if we can get some more feedback on the issue.

bourtemb commented 5 years ago

Could you tell us what is the version of the Tango C++ library (cppTango) you are using please?

Diego91RA commented 5 years ago

I have only two properties, ArchiverList, value is sys/tools/hdb_es_1 (i've edited DS names) and LibConfiguration

I use TangoSourceDistribution 9.3.4-rc1, so cppTango must be cppTango 9.3.4-rc1

bourtemb commented 5 years ago

Hi, your problem seems similar to several ones reported on the Tango forum: https://www.tango-controls.org/community/forum/c/general/other/error-in-starting-configuration-manager-ds-archiver/?page=1#post-3981 https://www.tango-controls.org/community/forum/c/general/installation/a-problem-with-start-of-a-configurationmanager-via-astor/?page=1#post-3364 https://www.tango-controls.org/community/forum/c/general/installation/hdb-installation/?page=1#post-1330 https://www.tango-controls.org/community/forum/c/general/installation/hdb-installation/?page=1#post-847 It seems like none of these forum threads gave any solution to this issue...

Could you please tell us what is the operating system you are using?

Diego91RA commented 4 years ago

Hi, I use Debian 10.1, glibc is 2.28.

Diego91RA commented 4 years ago

The same code compiled and works without any problems on Debian 9.7, Tango 9.2.5a, glibc 2.24, omniORB-4.2.2 (now 4.2.3), ZMQ 4.2.5 (now 4.3.2).

Btw, why libtango.so in /usr/local/lib has so strange version in filename? I have 8.4.3, but Tango is 9.3.4

chedburgh commented 4 years ago

I am not sure if Tango is tested on Debian10? Hopefully @bourtemb can help with this.

Diego91RA commented 4 years ago

Reynald mentioned above that the problem is not so rare and occured couple of years ago, so I think OS version is not really important.

bourtemb commented 4 years ago

Btw, why libtango.so in /usr/local/lib has so strange version in filename? I have 8.4.3, but Tango is 9.3.4

This has been fixed in TangoSourceDistribution 9.3.4-rc2

We have some Travis tests for cppTango on Debian 10.

bourtemb commented 4 years ago

Reynald mentioned above that the problem is not so rare and occured couple of years ago, so I think OS version is not really important.

Well, what is important is to be able to reproduce the problem and for this, we need to identify SW versions where the problem occurs.

bourtemb commented 4 years ago

Hi, I tried on Debian 10 with tango 9.3.4-rc2 and I didn't get the error.... Maybe something different in the environment variables you are defining?

Diego91RA commented 4 years ago

These are envvars that I have:

SHELL=/bin/bash WINDOWID=3670029 HISTCONTROL=ignoreboth XTERM_VERSION=XTerm(344) TANGO_HOST=localhost:10000 LANGUAGE=en_US:en XTERM_SHELL=/bin/bash PWD=/home/tango LOGNAME=tango XDG_SESSION_TYPE=tty MC_TMPDIR=/tmp/mc-tango MC_SID=10902 HOME=/home/tango LANG=en_US.UTF-8 LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:.tar=01;31:.tgz=01;31:.arc=01;31:.arj=01;31:.taz=01;31:.lha=01;31:.lz4=01;31:.lzh=01;31:.lzma=01;31:.tlz=01;31:.txz=01;31:.tzo=01;31:.t7z=01;31:.zip=01;31:.z=01;31:.dz=01;31:.gz=01;31:.lrz=01;31:.lz=01;31:.lzo=01;31:.xz=01;31:.zst=01;31:.tzst=01;31:.bz2=01;31:.bz=01;31:.tbz=01;31:.tbz2=01;31:.tz=01;31:.deb=01;31:.rpm=01;31:.jar=01;31:.war=01;31:.ear=01;31:.sar=01;31:.rar=01;31:.alz=01;31:.ace=01;31:.zoo=01;31:.cpio=01;31:.7z=01;31:.rz=01;31:.cab=01;31:.wim=01;31:.swm=01;31:.dwm=01;31:.esd=01;31:.jpg=01;35:.jpeg=01;35:.mjpg=01;35:.mjpeg=01;35:.gif=01;35:.bmp=01;35:.pbm=01;35:.pgm=01;35:.ppm=01;35:.tga=01;35:.xbm=01;35:.xpm=01;35:.tif=01;35:.tiff=01;35:.png=01;35:.svg=01;35:.svgz=01;35:.mng=01;35:.pcx=01;35:.mov=01;35:.mpg=01;35:.mpeg=01;35:.m2v=01;35:.mkv=01;35:.webm=01;35:.ogm=01;35:.mp4=01;35:.m4v=01;35:.mp4v=01;35:.vob=01;35:.qt=01;35:.nuv=01;35:.wmv=01;35:.asf=01;35:.rm=01;35:.rmvb=01;35:.flc=01;35:.avi=01;35:.fli=01;35:.flv=01;35:.gl=01;35:.dl=01;35:.xcf=01;35:.xwd=01;35:.yuv=01;35:.cgm=01;35:.emf=01;35:.ogv=01;35:.ogx=01;35:.aac=00;36:.au=00;36:.flac=00;36:.m4a=00;36:.mid=00;36:.midi=00;36:.mka=00;36:.mp3=00;36:.mpc=00;36:.ogg=00;36:.ra=00;36:.wav=00;36:.oga=00;36:.opus=00;36:.spx=00;36:*.xspf=00;36: XTERM_LOCALE=en_US.UTF-8 SSH_CONNECTION=159.93.120.96 57957 10.18.11.136 22 XDG_SESSION_CLASS=user TERM=xterm TANGO_INC=/usr/local/include/tango USER=tango DISPLAY=localhost:10.0 SHLVL=2 XDG_SESSION_ID=1520 LD_LIBRARY_PATH=/usr/local/lib XDG_RUNTIME_DIR=/run/user/1000 SSH_CLIENT=159.93.120.96 57957 22 PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus MAIL=/var/mail/tango SSHTTY=/dev/pts/0 OLDPWD=/home/tango/Downloads/hdbpp-cm =/usr/bin/env

I only had to define TANGO_INC and LD_LIBRARY_PATH by myself to compile ConfigManager, everything else was found without problems.

I'll try to reinstall the system and Tango (rc2 instead of rc1 I have now) next week to see if there will be the same result.

Diego91RA commented 4 years ago

Hello again. I've tried to install the same setup on the VirtualBox VM and have the same problem. Here is link to the .vdi https://yadi.sk/d/8rRYMSS6h1xafw (11.3 Gb) user/pw: tango/tango-cs

I hope it will be helpful

bourtemb commented 4 years ago

@Diego91RA , thank you very much for the VM. I managed to reproduce the issue with this VM. I see a crash at the following line: https://github.com/tango-controls-hdbpp/hdbpp-cm/blob/master/src/HdbConfigurationManager.cpp#L3090

The crash occurs during the second iteration of this loop. So, it looks like, in the case of this VM, the result variable assigned during getaddrinfo call is composed of 2 struct addrinfo results. The second struct addrinfo contains a field ai_canonname which is NULL. This causes a crash because the following line tries to create a string from a null pointer: https://github.com/tango-controls-hdbpp/hdbpp-cm/blob/master/src/HdbConfigurationManager.cpp#L3090

getaddrinfo man page on this Debian Buster VM says the following:

If hints.ai_flags includes the AI_CANONNAME flag, then the ai_canon‐ name field of the first of the addrinfo structures in the returned list is set to point to the official name of the host.

So the hdbpp-cm code should consider only the ai_canonname field of the first addrinfo structure. So it looks like the loop is not needed here.

Diego91RA commented 4 years ago

Yes, it worked! Good news, thanks!

I've edited the loop like this:

Do I have to do the same with a loop here? https://github.com/tango-controls-hdbpp/hdbpp-cm/blob/master/src/HdbConfigurationManager.cpp#L3173

bourtemb commented 4 years ago

Do I have to do the same with a loop here? https://github.com/tango-controls-hdbpp/hdbpp-cm/blob/master/src/HdbConfigurationManager.cpp#L3173

This code is executed only when _MULTI_TANGO_HOST is defined, which is probably not your case. But the patch fixing the problem you reported will have to change this other loop too indeed.

Diego91RA commented 4 years ago

Ok, I'll do it by myself for now. There is another problem - the DS started fine, but it crashes when I try to add attribute to the subscriber via Configuration Manager :(

1574244570 [139950771140352] DEBUG dserver/hdb++cm-srv/mpd-ecal-sc-1 DeviceImpl::command_inout(): command received : AttributeAdd 1574244570 [139950771140352] DEBUG dserver/hdb++cm-srv/mpd-ecal-sc-1 SubDevDiag::get_associated_device() entering ... 1574244570 [139950771140352] DEBUG dserver/hdb++cm-srv/mpd-ecal-sc-1 SubDevDiag::get_associated_device() found : No associated device name! 1574244570 [139950771140352] DEBUG dserver/hdb++cm-srv/mpd-ecal-sc-1 SubDevDiag::set_associated_device() entering ... 1574244570 [139950771140352] DEBUG dserver/hdb++cm-srv/mpd-ecal-sc-1 Entering DeviceClass::command_handler() method 1574244570 [139950771140352] DEBUG sys/tools/hdb_cm_1 HdbConfigurationManager::always_executed_hook() sys/tools/hdb_cm_1 1574244570 [139950771140352] INFO dserver/hdb++cm-srv/mpd-ecal-sc-1 AttributeAddClass::execute(): arrived 1574244570 [139950771140352] DEBUG sys/tools/hdb_cm_1 HdbConfigurationManager::AttributeAdd() - sys/tools/hdb_cm_1 1574244570 [139950771140352] INFO sys/tools/hdb_cm_1 find_archiver: unable to read AttributeList from tango://mpd-ecal-sc-1.he.jinr.ru:10000/sys/tools/hdb_es_1 1574244570 [139950771140352] DEBUG dserver/hdb++cm-srv/mpd-ecal-sc-1 SubDevDiag::get_associated_device() entering ... 1574244570 [139950771140352] DEBUG dserver/hdb++cm-srv/mpd-ecal-sc-1 SubDevDiag::get_associated_device() found : sys/tools/hdb_cm_1 1574244570 [139950771140352] DEBUG dserver/hdb++cm-srv/mpd-ecal-sc-1 SubDevDiag::register_sub_device() dev_name = sys/tools/hdb_cm_1 sub_dev_name = tango://mpd-ecal-sc-1.he.jinr.ru:10000/sys/tg_test/1 1574244570 [139950771140352] DEBUG sys/tools/hdb_cm_1 HdbConfigurationManager::AttributeAdd() - data_type=5 Tango::DEV_STATE=19 Tango::DEV_BOOLEAN=1 1574244570 [139950771140352] DEBUG sys/tools/hdb_cm_1 HdbConfigurationManager::AttributeAdd() - before read attribute config attr=double_scalar 1574244570 [139950771140352] DEBUG sys/tools/hdb_cm_1 HdbConfigurationManager::AttributeAdd() - read attribute config size=1 1574244570 [139950771140352] DEBUG sys/tools/hdb_cm_1 HdbConfigurationManager::AttributeAdd() - setting archive_abs_change=0.01 1574244570 [139950771140352] DEBUG sys/tools/hdb_cm_1 HdbConfigurationManager::AttributeAdd() - setting archive_period=60000 1574244570 [139950771140352] DEBUG sys/tools/hdb_cm_1 after set_attribute_config Segmentation fault

Strange that it can't read AttributeList from ES server.

bourtemb commented 4 years ago

Thanks again for the report.... What did you do exactly? How did you try to add an attribute via the Configuration Manager? Are you talking about the Configuration Manager device when you write "Configuration Manager" or are you talking about the Configuration Manager GUI?

It is strongly advised to use the HDB++ Configurator GUI (hdpp-configurator) to add attributes to be archived because this is very tricky otherwise. If you really need to do it by code, I would suggest to use the HdbConfigurator Java Device Server for this purpose because it will simplify the process of adding attributes to be archived.

Diego91RA commented 4 years ago

Sorry for confusing you, of course I used hdbpp-configurator GUI app, cloned from github.

And when I click "Subscribe", hdbpp-cm crashes with "Segmentation fault" error.

bourtemb commented 4 years ago

Strange that it can't read AttributeList from ES server.

You wrote this because of this log message I guess?:

1574244570 [139950771140352] INFO sys/tools/hdb_cm_1 find_archiver: unable to read AttributeList from tango://mpd-ecal-sc-1.he.jinr.ru:10000/sys/tools/hdb_es_1

I would not worry about that message. You get it simply because tango://mpd-ecal-sc-1.he.jinr.ru:10000/sys/tools/hdb_es_1 AttributeList was empty when you tried to add a new attribute (it is the first attribute you're trying to add to this archiver). This is not an error.

Could you please provide the stack trace from gdb when the crash occurs? Or do you have an easy way to reproduce it?

bourtemb commented 4 years ago

1574244570 [139950771140352] INFO sys/tools/hdb_cm_1 find_archiver: unable to read AttributeList from tango://mpd-ecal-sc-1.he.jinr.ru:10000/sys/tools/hdb_es_1

I would not worry about that message. You get it simply because tango://mpd-ecal-sc-1.he.jinr.ru:10000/sys/tools/hdb_es_1 AttributeList was empty when you tried to add a new attribute (it is the first attribute you're trying to add to this archiver). This is not an error.

Correction: I think this message appears when the configuration manager cannot contact the event subscriber (event subscriber not running or not responding).

Edit: I think this can appear when the attribute list is empty indeed.

Diego91RA commented 4 years ago

When the ES server is not running, configurator shows message

When I try to add new attribute I don't have it, so maybe the attr is just empty.

Diego91RA commented 4 years ago

gdb stack trace:

>Starting program: /home/tango/Downloads/hdbpp-cm/bin/hdb++cm-srv mpd-ecal-sc-1
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff6ca5700 (LWP 7966)]
[New Thread 0x7ffff6269700 (LWP 7967)]
[New Thread 0x7ffff5a68700 (LWP 7968)]
[New Thread 0x7fffed267700 (LWP 7969)]
[New Thread 0x7ffff5267700 (LWP 7970)]
[New Thread 0x7ffff4a66700 (LWP 7971)]
[New Thread 0x7fffeffff700 (LWP 7972)]
Starting version: 1.0.0:2019-11-20 14:36:37
[New Thread 0x7fffee9d0700 (LWP 7973)]
[New Thread 0x7fffee1cf700 (LWP 7974)]
[2019-11-21 16:52:01.922] [hdbpp] [debug] Logger set to level 1
[2019-11-21 16:52:01.922] [hdbpp] [info] Logging level: debug
[2019-11-21 16:52:01.922] [hdbpp] [info] Logging to file: false, logging to console: true
[2019-11-21 16:52:01.922] [hdbpp] [info] Logfile (if any): 
[2019-11-21 16:52:01.922] [hdbpp] [info] Starting libhdbpp-timescale shared library...
[2019-11-21 16:52:01.922] [hdbpp] [info] Mandatory config parameter connect_string: user=hdb_event_sub password=hdbpp host=localhost port=5432 dbname=hdb
[2019-11-21 16:52:01.922] [hdbpp] [info] Connecting to postgres database with string: "user=hdb_event_sub password=hdbpp host=localhost port=5432 dbname=hdb"
[2019-11-21 16:52:01.929] [hdbpp] [info] Connected to postgres successfully
[2019-11-21 16:52:01.929] [hdbpp] [info] Started libhdbpp-timescale shared library successfully
Ready to accept request
[New Thread 0x7fffeca66700 (LWP 7984)]
Thread 11 "hdb++cm-srv" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffeca66700 (LWP 7984)]
0x00007ffff41ce0a1 in void fmt::v5::internal::parse_format_string<false, char, fmt::v5::format_handler<fmt::v5::arg_formatter<fmt::v5::back_insert_range<fmt::v5::internal::basic_buffer<char> > >, char, fmt::v5::basic_format_context<std::back_insert_iterator<fmt::v5::internal::basic_buffer<char> >, char> >&>(fmt::v5::basic_string_view<char>, fmt::v5::format_handler<fmt::v5::arg_formatter<fmt::v5::back_insert_range<fmt::v5::internal::basic_buffer<char> > >, char, fmt::v5::basic_format_context<std::back_insert_iterator<fmt::v5::internal::basic_buffer<char> >, char> >&) () from /usr/local/lib/libhdb++timescale.so
(gdb) backtrace
#0  0x00007ffff41ce0a1 in void fmt::v5::internal::parse_format_string<false, char, fmt::v5::format_handler<fmt::v5::arg_formatter<fmt::v5::back_insert_range<fmt::v5::internal::basic_buffer<char> > >, char, fmt::v5::basic_format_context<std::back_insert_iterator<fmt::v5::internal::basic_buffer<char> >, char> >&>(fmt::v5::basic_string_view<char>, fmt::v5::format_handler<fmt::v5::arg_formatter<fmt::v5::back_insert_range<fmt::v5::internal::basic_buffer<char> > >, char, fmt::v5::basic_format_context<std::back_insert_iterator<fmt::v5::internal::basic_buffer<char> >, char> >&) () from /usr/local/lib/libhdb++timescale.so
#1  0x00007ffff41cf83d in fmt::v5::basic_format_context<std::back_insert_iterator<fmt::v5::internal::basic_buffer<char> >, char>::iterator fmt::v5::vformat_to<fmt::v5::arg_formatter<fmt::v5::back_insert_range<fmt::v5::internal::basic_buffer<char> > >, char, fmt::v5::basic_format_context<std::back_insert_iterator<fmt::v5::internal::basic_buffer<char> >, char> >(fmt::v5::arg_formatter<fmt::v5::back_insert_range<fmt::v5::internal::basic_buffer<char> > >::range, fmt::v5::basic_string_view<char>, fmt::v5::basic_format_args<fmt::v5::basic_format_context<std::back_insert_iterator<fmt::v5::internal::basic_buffer<char> >, char> >, fmt::v5::internal::locale_ref) () from /usr/local/lib/libhdb++timescale.so
#2  0x00007ffff41bc682 in void spdlog::details::fmt_helper::pad2<500ul>(int, fmt::v5::basic_memory_buffer<char, 500ul, std::allocator<char> >&) [clone .constprop.553] () from /usr/local/lib/libhdb++timescale.so
#3  0x0000000000000005 in ?? ()
#4  0x00007fffeca648a0 in ?? ()
#5  0x00007fffeca647a0 in ?? ()
#6  0x00007ffff41d6ee2 in hdbpp::HdbppTimescaleDb::configure_Attr(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int, int, int, unsigned int) () from /usr/local/lib/libhdb++timescale.so
#7  0x00007ffff749d3ea in HdbClient::configure_Attr(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int, int, int, unsigned int) () from /usr/local/lib/libhdb++.so.6
#8  0x000055555558cc8d in ?? ()
#9  0x00005555555727cd in ?? ()
#10 0x00007ffff7d0b9bd in Tango::DeviceClass::command_handler (this=0x555555614f70, device=device@entry=0x55555561be30, 
    command="AttributeAdd", in_any=...) at deviceclass.cpp:1197
#11 0x00007ffff7ce07ba in Tango::DeviceImpl::command_inout (this=0x55555561be30, in_cmd=0x7fffdc00cc80 "AttributeAdd", 
    in_any=...) at device.cpp:1537
#12 0x00007ffff7ce7118 in Tango::Device_2Impl::command_inout_2 (this=0x55555561be30, in_cmd=<optimized out>, in_data=..., 
    source=Tango::CACHE_DEV) at device_2.cpp:438
#13 0x00007ffff7cfaf25 in Tango::Device_4Impl::command_inout_4 (this=0x55555561be30, in_cmd=0x7fffdc00cc80 "AttributeAdd", 
    in_data=..., source=Tango::CACHE_DEV, cl_id=...) at device_4.cpp:473
#14 0x00007ffff7e76242 in _0RL_lcfn_6fe2f94a21a10053_a3000000 (cd=0x7fffeca659e0, svnt=<optimized out>) at tangoSK.cpp:5383
#15 0x00007ffff78ae489 in omniCallHandle::upcall(omniServant*, omniCallDescriptor&) () from /usr/local/lib/libomniORB4.so.2
#16 0x00007ffff7e84e5d in Tango::_impl_Device_4::_dispatch (this=<optimized out>, _handle=...) at tangoSK.cpp:5958
#17 0x00007ffff7e8543b in Tango::_impl_Device_5::_dispatch (this=<optimized out>, _handle=...) at tangoSK.cpp:7478
#18 0x00007ffff78a77e2 in omni::omniOrbPOA::dispatch(omniCallHandle&, omniLocalIdentity*) ()
   from /usr/local/lib/libomniORB4.so.2
#19 0x00007ffff78846b9 in omniLocalIdentity::dispatch(omniCallHandle&) () from /usr/local/lib/libomniORB4.so.2
--Type <RET> for more, q to quit, c to continue without paging--
#20 0x00007ffff78cd690 in omni::GIOP_S::handleRequest() () from /usr/local/lib/libomniORB4.so.2
#21 0x00007ffff78cdd88 in omni::GIOP_S::dispatcher() () from /usr/local/lib/libomniORB4.so.2
#22 0x00007ffff78ca635 in omni::giopWorker::execute() () from /usr/local/lib/libomniORB4.so.2
#23 0x00007ffff78776ae in omniAsyncWorker::real_run() () from /usr/local/lib/libomniORB4.so.2
#24 0x00007ffff78786bf in omniAsyncPoolServer::workerRun(omniAsyncWorker*) () from /usr/local/lib/libomniORB4.so.2
#25 0x00007ffff7877239 in omniAsyncWorker::mid_run() () from /usr/local/lib/libomniORB4.so.2
#26 0x00007ffff7e4d2da in Tango::create_PyPerThData (info=...) at utils.cpp:3255
#27 0x00007ffff787865e in omniAsyncWorker::run(void*) () from /usr/local/lib/libomniORB4.so.2
#28 0x00007ffff74aa0e8 in omni_thread_wrapper () from /usr/local/lib/libomnithread.so.4
#29 0x00007ffff6cbcfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
#30 0x00007ffff708e4cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

bourtemb commented 4 years ago

I managed to reproduce this with the VM you provided. The problem occurs only when libhdbpp-timescale is compiled in Release mode. In Debug Mode (if you comment out the lines described in https://github.com/tango-controls-hdbpp/libhdbpp-timescale/issues/4 to avoid a crash of the event subscriber), it seems to work fine. This doesn't ease the debugging process... :-(

I noticed you might get a similar problem as the one you originally reported in the following lines in libhdbpp-timescale too: https://github.com/tango-controls-hdbpp/libhdbpp-timescale/blob/1ac5bc999c0862ff7f7ca3b099d70b71595ba2d5/src/AttributeName.cpp#L108-L109

I suspect this issue is appearing when the host is configured to support IPv6... I think when removing the 2 last lines from /etc/hosts in your VM, this issue when getting the domain does not appear.

Diego91RA commented 4 years ago

Hello again.

I've installed Tango on Debian 9, disabled ipv6 and HDB++ works and archives data! Btw, I remember some problems with ipv6 even from Tango 8, so I think I will disable it on every tango host, just in case.

I've had some errors with adding privileges from users.sql, had to re-execute it a few times and check if it works. On the screenshot there is the output for privilege_type of att_conf table.

Changes from https://github.com/tango-controls-hdbpp/hdbpp-cm/issues/6#issuecomment-555878634 was necessary to compile the CM.

I think we'll stay on Debian 9, if there will be no other major problems with timescale-based hdb.

So the hdb works and I have a couple of questions: 1) Is there any drawn scheme or description how the hdb database is organized? It's a bit different from mysql and it could save me much time, e.g. how to find values for attributes by attr_name in different tables. 2) Is HDBViewer ready for timescaledb? Or how do you draw plots or export data now?

bourtemb commented 4 years ago

2. Is HDBViewer ready for timescaledb? Or how do you draw plots or export data now?

I can already reply to this question. Yes, the HDB Viewer is already ready for timescaledb. It is using https://github.com/tango-controls-hdbpp/libhdbpp-extraction-java underneath and this Java extraction library is now able to extract data from PostgreSQL database. Please refer to this documentation to understand how to configure it for PostgreSQL (timescaledb actually) (Have a look at the PostgreSQL specific environment variables): http://www.esrf.eu/computing/cs/tango/tango_doc/hdb_javadoc/index.html

Diego91RA commented 4 years ago

I'm not really familiar with Java, so I'm stuck again -_- I've installed libpostgresql-jdbc-java package and added /usr/share/java to the CLASSPATH, but still get "no driver" error. 2019-11-26_124558 Do I have to specify it somehow?

bourtemb commented 4 years ago

@JeanLucPons , Could you please help @Diego91RA to use the HDB Viewer with timescale?

bourtemb commented 4 years ago

I had a look at our script which starts up the HDB Viewer for timescale. The last line is like that:

$JAVA -Djdbc.drivers=org.postgresql.Driver -DTANGO_HOST=$TANGO_HOST $APPLI_PACKAGE.$APPLI_MAIN_CLASS $@

So it seems like the jdbc driver used is defined via this command line parameter when invoking java: -Djdbc.drivers=org.postgresql.Driver

Diego91RA commented 4 years ago

Yes, it helped. Thanks a lot, I hope it is all for now.

The only problem left is "Segmentation fault" on Debian 10, but it's not urgent, we can help with tests.

chedburgh commented 4 years ago

Closing, seems to be dealt with.

tango-controls-hdbpp / hdbpp-cm

TimescaleDB support #6

include <hdb++/HdbClient.h>

0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50

1 0x00007ffff6fb7535 in __GI_abort () at abort.c:79

2 0x00007ffff737f983 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6

3 0x00007ffff73858c6 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6

4 0x00007ffff7385901 in std::terminate() () from /lib/x86_64-linux-gnu/libstdc++.so.6

5 0x00007ffff7385b34 in __cxa_throw () from /lib/x86_64-linux-gnu/libstdc++.so.6

6 0x00007ffff73817d3 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6

7 0x000055555557f3e1 in void std::__cxx11::basic_string<char, std::char_traits, std::allocator >::_M_construct<char const>(char const, char const*, std::forward_iterator_tag) ()

8 0x0000555555581bb7 in ?? ()

9 0x0000555555582181 in ?? ()

10 0x0000555555582b48 in ?? ()

11 0x00005555555839ef in ?? ()

12 0x000055555557aaa3 in ?? ()

13 0x00007ffff7d24f0f in Tango::DServer::init_device (this=0x55555560c320) at /usr/include/c++/8/bits/stl_vector.h:930

14 0x00007ffff7d25f30 in Tango::DServer::DServer (this=0x55555560c320, cl_ptr=, n=,

15 0x00007ffff7d29b6c in Tango::DServerClass::device_factory (this=0x5555556096e0, devlist_ptr=0x7fffffffe260)

16 0x00007ffff7d2b5a7 in Tango::DServerClass::DServerClass (this=0x5555556096e0, s=...) at dserverclass.cpp:1574

17 0x00007ffff7d2b7e7 in Tango::DServerClass::init () at dserverclass.cpp:1623

18 0x00007ffff7e4d0d5 in Tango::Util::server_init (this=0x5555555f2720, with_window=) at utils.cpp:1857

19 0x0000555555571923 in ?? ()

20 0x00007ffff6fb909b in __libc_start_main (main=0x555555571910, argc=2, argv=0x7fffffffe448, init=,

21 0x0000555555571a1a in ?? ()