search

linz / QGIS-AIMS-Plugin

QGIS Plugin for managing LINZ address information
BSD 3-Clause "New" or "Revised" License
3 stars 4 forks source link

Address Points #228

Open TrentGulliver opened 5 years ago

TrentGulliver commented 5 years ago

As per the discussion between Simon, Joe and Trent on Friday 18 January, the Addressing Team have noticed that the Address Points are sometimes not loading up as expected. The points either take a long time to load, or they don't load at all. A work-around is to close out of the application and try again.

@SPlanzer, @jason-bedford can provide you with further detail if need be.

Thanks Trent.

SPlanzer commented 5 years ago

Thanks for opening this @TrentGulliver.

There have been no source code changes therefore we will have to investigate network and server performance.

We will comment once we have more. Cheers.

TrentGulliver commented 5 years ago

Great – thanks a lot!

From: Simon Planzer [mailto:notifications@github.com] Sent: Tuesday, 22 January 2019 10:48 a.m. To: linz/QGIS-AIMS-Plugin Cc: Trent Gulliver; Mention Subject: Re: [linz/QGIS-AIMS-Plugin] Address Points (#228)

Thanks for opening this @TrentGulliverhttps://github.com/TrentGulliver.

There have been no source code changes therefore we will have to investigate network and server performance.

We will comment once we have more. Cheers.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/linz/QGIS-AIMS-Plugin/issues/228#issuecomment-456207215, or mute the threadhttps://github.com/notifications/unsubscribe-auth/Al59ASCRORjZiJhzx_2AwoPJ-oUsvRr3ks5vFjV_gaJpZM4aLkPY.

This message contains information, which may be in confidence and may be subject to legal privilege. If you are not the intended recipient, you must not peruse, use, disseminate, distribute or copy this message. If you have received this message in error, please notify us immediately (Phone 0800 665 463 or info@linz.govt.nz) and destroy the original message. LINZ accepts no responsibility for changes to this email, or for any attachments, after its transmission from LINZ. Thank You.

josephramsay commented 5 years ago

@TrentGulliver Taking a look at the p1/p3 server cpu/mem/disk etc. Everything seems okay. Wondering if the issue might be backup related or just due to server load? Are you seeing this happen at any particular time? Only when n>X users are working on AIMS? edit: logs aren't showing anything extraordinary either. Pls flag to us next time you see problems occuring

jason-bedford commented 5 years ago

It was happening most days last week when we were firing up the plugin first thing in the morning. Closing QGIS and trying again straight away would work. Renni was the first person to open AIMS each morning and he wasn't having the problem, but the rest of us were.

josephramsay commented 5 years ago

@jason-bedford I'm seeing Rennie's (anti-social-o'clock in the morning) accesses. They all seem to be succeeding. The API hasn't logged any errors for those time periods... 

AAA.BBB.CCC.DDD - - [18/Jan/2019:06:53:03 +1300] "GET /aims/api/groups/resolutionfeed?count=1000&page=1 HTTP/1.1" 401 994
AAA.BBB.CCC.DDD - - [18/Jan/2019:06:53:03 +1300] "GET /aims/api/address/resolutionfeed?count=1000&page=1 HTTP/1.1" 401 994
AAA.BBB.CCC.DDD - rkautai [18/Jan/2019:06:53:03 +1300] "GET /aims/api/groups/resolutionfeed?count=1000&page=1 HTTP/1.1" 200 410
AAA.BBB.CCC.DDD - rkautai [18/Jan/2019:06:53:03 +1300] "GET /aims/api/groups/resolutionfeed/4355630 HTTP/1.1" 200 397
AAA.BBB.CCC.DDD - rkautai [18/Jan/2019:06:53:03 +1300] "GET /aims/api/address/resolutionfeed?count=1000&page=1 HTTP/1.1" 200 186
AAA.BBB.CCC.DDD - rkautai [18/Jan/2019:06:53:04 +1300] "GET /aims/api/groups/resolutionfeed/4355630/address?count=1000 HTTP/1.1" 200 15247
AAA.BBB.CCC.DDD - - [18/Jan/2019:06:53:13 +1300] "GET /aims/api/address/resolutionfeed?count=1000&page=1 HTTP/1.1" 401 994
AAA.BBB.CCC.DDD - rkautai [18/Jan/2019:06:53:13 +1300] "GET /aims/api/address/resolutionfeed?count=1000&page=1 HTTP/1.1" 200 186

AAA.BBB.CCC.DDD - - [17/Jan/2019:06:35:36 +1300] "GET /aims/api/address/resolutionfeed?count=1000&page=1 HTTP/1.1" 401 994
AAA.BBB.CCC.DDD - - [17/Jan/2019:06:35:36 +1300] "GET /aims/api/groups/resolutionfeed?count=1000&page=1 HTTP/1.1" 401 994
AAA.BBB.CCC.DDD - rkautai [17/Jan/2019:06:35:36 +1300] "GET /aims/api/groups/resolutionfeed?count=1000&page=1 HTTP/1.1" 200 410
AAA.BBB.CCC.DDD - rkautai [17/Jan/2019:06:35:36 +1300] "GET /aims/api/address/resolutionfeed?count=1000&page=1 HTTP/1.1" 200 761
AAA.BBB.CCC.DDD - rkautai [17/Jan/2019:06:35:37 +1300] "GET /aims/api/groups/resolutionfeed/4353211 HTTP/1.1" 200 396
AAA.BBB.CCC.DDD - rkautai [17/Jan/2019:06:35:37 +1300] "GET /aims/api/address/resolutionfeed/4353140 HTTP/1.1" 200 673
AAA.BBB.CCC.DDD - rkautai [17/Jan/2019:06:35:37 +1300] "GET /aims/api/groups/resolutionfeed/4353211/address?count=1000 HTTP/1.1" 200 8120

AAA.BBB.CCC.DDD - - [16/Jan/2019:06:34:00 +1300] "GET /aims/api/address/resolutionfeed?count=1000&page=1 HTTP/1.1" 401 994
AAA.BBB.CCC.DDD - - [16/Jan/2019:06:34:00 +1300] "GET /aims/api/groups/resolutionfeed?count=1000&page=1 HTTP/1.1" 401 994
AAA.BBB.CCC.DDD - rkautai [16/Jan/2019:06:34:00 +1300] "GET /aims/api/groups/resolutionfeed?count=1000&page=1 HTTP/1.1" 200 410
AAA.BBB.CCC.DDD - rkautai [16/Jan/2019:06:34:00 +1300] "GET /aims/api/address/resolutionfeed?count=1000&page=1 HTTP/1.1" 200 186
AAA.BBB.CCC.DDD - rkautai [16/Jan/2019:06:34:00 +1300] "GET /aims/api/groups/resolutionfeed/4351058 HTTP/1.1" 200 397
AAA.BBB.CCC.DDD - rkautai [16/Jan/2019:06:34:00 +1300] "GET /aims/api/groups/resolutionfeed/4351058/address?count=1000 HTTP/1.1" 200 7306
jason-bedford commented 5 years ago

Yeah, he's not having the issue. Have a look at mine or Richard's logs.

josephramsay commented 5 years ago

On the DB side am only seeing layer_styles errors

2019-01-17 06:35:35 NZDT:AAA.BBB.CCC.DDD(53591):RKautai@linz_db:[30384]:[QGIS]: ERROR:  relation "layer_styles" does not exist at character 22
2019-01-17 06:35:35 NZDT:AAA.BBB.CCC.DDD(53592):RKautai@linz_db:[30385]:[QGIS]: ERROR:  relation "layer_styles" does not exist at character 22
2019-01-17 06:35:35 NZDT:AAA.BBB.CCC.DDD(53593):RKautai@linz_db:[30386]:[QGIS]: ERROR:  relation "layer_styles" does not exist at character 22
2019-01-17 07:19:22 NZDT:AAA.BBB.CCC.DDD(54688):RKautai@linz_db:[36367]:[QGIS]: ERROR:  relation "layer_styles" does not exist at character 22
2019-01-17 07:19:23 NZDT:AAA.BBB.CCC.DDD(54689):RKautai@linz_db:[36368]:[QGIS]: ERROR:  relation "layer_styles" does not exist at character 22
2019-01-17 07:19:23 NZDT:AAA.BBB.CCC.DDD(54690):RKautai@linz_db:[36369]:[QGIS]: ERROR:  relation "layer_styles" does not exist at character 22

which are a known non-problem.

the only other ones for the 17th, not at daily startup include:

2019-01-17 09:07:48 NZDT:AAA.BBB.EEE.FFF(53942):gdbapp_api_server@linz_db:[51103]:[[unknown]]: ERROR:  more than one row returned by a subquery used as an expression
2019-01-17 10:58:35 NZDT:AAA.BBB.CCC.III(54136):YChen@linz_db:[2558]:[pgAdmin III - Query Tool]: ERROR:  relation "road_section" does not exist at character 15
2019-01-17 10:59:29 NZDT:AAA.BBB.CCC.III(54136):YChen@linz_db:[2558]:[pgAdmin III - Query Tool]: ERROR:  syntax error at or near "rs" at character 10
2019-01-17 10:59:42 NZDT:AAA.BBB.CCC.III(54136):YChen@linz_db:[2558]:[pgAdmin III - Query Tool]: ERROR:  column "road_secton_id" specified in USING clause does not exist in left table
2019-01-17 16:25:17 NZDT:AAA.BBB.CCC.JJJ(56444):YChen@linz_db:[50399]:[pgAdmin III - Query Tool]: ERROR:  relation "roads.road_name_associate" does not exist at character 45
2019-01-18 02:02:09 NZDT:AAA.BBB.GGG.HHH(47692):dab_user@linz_db:[63275]:[[unknown]]: ERROR:  column "ogc_fid" of relation "temp_meshblock_concordance" already exists
josephramsay commented 5 years ago

Okay, thanks @jason-bedford . For Richard, it looks like hes seeing network issues

2019-01-15 16:19:31 NZDT:AAA.BBB.CCC.KKK(61805):rfreeman@linz_db:[27218]:[[unknown]]: LOG:  could not receive data from client: Connection reset by peer
2019-01-15 16:19:31 NZDT:AAA.BBB.CCC.KKK(55824):rfreeman@linz_db:[28778]:[QGIS]: LOG:  could not receive data from client: Connection reset by peer

Connectivity errors for Rennie too. Can you confirm hes not seeing the problem pls

2019-01-17 14:57:04 NZDT:AAA.BBB.CCC.DDD(65493):RKautai@linz_db:[38615]:[QGIS]: LOG:  could not receive data from client: Connection reset by peer
jason-bedford commented 5 years ago

I've just had this happen again. I fired up the AIMS plugin at 8:29 and no address points were visible. Closed and re-opened QGIS and restarted the AIMS plugin at 8:31 and the address points are back.

josephramsay commented 5 years ago

Thanks for the detailed report. I can see your access logs but unfortunately no errors or any other information.

AAA.BBB.CCC.DDD - jbedford [23/Jan/2019:08:29:41 +1300] "GET /aims/api/address/resolutionfeed?count=1000&page=1 HTTP/1.1" 200 186
AAA.BBB.CCC.DDD - jbedford [23/Jan/2019:08:29:41 +1300] "GET /aims/api/groups/resolutionfeed?count=1000&page=1 HTTP/1.1" 200 185
AAA.BBB.CCC.DDD - jbedford [23/Jan/2019:08:29:41 +1300] "GET /aims/api/groups/resolutionfeed?count=1000&page=1 HTTP/1.1" 200 185
AAA.BBB.CCC.DDD - jbedford [23/Jan/2019:08:31:51 +1300] "GET /aims/api/address/resolutionfeed?count=1000&page=1 HTTP/1.1" 200 186
AAA.BBB.CCC.DDD - jbedford [23/Jan/2019:08:31:51 +1300] "GET /aims/api/groups/resolutionfeed?count=1000&page=1 HTTP/1.1" 200 185

@SPlanzer says the aims plugin is generating a log file. Would you be able to send this to me please

jason-bedford commented 5 years ago

There's two log files which I'll email you @josephramsay, but the first entries in each were at 8:31 which was when I was successful at opening the plugin, so they may not be of much use. If I have the problem tomorrow I'll check the log files before re-starting the plugin.

josephramsay commented 5 years ago

Yep got it, thx. Not much use since its reset on each initialisation. Don't know if the log tomorrow will show much either but its interesting that it only happens at the first startup of the day ... some kind of windows authentication sync issue perhaps?

josephramsay commented 5 years ago 
2019-01-23 08:30:35 NZDT:AAA.BBB.CCC.DDD(60677):jbedford@linz_db:[8538]:[QGIS]: LOG:  could not receive data from client: Connection reset by peer

...above is the postgres connection failure which occurs after your first login attempt.

jason-bedford commented 5 years ago

I've had this happen again and I've saved a copy of the log files which I'll email to @josephramsay. Also noted that I am able to view the address points which are in the Review queue, but not the "live" addresses.

jason-bedford commented 5 years ago

@josephramsay looked at the log file and noticed the initial operation to refresh the address points within the saved bounding box is timing out. I amended the value "thread_join_timeout" in the aimsconfig.ini file from 1s to 5s, which lengthened the timeouts but didn't fix the issue.

josephramsay commented 5 years ago

Plan to add setBB specific logging and add tests to determine fault.

jason-bedford commented 5 years ago

A workaround for this issue is to load a different plugin, for example the QGIS IntraMaps Connector, before loading the AIMS plugin. Since loading the plugins in this order I haven't been having the issue with the addresses not displaying.